NAT-3D: A Non-Rigid Approach for Accurate 3D Tracking of Basic Surgical Instruments

3D tracking technology serves as an essential enabler in numerous domains, most notably within digital healthcare, where it supports critical procedures such as surgical navigation, trajectory planning, and high-fidelity surgical simulation, etc. However, due to the slender, textureless, and reflective characteristics of basic surgical instruments, coupled with the complexity of their diverse motion patterns, existing systems and methods face significant limitations in achieving accurate 3D tracking of these instruments. We propose NAT-3D, an effective and efficient non-rigid 3D tracking approach based on multi-modal region estimation, integrating kinematic structural constraints and nonrigid constraints. This method enables accurate tracking and dynamic mapping of a variety of basic surgical instruments, including scalpels, scissors, clamps, and forceps, without the need for additional markers. The tracking covers various movement modes, including rigid motion, local mechanical motion, and non-rigid composite motion. Extensive experiments demonstrate that our method outperforms previous algorithms in terms of robustness, accuracy, applicability, and real-time performance. In addition, we introduce a novel dataset for Basic Surgical Instrument Tracking (BSIT), which can serve as a benchmark for future related research.