architecture

A Novel Multimodal Surgical Navigation System Assisted By Ultrasound Imaging and Optical Tracking for Spine Surgery

一种新型的超声成像和光学跟踪辅助多模态脊柱手术导航系统

Abstract:

3-D surgical navigation provides surgeons with precise intraoperative guidance, improves surgical outcomes, and reduces the risk of reoperation. Unlike traditional surgery, which relies heavily on the surgeon’s knowledge and experience, 3D surgical navigation brings patient medical imaging into the operating room, offering real-time anatomical information of the surgical site. However, existing surgical navigation systems typically rely on intraoperative computed tomography (CT) for image registration, which exposes patients to ionizing radiation, involves lengthy scanning times, and incurs high costs. To overcome these challenges, we developed a novel 3D multimodal surgical navigation system for spinal surgery. This system integrates ultrasound imaging for intraoperative 3D surface topography reconstruction, ultrasound-to-CT image registration, and optical tracking for real-time surgical instrument localization. As a result, the system provides intraoperative ultrasound imaging, dynamic surgical navigation based on preoperative CT images, and real-time surgical instrument tracking, delivering updated surgical navigation throughout the procedure. We evaluated the system using an agar-based spine phantom, and the experimental results demonstrate its potential to enhance the accuracy and efficacy of spinal procedures.

Published in: IEEE Open Journal of Instrumentation and Measurement ( Volume: 4)

Article Sequence Number: 4000509

Date of Publication: 19 September 2025

Electronic ISSN: 2768-7236

DOI: 10.1109/OJIM.2025.3605242

Funding Agency:

SECTION I.

Introduction

Traditionally, medical imaging was primarily used for diagnosis and surgical planning and was not integrated into the operating room, making surgical outcomes heavily reliant on the surgeon’s knowledge and experience. The introduction of surgical navigation systems has transformed this paradigm by providing accurate guidance through preoperative and intraoperative medical imaging and real-time tracking of surgical instruments [1], [2]. Currently, surgical navigation is widely employed across various surgical procedures, including neurosurgery [3], spinal surgery [4], [5], gynecology [6], and obstetrics, among others. This technology has been shown to improve resection rates, reduce complications, and lower mortality [7]. 3-D surgical navigation further enhances surgical execution by offering real-time guidance and visualizing the position of surgical instruments relative to the patient’s anatomy. It integrates multiple medical imaging modalities with image registration techniques, providing surgeons with comprehensive anatomical and functional information throughout the procedure.

Some surgical navigation systems rely solely on presurgically acquired data, limiting their ability to account for intraoperative changes such as brain shift, tissue deformation, and tissue removal during the procedure [8], [9]. Therefore, incorporating intraoperative imaging is essential for maintaining accuracy throughout surgery. Intraoperative ultrasound imaging (iUS) has become a widely used modality in modern surgical navigation [10], [11], [12], [13], [14]. Compared to intraoperative computed tomography (iCT) [15] and magnetic resonance imaging (iMRI) [16], iUS does not expose patients to additional radiation and is more cost-effective and easier to operate. Furthermore, unlike intraoperative optical topography imaging (iOTI), iUS provides detailed information on anatomical structures beneath the surface of the surgical site. Both 2-D and 3-D ultrasound imaging offer real-time, high-resolution images that are easily interpreted by surgeons [14]. Additionally, intraoperative ultrasound images can be supplemented by preoperative CT or MRI scans, enhancing anatomical accuracy during the surgical procedure.

To assist surgeons during procedures, providing anatomical information alone is insufficient; accurate tracking of surgical instruments is also essential. Typically, a reference image of the patient is obtained either preoperatively or intraoperatively. Through image registration, the system aligns the reference image with the patient’s real-world anatomy—a process that often involves placing reference markers on or within the patient at designated locations [17]. Sensors integrated into the surgical navigation platform track the position of surgical instruments relative to these markers. This real-time tracking enhances the surgeon’s spatial awareness of the instrument’s location and trajectory within the surgical site, facilitating more precise and accurate procedures. There are two widely used technologies for surgical instrument tracking: optical-based and electromagnetic-based systems [18]. Electromagnetic tracking is susceptible to distortion from nearby metal objects and generally exhibits lower precision. In contrast, optical tracking offers higher accuracy but is limited by the need for an unobstructed line of sight. Several commercial optical tracking systems are commonly used in surgical navigation, including the MicronTracker (Claron Technology Inc., Canada), the Polaris optical tracking system (Northern Digital Inc., Canada), and the StealthStation system (Medtronic Inc., United States). These systems typically utilize charge-coupled device (CCD) units as camera modules and employ near-infrared (NIR) LEDs or retro-reflective spheres as markers mounted on surgical instruments for accurate tracking.

Optical tracking enables the precise localization of the ultrasound scanner during surgery, facilitating the navigation of the ultrasound imaging plane within a 3-D space. A 3-D ultrasound volume can be reconstructed by stacking multiple navigated 2-D ultrasound images. This intraoperative 3-D volume can then be registered with preoperative CT or MRI images, providing a more accurate and comprehensive representation of the patient’s anatomical structures. By integrating preoperative CT or MRI with intraoperative ultrasound imaging, surgeons can perform procedures with improved accuracy and real-time guidance. In spinal procedures, intraoperative ultrasound imaging allows for continuous monitoring of the spinal cord while providing visualization of surrounding soft tissues and bony anatomy throughout the operation [19]. Preoperative CT or MRI images serve as a reference, enhancing the accuracy of anatomical localization and facilitating more precise surgical navigation.

In this article, we present a multimodal surgical navigation system that integrates intraoperative ultrasound imaging, preoperative CT imaging, and real-time surgical instrument tracking for spinal procedures. The system can reconstruct a 3-D volume from navigated ultrasound images. This intraoperative 3-D volume is registered with preoperative CT images to provide more accurate anatomical information. During spinal procedures, the optical tracking device localizes spinal anatomy and the ultrasound scanner, enabling real-time ultrasound navigation. We conducted experiments on a spine agar ultrasound phantom to evaluate the system’s performance. The results demonstrate the system’s potential to enhance spinal procedures. Additionally, we performed quantitative experiments to assess the accuracy of 3-D image registration. The findings indicate that our system provides precise and reliable navigation information to support surgeons during spinal operations.

SECTION II.

Materials and Methods

A. Multimodal Surgical Navigation Assisted by Optical Tracking

We have developed a Multimodal Surgical Navigation System that integrates optical tracking and ultrasound imaging. This system consists of an optical tracking camera and an ultrasound scanner. Fig. 1 illustrates the complete system.

FIGURE 1.

The multimodal surgical navigation system. The system comprises (1) an optical tracking camera, (2) an ultrasound scanner, (3) an ultrasound phantom, and (4) an iPad for real-time ultrasound image visualization.

The optical tracking camera (OptiTrack V120-TRIO, NaturalPoint Inc., OR, USA) comprises three sensors surrounded by LED rings that emit infrared (IR) light at a wavelength of 850 nm. To track surgical instruments, retro-reflective markers are fixed to the instruments. The IR light reflected from these markers is captured by the camera sensors, and the positions of these markers are determined using the principle of triangulation [20]. To enhance tracking precision, the sensors are equipped with 800 nm IR pass filters, which reduce interference from ambient light.

For accurate surgical instrument tracking, three retro-reflective markers are attached to each instrument. Suppose the initial positions of these three markers are defined as Pm1=(Xm1, Ym1, Zm1) , Pm2=(Xm2, Ym2, Zm2) , Pm3=(Xm3, Ym3, Zm3) . Given the 3-D CAD model of the surgical instrument, the initial positions of its vertices S are described by (1).

In 3-D space, each vertex of the 3-D CAD model can be represented as a weighted sum of three non-coplanar vectors OPm1−→−− , OPm2−→−− , and OPm3−→−− , where O represents the origin. Let Mm−s denote the matrix representing the coordinates of all vertices of the surgical instrument model relative to the basis formed by OPm1−→−− , OPm2−→−− , and OPm3−→−− . The relationship between matrix S , the representation matrix Mm−s , and the basic vectors can be described by (2) and (3).

S = S = M m - s = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ P s 1 P s 2 \dots P s i \dots P s n ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ X s 1 Y s 1 Z s 1 X s 2 Y s 2 Z s 2 \dots X s i Y s i Z s i \dots X s n Y s n Z s n ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ M m - s ⎡ ⎣ ⎢ ⎢ ⎢ O P m 1 - \to - - O P m 2 - \to - - O P m 3 ⎤ ⎦ ⎥ ⎥ ⎥ S ⎡ ⎣ ⎢ ⎢ ⎢ O P m 1 - \to - - O P m 2 - \to - - O P m 3 ⎤ ⎦ ⎥ ⎥ ⎥ - 1 . (1) (2) (3)

View Source

Using Mm−s , we can continuously track each vertex of the surgical instrument model as long as the positions of the three markers are known. Suppose the positions of the three markers at a given time τ are defined as: Pm1_τ=(Xm1_τ, Ym1_τ, Zm1_τ) , Pm2_τ=(Xm2_τ, Ym2_τ, Zm2_τ) , Pm3_τ=(Xm3_τ, Ym3_τ, Zm3_τ) . Given these positions, the location of the initial origin Oτ can be calculated using the geometric properties of the tetrahedron formed by the three markers and the origin. Consequently, the positions of the surgical instrument’s vertices at time τ , denoted as Sτ , can be computed using (4). This allows for the continuous tracking of both the positions and orientations of the surgical instrument. Fig. 2(a) illustrates the principle of surgical instrument tracking.

S τ = M m - s ⎡ ⎣ ⎢ ⎢ ⎢ O τ P m 1_τ - \to - - - - O τ P m 2_τ - \to - - - - O τ P m 3_τ ⎤ ⎦ ⎥ ⎥ ⎥ . (4)

View Source

FIGURE 2.

The ultrasound scanner and its tracking mechanism. (a) Principle of tracking the ultrasound scanner using retro-reflective markers affixed to a dedicated marker fixture. (b) The ultrasound scanner equipped with the marker fixture, with retro-reflective markers securely attached. (c) The CAD model of the ultrasound scanner.

To track the ultrasound scanner, we designed and 3-D-printed a marker fixture, which was attached to the end of the scanner. Retro-reflective markers were then fixed onto this fixture. With the known CAD models of both the ultrasound scanner and the marker fixture, the position and orientation of the ultrasound scanner can be accurately tracked using the method described above. Fig. 2(b) shows an image of the ultrasound scanner with the attached markers, Fig. 2(c) presents the tracked ultrasound scanner in our system’s software.

B. Navigated Intraoperative Ultrasound Imaging

The ultrasound scanner used in our system is the Clarius L15 HD3 (Clarius, Vancouver, Canada). This wireless scanner provides high-definition imaging of superficial structures, including nerves, vessels, musculoskeletal tissues, and lung anatomy, with a scanning depth of up to 7 cm. The ultrasound images are transmitted from the scanner to a mobile device (such as a smartphone or tablet) via Wi-Fi. In our setup, we used an iPad to display the ultrasound images captured by the scanner. After installing the dedicated Clarius application on the iPad, the ultrasound scanner can be connected through Wi-Fi, allowing real-time visualization of the ultrasound images. Clarius also provides an open-source C++ API for ultrasound image streaming. Once the connection between the ultrasound scanner and the iPad is established, the scanner’s IP address and port number can be retrieved, enabling ultrasound image casting to a C++ program running on a computer. In our system’s visualization interface, the ultrasound image plane is anchored at the end of the ultrasound scanner’s CAD model. Additionally, the ultrasound image is displayed in a picture-in-picture mode within the visualization window for enhanced accessibility. Fig. 3(a) shows a screenshot of the software interface, where the tracked ultrasound scanner, ultrasound image plane, and picture-in-picture display are presented. Fig. 3(b) displays the interface of the Clarius iOS application running on the iPad.

FIGURE 3.

The system software interface and ultrasound imaging application. (a) The custom system software interface, displaying the tracked ultrasound scanner with a fixed image plane at its distal end. The ultrasound image is overlaid in a picture-in-picture mode for enhanced visualization. (b) The Clarius iOS application interface, providing real-time ultrasound image visualization and interactive user control of the ultrasound scanner.

Using the navigated ultrasound images, we can reconstruct a 3-D volume of the scanned surgical site. Fig. 4 illustrates the principle for determining the 3-D coordinates of a given pixel. Assuming the ultrasound image has size H×W , the horizontal axis is denoted as u , and the vertical axis as v . For any pixel P on the 2-D image plane, its coordinates are represented as (uP0, vP0) . The ultrasound image plane is positioned at the end of the ultrasound scanner, with the center of the scanner’s end defined as C=(CX, CY, CZ) . The corresponding position of C on the 2-D image plane is ((W/2), 0) . The 3-D coordinates of C , can be continuously tracked in real time by the optical tracking camera. Consequently, the 3-D coordinates of any pixel P= (PX, PY, PZ) can be calculated using (5). Here, uP= uP0− (W/2) , vP= vP0 , CX−→−=(CXX, CXY, CXZ) represents the unit vector from C pointing in the negative u direction, and CY−→− = (CYX, CYY, CYZ) represents the unit vector from C pointing in the positive v direction. The variables u_scale and v_scale indicate the conversion factors that map pixel coordinates to spatial coordinates along the u and v axes, respectively. Using (6), the 3-D position of each pixel on the ultrasound image can be accurately determined.

FIGURE 4.

Principle of 3-D volume reconstruction from navigated ultrasound images. Given a pixel P on the ultrasound image plane, its 3-D coordinates can be determined by its 2-D coordinates, along with the position and orientation of the ultrasound scanner.

By collecting multiple navigated ultrasound images, we can reconstruct a 3-D volume of the scanned area using (6). However, to ensure accuracy, it is necessary to determine the values of u_scale and v_scale . To achieve this, we designed and 3-D-printed a fiducial object composed of several cuboids with known dimensions. This fiducial object was placed in a container to create an ultrasound phantom with agar. We then captured multiple ultrasound images and measured the length, width, and height of the cuboids. By comparing these measurements with their actual dimensions, we calculated the precise values of u_scale and v_scale .

View Source

C. Spine 3-D Volume Reconstruction and Registration with Preoperative CT Image

To evaluate the effectiveness of our surgical navigation system in spinal procedures, we created a spine ultrasound phantom using a lumbar spine model embedded in agar, as shown in Fig. 5(a). The spine model was scanned using a clinical CT scanner (SOMATOM Force, Siemens Healthcare, Germany) at X-ray voltage of 120 kV, current of 138 mA, exposure time of 1000 ms, and slice thickness of 3 mm. We collected navigated ultrasound images of the L1 vertebrae to reconstruct the 3-D volume of it. To speed the reconstruction process, we threshold the ultrasound image and only use pixels on the surface of the vertebrae to reconstruct its 3-D volume. Fig. 5(b) shows a sample captured ultrasound image, where red pixels denote the surface of the vertebrae and were used to reconstruct the 3-D volume, and Fig. 5(c) shows the reconstructed 3-D volume of the L1 vertebrae.

FIGURE 5.

3-D reconstruction of spine vertebrae from navigated ultrasound images. (a) The spine ultrasound phantom, created by submerging a lumbar spine model in agar. (b) A sample ultrasound image of the phantom, where the spine vertebrae boundaries (highlighted in red) are extracted for 3-D volume reconstruction. (c) The reconstructed 3-D volume of the spine vertebrae.

Although the reconstructed 3-D volume reflects the real-time position of the spine model, its can only provide partial anatomical information and the 3-D reconstruction process is time-consuming. To enhance the anatomical fidelity, we registered the reconstructed 3-D volume with the preoperative CT image of the lumbar spine phantom. The CT data were converted into a 3-D point cloud using MicroView 3-D Image Viewer (Parallax Innovations Inc., Canada) and MeshLab (Visual Computing Lab, Italy). For alignment, we first performed a manual coarse registration followed by fine-tuning using the Iterative Closest Point (ICP) algorithm. After completing the registration process, the preoperative CT image was overlaid onto the reconstructed 3-D volume for more comprehensive visualization.

A significant limitation of relying solely on point cloud registration is the need to reconstruct the 3-D volume and re-perform the registration each time the spine phantom is repositioned. To address this challenge, we integrated optical tracking for real-time spine localization. We designed and 3-D-printed a marker fixer and securely attached it to the spine phantom, retro-reflective markers were fixed onto the marker fixer. Once the initial registration between the reconstructed 3-D volume and the CT image was established, our system software recorded the initial positions of both the CT image and the markers. The optical tracking camera continuously monitors the markers’ positions, and by comparing their real-time positions with the recorded initial positions, we compute the rigid body transformation using the Singular Value Decomposition (SVD) method. This transformation is then applied to the CT image, enabling real-time tracking of the spine’s position.

By combining spine tracking with surgical instrument tracking, our system provides both the real-time position and anatomical context of the spine while accurately tracking surgical instruments. This multimodal integration enhances the precision of surgical procedures. Furthermore, navigated ultrasound imaging allows visualization of internal anatomical structures in 3-D space, offering surgeons critical intraoperative information.

D. Evaluation of Ultrasound-to-CT Registration Error

Accurate navigation information is crucial for surgical navigation, as even minor deviations can lead to severe consequences. To ensure the precision of our system, we conducted experiments to quantitatively evaluate the registration error between the reconstructed 3-D ultrasound volume and the preoperative CT image. We used an optical topography imaging camera to capture the surface topography of the surgical site as a reference. Fig. 6(a) is a photo of the optical topography imaging camera. Since the system’s primary coordinate frame is defined by the optical tracking camera, it is necessary to align the coordinate systems of the optical tracking camera and the optical topography imaging camera.

FIGURE 6.

Coordinate system calibration between the optical tracking camera and the optical topography imaging camera. (a) The optical topography imaging camera. (b) The fiducial model mounted with four retro-reflective markers used for coordinate calibration. (c) Overlaid point cloud visualization, showing the fiducial model’s surface point cloud aligned with the captured topography; white spheres represent the CAD models of the retro-reflective markers. (d) The principle of coordinate system calibration between two cameras.

To achieve this, we designed and 3-D-printed a fiducial model equipped with four retro-reflective markers, as shown in Fig. 6(b). During the calibration process, the fiducial model is placed under the optical topography imaging camera to capture its surface topography. This topography is then registered with the surface point cloud of the fiducial model, allowing us to determine the positions of the markers within the coordinate system of the optical topography imaging camera. Fig. 6(c) shows the overlaid point cloud of the fiducial model’s surface point cloud and the captured topography. Using these corresponding points, we calculate the transformation matrix [R|t] between the coordinate systems of the optical tracking camera and the optical topography imaging camera through SVD. Given a point Ptracking=(Xtracking, Ytracking, Ztracking) in the optical tracking camera coordinate system, its corresponding coordinates Ptopography=(Xtopography, Ytopography, Ztopography) in the optical topography imaging camera coordinate system can be computed using

⎡ ⎣ ⎢ X t o p o g r a p h y Y t o p o g r a p h y Z t o p o g r a p h y ⎤ ⎦ ⎥ = [R | t] ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ X t r a c k i n g Y t r a c k i n g Z t r a c k i n g 1 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ . (7)

View Source

To quantify the ultrasound-to-CT registration error, we designed and 3-D-printed a separate fiducial model and fixed it within a container. This model contains 12 identifiable fiducial points. We partially covered the fiducial object with agar, leaving the rest exposed. Using navigated ultrasound imaging, we reconstructed the 3-D volume of the submerged portion of the model and registered this volume with a 3-D point cloud generated from the model’s CAD data. Subsequently, we captured the surface topography of the exposed portion using the optical topography imaging camera. By comparing the positions of the fiducial markers on the captured surface topography with their corresponding positions on the model’s point cloud, we calculated the average distance between these points, which represents the registration error.

SECTION III.

Results

A. Ultrasound Imaging 3-D Volume Reconstruction Scaling Factor

In the process of reconstructing a 3-D spinal volume from navigated ultrasound images, it is essential to determine the scaling factors u_scal and v_scale , which convert pixel coordinates into spatial coordinates. To achieve this, we designed and 3-D-printed a fiducial object featuring two cuboids on its top surface, whose dimensions are precisely known. Fig. 7(a) presents a photograph of the fiducial object.

FIGURE 7.

Calculation of the scaling factor. (a) The fiducial model featuring two cuboids on the top. (b) Ultrasound image depicting the cross-section of the left cuboid. (c) Ultrasound image showing the cross-section of the right cuboid.

Subsequently, we embedded this fiducial object in agar to create an ultrasound phantom. Using the ultrasound scanner, we captured images of this phantom to obtain cross-sectional views of the cuboids. Fig. 7(b) and (c) display two sample ultrasound images depicting the cross-section of these cuboid. By comparing the measured dimensions of the cuboids in the ultrasound images with their actual dimensions, we determined the values of u_scale and v_scale . Equations (8) and (9) define the calculations for u_scale and v_scale , where Wobject and Hobject denote the actual width and height of the cuboid (in millimeters), and Wimage and Himage represent the width and height of the cuboid in the ultrasound image (in pixels).

To ensure accuracy, we captured 10 ultrasound images for each cuboid, obtaining 10 data points for both Wimage and Himage . The average values of these measurements were then calculated to determine Wimage and Himage for each cuboid. The average values of u_scale and v_scale , determined to be 0.083 mm/pixel and 0.077 mm/pixel , respectively, were subsequently used for the 3-D reconstruction of the spinal volume.

u_s c a l e = W o b j e c t W i m a g e v_s c a l e = H o b j e c t H i m a g e . (8) (9)

View Source

The scaling factors is critical for 3-D spinal volume reconstruction and they are dependent on the ultrasound probe setting. In real clinical applications, these factors should be determined and verified in surgical preparation stage.

B. Ultrasound Imaging Guided Spinal Procedure

We built a spine ultrasound phantom using agar. After acquiring ultrasound images of the L1 vertebra, we reconstructed its 3-D volume. Subsequently, we overlaid the preoperative CT image onto the reconstructed 3-D volume through a two-step process: manual registration followed by ICP registration. Following this step, we integrated optical tracking by attaching retro-reflective markers to the spine ultrasound phantom and utilizing the optical tracking camera to monitor these markers. This setup allows real-time tracking of the preoperative CT image, ensuring an accurate overlay. Furthermore, we incorporated navigated ultrasound imaging to provide real-time anatomical structural information of the spine.

One significant advantage of this approach is that it enables surgeons to visualize the scanned region of the spine. By combining complementary information from both imaging modalities, surgeons can perform procedures with enhanced accuracy and guidance. Fig. 8(a) depicts an individual holding the ultrasound scanner, illustrating the attachment of retro-reflective markers to both the scanner and the spine ultrasound phantom. Fig. 8(b) presents an individual holding a scalpel with retro-reflective markers fixed. On our system software visualization window, both the spine phantom and the surgical instruments are tracked. This provides accurate navigation information to surgeons. Our surgical navigation system achieves real-time delivery of CT volume overlay, tracking of ultrasound probe and surgical instruments, the end-to-end delay is less than 100 ms, which satisfies clinical requirements.

FIGURE 8.

Demonstration of the navigated spine procedure. (a) An operator holds the ultrasound scanner to acquire ultrasound images, with retro-reflective markers affixed to both the ultrasound scanner and the spine phantom, enabling the system to track the scanner and the corresponding spine CT image. (b) An operator holds a scalpel, which was fixed with markers. Both the scalpel and the spine are tracked by the system.

C. Assessment of Spine Point Cloud Registration Accuracy

In our surgical navigation system, accurate tracking of the preoperative CT image of the spine relies on point cloud registration to align the reconstructed 3-D volume with the preoperative CT scan. Therefore, it is crucial to evaluate the accuracy of point cloud registration. To quantitatively assess registration error, we designed and 3-D-printed a custom model.

Fig. 9(a) presents a photograph of this model, which consists of two parts: one part contains a spine vertebra model, while the other features a cuboid with raised fiducial points on its surface. The model was placed inside a container, where the vertebra portion was submerged in agar, leaving the cuboid section exposed. Fig. 9(b) shows the resulting ultrasound phantom. During the reconstruction process, we generated a 3-D volume of the submerged vertebra using navigated ultrasound images and registered this volume with the surface point cloud of the entire model. The exposed cuboid section contains 12 fiducial points, marked in white, which serve as reference landmarks. Additionally, we used an optical topography camera to capture the surface topography of the exposed portion of the model. By computing the distances between these fiducial points in the registered point cloud and the captured surface topography, we quantified the average point cloud registration error. In Fig. 9(c), we present the overlaid point cloud, where yellow dots indicate the fiducial points on the model point cloud, and white dots indicate the fiducial points on the captured surface topography.

FIGURE 9.

Quantitative analysis of ultrasound-to-CT registration error. (a) The fiducial model featuring 12 fiducial points on its top surface, highlighted in white. (b) The ultrasound phantom, created by partially submerging the fiducial model in agar. (c) Overlaid point cloud visualization, showing the alignment between the fiducial model’s surface point cloud and the captured surface topography. (d) Graph illustrating the distance between each fiducial point on the surface point cloud and the corresponding points on the captured topography, with the average distance computed as the registration error.

We measured the average point cloud registration error at a working distance of 60 cm. After registration, we manually selected the centers of the fiducial points to compute the registration error. The registration error for each fiducial markers are plotted Fig. 9(d), and the average registration error is 2.872 ± 0 .197 mm.

SECTION IV.

Discussion

By integrating optical-based surgical instrument tracking, navigated intraoperative ultrasound imaging, and CT-based navigation, our surgical navigation system provides surgeons with comprehensive navigation information. Utilizing optical tracking, the system reconstructs a 3-D volume of the scanned surgical site from navigated ultrasound images. To enhance anatomical accuracy, the preoperative CT image is overlaid using 3-D point cloud registration. With real-time optical tracking, the preoperative CT image can be dynamically updated and aligned, offering continuous and accurate anatomical guidance. The fusion of multiple imaging modalities mitigates the limitations inherent in each technique while leveraging their respective strengths, thereby enhancing surgical precision and effectiveness.

Surgical navigation systems that rely on intraoperative CT or MRI are often bulky, costly, and require complex setup procedures. Moreover, these systems expose both surgical teams and patients to additional ionizing radiation, increasing associated health risks. Intraoperative optical topography imaging is another widely used navigation technique. While it is radiation-free and easy to operate, its primary limitation is its inability to capture anatomical structures beneath the surface. To address this limitation, we employ intraoperative ultrasound imaging, which is cost-effective, portable, and radiation-free. Ultrasound imaging provides subsurface anatomical information, and when combined with optical tracking, our system can reconstruct a 3-D volume from navigated ultrasound images. Additionally, the integration of preoperative CT images provides complementary anatomical data. Through point cloud registration and optical tracking, the CT image can be accurately aligned and overlaid. In spinal procedures, preoperative CT data offers detailed anatomical information about the spinal structure, while ultrasound imaging aids in identifying soft tissues. By combining these modalities, surgeons receive enhanced guidance, leading to improved surgical outcomes.

To demonstrate the application of our system in spinal procedures, we conducted experiments using a spinal ultrasound phantom. During these experiments, we successfully captured the bony structure of the spine, reconstructed the 3-D volume of the vertebrae, and registered it with the corresponding CT image. The experimental results confirm that our system provides accurate anatomical information and improves surgical efficiency. Additionally, our software features a user-friendly graphical user interface (GUI), facilitating intuitive interaction between the surgical team and the navigation platform, thereby streamlining the surgical workflow.

Image registration plays a critical role in our system, as it serves as the foundation for delivering precise anatomical structural information. To investigate registration accuracy, we conducted experiments to quantitatively evaluate the alignment between the 3-D ultrasound volume and the preoperative CT image. The results demonstrate that our system maintains an average registration error of less than 3 mm. This capability ensures accurate surgical guidance, enhances surgical performance, and minimizes patient discomfort.

SECTION V.

Conclusion

In this paper, we presented a novel multimodal spinal surgical navigation system that integrates optical tracking and ultrasound imaging, enabling real-time surgical instrument tracking and navigated ultrasound imaging. To enhance anatomical accuracy, we developed a 3-D volume reconstruction algorithm based on navigated ultrasound images. This reconstructed 3-D volume is registered with preoperative CT images, allowing comprehensive visualization of patient anatomy. After registration, the preoperative CT image can be dynamically aligned using optical tracking, ensuring real-time updates during surgical procedures. To the best of our knowledge, this is the first system to combine surgical instrument tracking, navigated ultrasound imaging, 3-D volume reconstruction, and CT-based navigation in a unified platform for spinal surgery. We investigated the system’s performance through experiments on a spine ultrasound phantom, demonstrating its potential to enhance spinal procedures. The results indicate that our system provides precise, real-time alignment of preoperative CT images, intraoperative ultrasound data, and surgical instrument tracking. By combining these multimodal imaging techniques, our system offers improved surgical guidance, which can reduce operative time and minimize the risks associated with spinal surgery. Overall, our multimodal surgical navigation system represents a promising and adaptable platform that can improve surgical precision and patient outcomes. This paper aims to demonstrate the feasibility of our surgical navigation system on spinal procedures, and future work will focus on conducting experiments on biology tissues or cadavers, extending the system to other surgical applications and further optimizing its accuracy and usability.