Top > IJAT > Proposal of Simulation-Based Surgical Navigation and Developme ...

single-au.php

IJAT Vol.17 No.3 pp. 262-276
doi: 10.20965/ijat.2023.p0262
(2023)

Research Paper:

Views over last 60 days: 245

Proposal of Simulation-Based Surgical Navigation and Development of Laparoscopic Surgical Simulator that Reflects Motion of Surgical Instruments in Real-World

Sayaka Shibuya*1, Noriyuki Shido*1, Ryosuke Shirai*1, Kazuya Sase*2 ORCID Icon, Koki Ebina*1 ORCID Icon, Xiaoshuai Chen*3 ORCID Icon, Teppei Tsujita*4 ORCID Icon, Shunsuke Komizunai*1 ORCID Icon, Taku Senoo*1 ORCID Icon, and Atsushi Konno*1,† ORCID Icon

*1Graduate School of Information Science and Technology, Hokkaido University
Kita 14, Nishi 9, Kita-ku, Sapporo, Hokkaido 060-0814, Japan

†Corresponding author

*2Faculty of Engineering, Tohoku Gakuin University
Tagajo, Japan

*3Graduate School of Science and Technology, Hirosaki University
Hirosaki, Japan

*4Department of Mechanical Engineering, National Defense Academy of Japan
Yokosuka, Japan

Received:
November 4, 2022
Accepted:
February 20, 2023
Published:
May 5, 2023
Creative Commons License
Keywords:
real-time digital-twin, laparoscopic surgery, finite element method, collision detection, surgical navigation
Abstract

This study proposes simulation-based surgical navigation concept and describes the development of a laparoscopic surgical simulator that reflects the motion of surgical instruments in the real world. In the proposed simulation-based surgical navigation, movements of the surgical instruments are captured by a motion capture system, and the movements of the real surgical instruments are reflected in the movements of the virtual instruments in the simulation in real time. Contact of the virtual surgical instruments with organ model is detected based on the signed distance field (SDF) made around the organ model. The deformations of organs caused by contacts are calculated using dynamic finite element method (FEM). Using a cubic elastic object made of urethane resin, the accuracy of the calculation of the deformation was verified. The average error in the deformation verification experiments was within 1 mm. Simulations using hepato-biliary-pancreatic finite element (FE) models were performed, and computational costs of the simulation were validated. The time for one loop simulation with a hepato-biliary-pancreatic FE model of 3,225 elements and 1,663 nodes was 50 ms. The developed simulator can be applied to a simulation-based navigation system to update the states of organs in real time.

Cite this article as:
S. Shibuya, N. Shido, R. Shirai, K. Sase, K. Ebina, X. Chen, T. Tsujita, S. Komizunai, T. Senoo, and A. Konno, “Proposal of Simulation-Based Surgical Navigation and Development of Laparoscopic Surgical Simulator that Reflects Motion of Surgical Instruments in Real-World,” Int. J. Automation Technol., Vol.17 No.3, pp. 262-276, 2023.
Data files:
References
  1. [1] G. W. Yu and H. C. Miller, “Critical Operative Maneuvers in Urologic Surgery,” Mosby-Year Book, Inc., 1996.
  2. [2] C. Esposito, A. Settimi, F. Del Conte, M. Cerulo, V. Coppola, A. Farina, F. Crocetto, E. Ricciardi, G. Esposito, and M. Escolino, “Image-Guided Pediatric Surgery Using Indocyanine Green (ICG) Fluorescence in Laparoscopic and Robotic Surgery,” Frontiers in Pediatrics, Vol.8, 314, 2020.
  3. [3] M. Franz, J. Arend, S. Wolff, A. Perrakis, M. Rahimli, V.-R. Negrini, J. Stockheim, E. Lorenz, and R. Croner, “Tumor Visualization and Fluorescence Angiography with Indocyanine Green (ICG) in Laparoscopic and Robotic Hepatobiliary Surgery – Valuation of Early Adopters from Germany,” Innovative Surgical Sciences, Vol.6, No.2, pp. 59-66, 2021.
  4. [4] B. S. Peters, P. R. Armijo, C. Krause, S. A. Choudhury, and D. Oleynikov, “Review of Emerging Surgical Robotic Technology,” Surgical Endoscopy, Vol.32, No.4, pp. 1636-1655, 2018.
  5. [5] A. Radtke, G. C. Sotiropoulos, E. P. Molmenti, T. Schroeder, H. O. Peitgen, A. Frilling, D. C. Broering, C. E. Broelsch, and M. Malago, “Computer-Assisted Surgery Planning for Complex Liver Resections: When Is It Helpful? A Single-Center Experience Over an 8-Year Period,” Annals of Surgery, Vol.252, No.5, pp. 876-883, 2010.
  6. [6] P. Pessaux, M. Diana, L. Soler, T. Piardi, D. Mutter, and J. Marescaux, “Towards Cybernetic Surgery: Robotic and Augmented Reality-Assisted Liver Segmentectomy,” Langenbeck’s Archives of Surgery, Vol.400, No.3, pp. 381-385, 2015.
  7. [7] R. M. Viglialoro, N. Esposito, S. Condino, F. Cutolo, S. Guadagni, M. Gesi, M. Ferrari, and V. Ferrari, “Augmented Reality to Improve Surgical Simulation: Lessons Learned Towards the Design of a Hybrid Laparoscopic Simulator for Cholecystectomy,” IEEE Trans. on Biomedical Engineering, Vol.66, No.7, pp. 2091-2104, 2019.
  8. [8] J. Marescaux, F. Rubino, M. Arenas, D. Mutter, and L. Soler, “Augmented-Reality-Assisted Laparoscopic Adrenalectomy,” JAMA, Vol.292, No.18, pp. 2211-2215, 2004.
  9. [9] R. Tang, L.-F. Ma, Z.-X. Rong, M.-D. Li, J.-P. Zeng, X.-D. Wang, H.-E. Liao, and J.-H. Dong, “Augmented Reality Technology for Preoperative Planning and Intraoperative Navigation During Hepatobiliary Surgery: A Review of Current Methods,” Hepatobiliary & Pancreatic Diseases Int., Vol.17, No.2, pp. 101-112, 2018.
  10. [10] S. Nicolau, L. Soler, D. Mutter, and J. Marescaux, “Augmented Reality in Laparoscopic Surgical Oncology,” Surgical Oncology, Vol.20, No.3, pp. 189-201, 2011.
  11. [11] H. G. Kenngott, M. Wagner, M. Gondan, F. Nickel, M. Nolden, A. Fetzer, J. Weitz, L. Fischer, S. Speidel, H.-P. Meinzer, D. Böckler, M. W. Büchler, and B. P. Müller-Stich, “Real-Time Image Guidance in Laparoscopic Liver Surgery: First Clinical Experience with a Guidance System Based on Intraoperative CT Imaging,” Surgical Endoscopy, Vol.28, No.3, pp. 933-940, 2014.
  12. [12] W. H. Nam, D.-G. Kang, D. Lee, J. Y. Lee, and J. B. Ra, “Automatic Registration Between 3D Intra-Operative Ultrasound and Pre-Operative CT Images of the Liver Based on Robust Edge Matching,” Physics in Medicine & Biology, Vol.57, No.1, pp. 69-92, 2011.
  13. [13] X. Chen, R. Shirai, K. Masamune, M. Tamura, Y. Muragaki, K. Sase, T. Tsujita, and A. Konno, “Numerical Calculation Method for Brain Shift Based on Hydrostatics and Dynamic FEM,” IEEE Trans. on Medical Robotics and Bionics, Vol.4, No.2, pp. 368-380, 2022.
  14. [14] A. Ema, X. Chen, K. Sase, T. Tsujita, and A. Konno, “Moving Particle Semi-Implicit and Finite Element Method Coupled Analysis for Brain Shift Estimation,” J. Robot. Mechatron., Vol.34, No.6, pp. 1306-1317, 2022.
  15. [15] G. Székely, C. Brechbühler, R. Hutter, A. Rhomberg, N. Ironmonger, and P. Schmid, “Modelling of Soft Tissue Deformation for Laparoscopic Surgery Simulation,” Medical Image Analysis, Vol.4, No.1, pp. 57-66, 2000.
  16. [16] Surgical Science, “Lap Mentor.” https://simbionix.com/simulators/lap-mentor/ [Accessed October 1, 2022]
  17. [17] Surgical Science. https://surgicalscience.com/ [Accessed October 1, 2022]
  18. [18] K. Ebina, T. Abe, S. Komizunai, T. Tsujita, K. Sase, X. Chen, M. Higuchi, J. Furumido, N. Iwahara, Y. Kurashima, N. Shinohara, and A. Konno, “Development and Validation of a Measurement System for Laparoscopic Surgical Procedures,” SICE J. of Control, Measurement, and System Integration, Vol.13, No.4, pp. 191-200, 2020.
  19. [19] K. Ebina, T. Abe, K. Hotta, M. Higuchi, J. Furumido, N. Iwahara, M. Kon, S. Komizunai, Y. Kurashima, H. Kikuchi, R. Matsumoto, T. Osawa, S. Murai, T. Tsujita, K. Sase, X. Chen, N. Shinohara, and A. Konno, “Development and Validation of a Measurement System for Laparoscopic Surgical Procedures in Practical Surgery Training,” 2023 IEEE/SICE Int. Symp. on System Integration (SII), 2023. https://doi.org/10.1109/SII55687.2023.10039340
  20. [20] A. Fedorov, R. Beichel, J. Kalpathy-Cramer, J. Finet, J.-C. Fillion-Robin, S. Pujol, C. Bauer, D. Jennings, F. Fennessy, M. Sonka, J. Buatti, S. Aylward, J. V. Miller, S. Pieper, and R. Kikinis, “3D Slicer as an Image Computing Platform for the Quantitative Imaging Network,” Magnetic Resonance Imaging, Vol.30, No.9, pp. 1323-1341, 2012.
  21. [21] M. Muller, M. Teschner, and M. Gross, “Physically-Based Simulation of Objects Represented by Surface Meshes,” Proc. Computer Graphics Int. 2004, pp. 26-33, 2004.
  22. [22] Y. Masutani, Y. Inoue, K. Ishii, N. Kumai, F. Kimura, and I. Sakuma, “Development of Surgical Simulator Based on FEM and Deformable Volume-Rendering,” Medical Imaging 2004: Visualization, Image-Guided Procedures, and Display (Proc. Vol.5367), pp. 500-507, 2004.
  23. [23] K. Sase, T. Tsujita, and A. Konno, “Embedding Segmented Volume in Finite Element Mesh with Topology Preservation,” Proc. of the 19th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2016), Part 3, pp. 116-123, 2016.
  24. [24] K. Sase, T. Tsujita, and A. Konno, “Haptic Interaction with Segmented Medical Image Embedded in Finite Element Mesh,” J. of Japan Society of Computer Aided Surgery, Vol.19, No.2, pp. 89-99, 2017.
  25. [25] M. Nesme, P. G. Kry, L. Jeřábková, and F. Faure, “Preserving Topology and Elasticity for Embedded Deformable Models,” ACM Trans. on Graphics, Vol.28, No.3, 52, 2009.
  26. [26] J. Wu, C. Dick, and R. Westermann, “Efficient Collision Detection for Composite Finite Element Simulation of Cuts in Deformable Bodies,” The Visual Computer, Vol.29, Nos.6-8, pp. 739-749, 2013.
  27. [27] X. Chen, K. Sase, A. Konno, T. Tsujita, and S. Komizunai, “A Simple Damage and Fracture Model of Brain Parenchyma for Haptic Brain Surgery Simulations,” J. of Biomechanical Science and Engineering, Vol.11, No.4, 16-00323, 2016.
  28. [28] M. Müller, J. Dorsey, L. McMillan, R. Jagnow, and B. Cutler, “Stable Real-Time Deformations,” Proc. of the 2002 ACM SIGGRAPH/Eurographics Symp. on Computer Animation (SCA’02), pp. 49-54, 2002.
  29. [29] K. Sase, A. Fukuhara, T. Tsujita, and A. Konno, “GPU-Accelerated Surgery Simulation for Opening a Brain Fissure,” ROBOMECH J., Vol.2, No.1, 17, 2015.
  30. [30] Intel, “Depth Resolution of Intel® RealSense™ Depth Camera D435 and Intel® RealSense™ Camera SR300.” https://www.intel.com/content/www/us/en/support/articles/000026260/emerging-technologies/intel-realsense-technology.html?wapkw=realsense%20d435 [Accessed January 28, 2023]
  31. [31] K. Ebina, T. Abe, M. Higuchi, J. Furumido, N. Iwahara, M. Kon, K. Hotta, S. Komizunai, Y. Kurashima, H. Kikuchi, R. Matsumoto, T. Osawa, S. Murai, T. Tsujita, K. Sase, X. Chen, A. Konno, and N. Shinohara, “Motion Analysis for Better Understanding of Psychomotor Skills in Laparoscopy: Objective Assessment-Based Simulation Training Using Animal Organs,” Surgical Endoscopy, Vol.35, No.8, pp. 4399-4416, 2021.
  32. [32] K. Ebina, T. Abe, K. Hotta, M. Higuchi, J. Furumido, N. Iwahara, M. Kon, K. Miyaji, S. Shibuya, L. Yan, S. Komizunai, Y. Kurashima, H. Kikuchi, R. Matsumoto, T. Osawa, S. Murai, T. Tsujita, K. Sase, X. Chen, A. Konno, and N. Shinohara, “Objective Evaluation of Laparoscopic Surgical Skills in Wet Lab Training Based on Motion Analysis and Machine Learning,” Langenbeck’s Archives of Surgery, Vol.407, No.5, pp. 2123-2132, 2022.
  33. [33] Open Anatomy Project, “SPL Abdominal Atlas.” https://www.openanatomy.org/atlas-pages/atlas-spl-abdomen.html [Accessed October 1, 2022]
  34. [34] Plastimatch. http://plastimatch.org/ [Accessed October 1, 2022]

Creative Commons License  This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Apr. 22, 2024