Transferable situation recognition system for scenario-independent context-aware surgical assistance systems: a proof of concept

Katić D, Wekerle A-L, Görtler J, Spengler P, Bodenstedt S, Röhl S, Suwelack S, Kenngott HG, Wagner M, Müller-Stich BP, Dillmann R, Speidel S (2013) Context-aware augmented reality in laparoscopic surgery. Comput Med Imaging Graph 37:174–182. https://doi.org/10.1016/j.compmedimag.2013.03.003

Article  PubMed  Google Scholar 

Avrunin GS, Clarke LA, Conboy HM, Osterweil LJ, Dias RD, Yule SJ, Goldman JM, Zenati MA (2018) Toward improving surgical outcomes by incorporating cognitive load measurement into process-driven guidance. Softw Eng Healthc Syst SEHS IEEE ACM Int Workshop 2018:2–9. https://doi.org/10.1145/3194696.3194705

Jin Y, Li H, Dou Q, Chen H, Qin J, Fu C-W, Heng P-A (2020) Multi-task recurrent convolutional network with correlation loss for surgical video analysis. Med Image Anal 59:101572. https://doi.org/10.1016/j.media.2019.101572

Article  PubMed  Google Scholar 

Malpani A, Lea C, Chen CCG, Hager GD (2016) System events: readily accessible features for surgical phase detection. Int J Comput Assist Radiol Surg 11:1201–1209. https://doi.org/10.1007/s11548-016-1409-0

Article  PubMed  Google Scholar 

Meißner C, Meixensberger J, Pretschner A, Neumuth T (2014) Sensor-based surgical activity recognition in unconstrained environments. Minim Invasive Ther Allied Technol 23:198–205. https://doi.org/10.3109/13645706.2013.878363

Article  PubMed  Google Scholar 

Junger D, Frommer SM, Burgert O (2022) State-of-the-art of situation recognition systems for intraoperative procedures. Med Biol Eng Comput 60:921–939. https://doi.org/10.1007/s11517-022-02520-4

Article  CAS  PubMed  PubMed Central  Google Scholar 

Junger D, Hirt B, Burgert O (2022) Concept and basic framework prototype for a flexible and intervention-independent situation recognition system in the OR. Comput Methods Biomech Biomed Eng: Imag Vis 10:283–288. https://doi.org/10.1080/21681163.2021.2004446

Article  Google Scholar 

Junger D, Beyersdorffer P, Kücherer C, Burgert O (2022) Service-oriented Device Connectivity interface for a situation recognition system in the OR. Int J Comput Assist Radiol Surg 17:2161–2171. https://doi.org/10.1007/s11548-022-02666-4

Article  PubMed  PubMed Central  Google Scholar 

Junger D, Just E, Brandenburg JM, Wagner M, Schaumann K, Klenzner T, Burgert O (2024) Toward an interoperable, intraoperative situation recognition system via process modeling, execution, and control using the standards BPMN and CMMN. Int J Comput Assist Radiol Surg 19:69–82. https://doi.org/10.1007/s11548-023-03004-y

Article  PubMed  Google Scholar 

ISO/IEC (2011) ISO/IEC 25010:2011 Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models

ISO/IEC/IEEE (2018) ISO/IEC/IEEE 29148:2018 Systems and software engineering - Life cycle processes - Requirements engineering

Brandenburg J, Junger D, Schulze A, Pieumi Nzeuhang MA, Just E, Billeter A, Bodenstedt S, Speidel S, Burgert O, Wagner M, Müller-Stich B (2022) Interaktive Workflow Analyse am Beispiel der roboter-assistierten Ösophagusresektion: Ein Checklisten-Tool für Chirurgen im Operationssaal. 20. Jahrestagung der Deutschen Gesellschaft für Computer- und Roboterassistierte Chirurgie e. V., 16.-19. Sept 2021. https://doi.org/10.34645/opus-3188

Ryniak C, Frommer SM, Junger D, Lohmann S, Stadelmaier M, Schmutz P, Stenzl A, Hirt B, Burgert O (2023) A high-fidelity prototype of a sterile information system for the perioperative area: OR-Pad. Int J Comput Assist Radiol Surg 18:575–585. https://doi.org/10.1007/s11548-022-02787-w

Article  CAS  PubMed  Google Scholar 

Berlet M, Vogel T, Ostler D, Czempiel T, Kähler M, Brunner S, Feussner H, Wilhelm D, Kranzfelder M (2022) Surgical reporting for laparoscopic cholecystectomy based on phase annotation by a convolutional neural network (CNN) and the phenomenon of phase flickering: a proof of concept. Int J Comput Assist Radiol Surg 17:1991–1999. https://doi.org/10.1007/s11548-022-02680-6

Article  CAS  PubMed  PubMed Central  Google Scholar 

Nwoye CI, Yu T, Gonzalez C, Seeliger B, Mascagni P, Mutter D, Marescaux J, Padoy N (2022) Rendezvous: attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Med Image Anal 78:102433. https://doi.org/10.1016/j.media.2022.102433

Article  PubMed  Google Scholar 

Wagner M, Müller B, Kenngott HG (2018) Verbund: InnOPlan - Innovative, datengetriebene Effizienz OP-übergreifender Prozesslandschaften, Thema: Modellierung und Analyse perioperativer chirurgischer Prozesse als Grundlage für Smart-Data-getriebene Assistenzsysteme in OP-Management und OP-Dokumentation : Schlussbericht. Verbund: InnOPlan - Innovative, datengetriebene Effizienz OP-übergreifender Prozesslandschaften, Thema: Modellierung und Analyse perioperativer chirurgischer Prozesse als Grundlage für Smart-Data-getriebene Assistenzsysteme in OP-Management und OP-Dokumentation. https://doi.org/10.2314/GBV:1663370621

Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N (2017) EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36:86–97. https://doi.org/10.1109/TMI.2016.2593957

Article  PubMed  Google Scholar 

Just E, Schaumann K, Junger D, Ryniak C, Klenzner T, Burgert O (2021) Towards Automated Surgical Documentation using automatically generated checklists from BPMN models. Curr Direct Biomed Eng 7:135–139. https://doi.org/10.1515/cdbme-2021-1029

Article  Google Scholar 

Camunda (2021) BPMN Workflow Engine. https://camunda.com/platform-7/workflow-engine/. Accessed 09 Jan 2024

Camunda Services GmbH (2021) Rest Api Reference: Camunda Docs. https://docs.camunda.org/manual/7.9/reference/rest/. Accessed 09 Jan 2024

Franke S, Rockstroh M, Hofer M, Neumuth T (2018) The intelligent OR: design and validation of a context-aware surgical working environment. Int J Comput Assist Radiol Surg 13:1301–1308. https://doi.org/10.1007/s11548-018-1791-x

Article  PubMed  Google Scholar 

Dergachyova O, Bouget D, Huaulmé A, Morandi X, Jannin P (2016) Automatic data-driven real-time segmentation and recognition of surgical workflow. Int J Comput Assist Radiol Surg 11:1081–1089. https://doi.org/10.1007/s11548-016-1371-x

Article  PubMed  Google Scholar 

Quellec G, Charrière K, Lamard M, Droueche Z, Roux C, Cochener B, Cazuguel G (2014) Real-time recognition of surgical tasks in eye surgery videos. Med Image Anal 18:579–590. https://doi.org/10.1016/j.media.2014.02.007

Article  PubMed  Google Scholar 

Katić D, Wekerle A-L, Gärtner F, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2014) Knowledge-Driven Formalization of Laparoscopic Surgeries for Rule-Based Intraoperative Context-Aware Assistance. In: Hutchison D, Kanade T, Kittler J, Kleinberg JM, Kobsa A, Mattern F, Mitchell JC, Naor M, Nierstrasz O, Pandu Rangan C, Steffen B, Terzopoulos D, Tygar D, Weikum G, Stoyanov D, Collins DL, Sakuma I, Abolmaesumi P, Jannin P (eds) Information Processing in Computer-Assisted Interventions, vol 8498. Springer International Publishing, Cham, pp 158–167. https://doi.org/10.1007/978-3-319-07521-1_17

Wohlin C, Höst M, Henningsson K (2006) Empirical research methods in web and software engineering. In: Mendes E, Mosley N (eds) Web Engineering. Springer-Verlag, Berlin/Heidelberg, pp 409–430. https://doi.org/10.1007/3-540-28218-1_13

Demir KC, Schieber H, Weise T, Roth D, May M, Maier A, Yang SH (2023) Deep learning in surgical workflow analysis: a review of phase and step recognition. IEEE J Biomed Health Inform 27:5405–5417. https://doi.org/10.1109/JBHI.2023.3311628

Article  PubMed  Google Scholar 

Maier-Hein L, Eisenmann M, Sarikaya D, März K, Collins T, Malpani A, Fallert J, Feussner H, Giannarou S, Mascagni P, Nakawala H, Park A, Pugh C, Stoyanov D, Vedula SS, Cleary K, Fichtinger G, Forestier G, Gibaud B, Grantcharov T, Hashizume M, Heckmann-Nötzel D, Kenngott HG, Kikinis R, Mündermann L, Navab N, Onogur S, Roß T, Sznitman R, Taylor RH, Tizabi MD, Wagner M, Hager GD, Neumuth T, Padoy N, Collins J, Gockel I, Goedeke J, Hashimoto DA, Joyeux L, Lam K, Leff DR, Madani A, Marcus HJ, Meireles O, Seitel A, Teber D, Ückert F, Müller-Stich BP, Jannin P, Speidel S (2022) Surgical data science - from concepts toward clinical translation. Med Image Anal 76:102306. https://doi.org/10.1016/j.media.2021.102306

Article  PubMed  Google Scholar 

Nyangoh Timoh K, Huaulme A, Cleary K, Zaheer MA, Lavoué V, Donoho D, Jannin P (2023) A systematic review of annotation for surgical process model analysis in minimally invasive surgery based on video. Surg Endosc 37:4298–4314. https://doi.org/10.1007/s00464-023-10041-w

Article  PubMed  Google Scholar 

Wagner M, Brandenburg JM, Bodenstedt S, Schulze A, Jenke AC, Stern A, Daum MTJ, Mündermann L, Kolbinger FR, Bhasker N, Schneider G, Krause-Jüttler G, Alwanni H, Fritz-Kebede F, Burgert O, Wilhelm D, Fallert J, Nickel F, Maier-Hein L, Dugas M, Distler M, Weitz J, Müller-Stich B-P, Speidel S (2022) Surgomics: personalized prediction of morbidity, mortality and long-term outcome in surgery using machine learning on multimodal data. Surg Endosc 36:8568–8591. https://doi.org/10.1007/s00464-022-09611-1

Article  PubMed  PubMed Central  Google Scholar 

Hutchinson K, Reyes I, Li Z, Alemzadeh H (2023) COMPASS: a formal framework and aggregate dataset for generalized surgical procedure modeling. Int J Comput Assist Radiol Surg 18:2143–2154. https://doi.org/10.1007/s11548-023-02922-1

Article  PubMed  Google Scholar 

留言 (0)

沒有登入
gif