Closing the Loop by Operationalizing Systems Engineering and Design (CLOSED)
Motivation:
Specific Aims :
Aim 1:​Use systems engineering and patient engagement to design, develop, and refine a highly reliable “closed loop” system for diagnostic tests and referrals that ensures diagnostic orders and follow-up occur reliably within clinically- and patient-important time-frames.
​
Aim 2: Use systems engineering and patient engagement to design, develop, and refine a highly reliable “closed loop” system for symptoms that ensures clinicians receive and act on feedback about evolving symptoms and physical findings of concern to patients or clinicians.
​
Aim 3: Design for generalizability across health systems more broadly so that the processes created in Aims 1 and 2 are effective in (1) a practice in an underserved community, (2) a large tele-medicine system, and (3) a representative range of simulated other health system settings and populations.
Partners:
Sunday, June 2, 2019
Sunday, June 2, 2019
Approach:
Sunday, June 2, 2019
Results to Date:
Publications:
Phillips, R.S., Benneyan, J., Bargal, B. et al. Closing the Loop: Re-engineering the Assessment and Tracking of Symptoms in Primary Care. J GEN INTERN MED (2022). https://doi.org/10.1007/s11606-022-07886-7
White T, Aronson M, Sternberg S, Shafiq U, Berkowitz S, Benneyan J, Phillips R, Schiff G (2022), Closing the loop on radiology recommendations: Developing and applying a taxonomy to track performance of recommended actions, JAMA Network Open, 2022;5(7):e2222549. doi: https://doi.org/10.1001/jamanetworkopen.2022.22549
Phillips, R, Benneyan J, Bargal B, Schiff G (2022), Closing the loop: Re-engineering the assessment and tracking of symptoms in primary care, Journal of General Internal Medicine, accepted for publication.
​
Pagani K, Lukac D, Olbricht S, Aronson M, Benneyan J, Fernandez L, Salant T, Schiff G, Shafiq U, Sternberg S, McGee (2022). Urgent referrals from primary care to dermatology within a tertiary care institution: Patterns, outcomes, and the need for systems improvement, Archives of Dermatological Research, in second review.
​
Benneyan J, White T, Nehls N, Yap T, Aronson M, Sternberg S, Anderson T, Goyal K, Lindenberg K, Kim H, Cohen M, Phillips R, Schiff G (2020). Systems analysis of a dedicated ambulatory respiratory unit for seeing and ensuring follow-up of patients with COVID-19 symptoms, Journal of Ambulatory Care Management, 44(4): 293-303, 2021. doi: 10.1097/JAC.0000000000000390. PMID: 34319924; PMCID: PMC8386384.
​
Bargal B, Benneyan J, Eisner J, Alev A, Singer S (2018), Use of systems-theoretic process analysis to design safer opioid prescribing processes, IISE Transactions on Occupation Ergonomics and Human Factors, 6:3-4, 200-208, doi: 10.1080/24725838.2018.1521887.
​
Bargal B, Benneyan J Eisner J, Ergai A, Chen K, Chari S. Alternate safety methods to root cause analysis for learning from retrospective healthcare adverse events, working paper.
​
Benneyan J, Shutt A, Ramirez N (2017). A human-centered approach to improving healthcare: The SEIPS model, Proceedings of the 2017 Industrial and Systems Engineering Conference.
​
Benneyan J, Bargal B, Chen K, Singer S, others tbd (2021), Development and evaluation of a systems engineering high reliability patient safety learning lab, working paper.
​
Atkinson M, Benneyan J, Schiff G, Phillips R, Singer S (2021). Evaluating a patient safety learning lab intervention to create an interdisciplinary ecosystem for healthcare innovation, Health Care Management Review, accepted for publication.
​
Atkinson M, Benneyan J, Schiff G, Phillips R, Hunt L, Singer S (2021). Patient engagement in system redesign teams: a process of social identity, Journal of Health Organization and Management, https://doi.org/10.1108/JHOM-02-2021-0064
​
Bersani K., Fuller TE, Garabedian P, Espares J, Mlaver E, Businger A, Chang F, Boxer R, Schnock KO, Rozenblum R, Dykes PC, Dalal AK, Benneyan J, Lehmann L, Gershanik E, Bates DW, Schnipper JL (2020). Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform, 11(1):34-45, doi: 10.1055/s-0039-3402756.
​
Businger A, Fuller T, Rozenblum R, Dalal A, Schnock K, Rossetti S, Schnipper J, Benneyan J, Bates B, Dykes P (2019). Lessons learned implementing a complex and innovative patient safety learning laboratory project in a large academic medical center, Journal of the American Medical Informatics Association, doi: 10.1093/jamia/ocz193
​
Carayon P, Wooldridge A, Hose B, Salwei M, Benneyan J (2018). Improving patient safety through human factors and systems engineering: experiences, opportunities, and challenges, Health Affairs, 37, 1862-1869.
​
Carlile N, Fuller T, Benneyan J, Bargal B, Schiff G, Hunt L, Singer S (2021). Lessons learned in implementing a chronic opioid therapy management system, Journal of Patient Safety, accepted for publication.
​
Collins S, Couture B, DeBord-Smith A, Gershanik E, Lilley E, Chang F, Yoon C, Lipsitz S, Benneyan J, Bates D (2017), Mixed-methods evaluation of real-time safety reporting by hospitalized patients and their care partners: The MySafeCare application, Journal of Patient Safety. doi: 10.1097/PTS.0000000000000493
Collins S, Couture B, Dykes P, Schnipper J, Fagan M, Benneyan J, Sheikh A, Bates D, Sordo M (2018), Implementation, evaluation, and recommendations for extension of AHRQ common formats to capture patient and care partner generated safety data, Journal of the American Medical Informatics Association, https://doi.org/10.1093/jamiaopen/ooy004.
Couture B, Fagan M, Gershanik E, Yoon C, Benneyan J, Bates DW, Collins SA. Towards analytics of the patient and family perspective: A case study and recommendations for data capture of safety and quality concerns. AMIA Annual Symposium Proceedings. Washington, D.C.; 2017:615-624.
Dalal A, Fuller T, Garabedian P, Ergai A, Balint C, Bates D, Benneyan J (2019). Systems engineering and human factors support of a system of novel EHR-integrated tools to prevent harm in the hospital, Journal of the American Medical Informatics Association, 26(6): 553-560, https://doi.org/10.1093/jamia/ocz002
Das P, Benneyan J, Powers L, Carmody M, Kerwin J, Singer S (2018), Engineering safer care coordination from hospital to home: lessons from the USA, Future Healthcare Journal, 15(3): 164-170.
Dykes P, Burns Z, Adelman J, Benneyan J, Bogaisky M, Carter E, Ergai A, Lindros M, Lipsitz S, Scanlan M, Shaykevich S, Bates D (2020). Evaluation of a patient-centered fall prevention toolkit to reduce falls and injuries: a nonrandomized controlled trial, JAMA Open, 2020;3(11):e2025889. doi:10.1001/jamanetworkopen.2020.25889.
Fuller T, Garabedian P, Lemonias D, Joyce E, Schnipper J, Harry E, Bates D, Dalal A, Benneyan J (2020). Assessing the cognitive and work load of an electronic patient safety dashboard in the context of opioid prescribing, Applied Ergonomics, 85, https://doi.org/10.1016/j.apergo.2020.103047
Katsulis Z, Ergai A, Leung W, Schenkel L, Rai A, Adelman J, Benneyan J, Bates D, Dykes P (2016), Iterative user centered design for development of a patient-centered fall prevention toolkit, Applied Ergonomics, 56:117-126.
Mlaver E, Schnipper J, Boxer R, Breuer D, Gorbovitsky G, Harry E, Gershanik E, Wolf L, Dykes P, Benneyan J, Bates D, Lehmann L (2017), User-centered collaborative design and development of an inpatient safety dashboard, Joint Commission Journal on Quality Improvement, 43(12): 676–685. http://dx.doi.org/10.1016/j.jcjq.2017.05.010
Nehls N, Yap T, Salant T, Aronson M, Schiff G, Olbricht S, Reddy S, Sternberg S, Anderson T, Phillips R, Benneyan J (2021), Systems engineering analysis of diagnostic referral closed loop processes, BMJ Open Quality 2021;10:e001603. doi: 10.1136/bmjoq-2021-001603.
Nehls N, Benneyan J, Poulsen K, Schiff G, Salant T, Aronson M, Sternberg S, Fernandez L, Phillips R5 (2022), A systems reliability view of studies to improve closing diagnostic loops, working paper.
Phillips R, Schiff G, Benneyan J. (2022). Designing reliable systems for following worrisome symptoms, working paper.
Smith C, Benneyan J, Nehls N, Jacobson M, Van der Kloot T, Bates P, Ning-You R, Han P, Ergai A (2018). Workflow analysis of interprofessional medical education in the clinical learning environments to improve learning outcomes and patient safety, Healthcare Systems Engineering Institute Research Report.
Systems process analysis of graduate medical education in clinical learning environments (2018). Healthcare Systems Engineering Institute Research Report.
White T, Aronson M, Sternberg S, Shafiq U, Berkowitz S, Benneyan J, Phillips R, Schiff G (2022). Closing the loop on radiology recommendations: Developing and applying a taxonomy to track performance of recommended actions, in review.
Wiggins M, Bates P, Benneyan J (2019). Patient safety in graduate medical education, Health Affairs, 38(2), p. 328, doi 10.1377/hlthaff.2018.05474.
Conference Presentations:
Bargal B, Benneyan J, Jacobson M, Atalay A (2018). Using systems-theoretic process analysis (STPA) to design safer opioid prescribing processes, James Benneyan (Healthcare Systems Engineering Inst.), Human Factors in Healthcare annual symposium.
Bargal B, Benneyan J, Eisner J, Atalay A, Singer S (2018). Use of systems-theoretic process analysis (STPA) to design safer opioid prescribing processes, MIT STAMP Annual Workshop.
Bargal B, Eisner J, Chen K, Chari S, Ergai A, Benneyan J (2017). Alternatives to root cause analysis for understanding healthcare adverse events, The International Conference on Applied Human Factors and Ergonomics.
Benneyan J, Phillips R, Singer S (2017). An AHRQ patient safety learning laboratory for primary care – The engineering high reliability learning lab (EHRLL), Improving Primary Care Through Systems and Industrial Engineering (I-PrACTISE) annual conference.
Benneyan J, Phillips R, Singer S (2018). Exploring new approaches to patient safety: The AHRQ engineering high reliability learning lab (EHRLL), Society for Health Systems Healthcare Process Improvement annual conference.
Benneyan J, Taffe K (2020). The AHRQ patient safety learning labs: panel discussion (What are we learning?), Society for Health Systems Healthcare Process Improvement annual conference.
Benneyan J, Shutt A, Atalay A, Sherritt K, Singer S (2018). Designing safer opioid prescribing processes in primary care through use of systems engineering, Society for Health Systems Conference (best poster award).
Benneyan J, Shutt A, (2018). A systems approach to designing safer opioid prescribing processes in primary care, Human Factors in Healthcare annual symposium.
Benneyan J, Holmes A (2018). From safety-I to safety-II: using systems engineering to redesign a hospital care to home care transition process, Human Factors in Healthcare annual symposium.
Benneyan J, Holmes A, Lemonias D, Cyr M (2018). Examples and experiences using FMEA to design better healthcare processes, Human Factors in Healthcare annual symposium.
Benneyan J (2018). The AHRQ patient safety learning laboratories: What are we learning?, Human Factors in Healthcare annual symposium.
Benneyan J, Ergai A, Jacobson M, Silverstein S (2017). Systems analysis of graduate medical education processes, Society for Health Systems Annual Conference.
Benneyan J, Jacobson M, Ergai A, Shukla M, Han P, Waterson L, Peck J, Meyer D, Burton L, Van der Kllot T, Bing-You R, Bates P (2018). Systems engineering collaboration to analyze internal medicine residency, MaineHealth Costas Lambrew Research Conference.
Couture B, Lilley E, Chang F, DeBord Smith A, Cleveland J, Ergai A, Katsulis Z, Benneyan J, Gershanik E, Bates D, Collins S. Applying user-centered design methods to the development of an mHealth application for use in the hospital setting by patients and care partners, Appl Clin Inform 2018, 9(2):302-12.
Dalal A, Fuller T, Garabedian P, ,.., Benneyan J. An interdisciplinary team to support implementation of a “System-of-systems” to identify, assess, and mitigate threats to patient safety in real-time. Society of Hospital Medicine Conference. 2018, Apr 8-11.
Dalal A, Fuller T, Neri P, Breuer D, Bates D, Benneyan J (2017), Opportunities and Challenges for an Interdisciplinary Team to Guide Adoption of Technology to Dissipate Threats to Patient Safety in Real-Time, American Medical Informatics Association, Washington, DC.
​
Fuller T, Neri P, Lemonias D, Harry E, Schnipper J, Joyce E, Martell T, Nehls N, Bates D, Dalal A, Benneyan J (2018), Assessing the usability and cognitive load of an electronic safety dashboard in the context of opioid prescribing: Preliminary results, Human Factors and Ergonomics in Health Care, Boston, MA
Fuller T, Espares J, Duckworth M, Couture B, Schnipper J, Dykes P, Collins S, Dalal A, Rozenblum R, Schnock K, Gershanik E, Lipsitz S, Businger A, Neri P, Benneyan J, Bates D (2017) Making acute care more patient-centered: Implementing a learning lab, iHealth 2017 Clinical Informatics Conference, May 2-4, 2017, Philadelphia, PA
Fuller T, Dalal A, Breuer D, Neri P, Bates D, Benneyan J (2017), Using systems engineering methods to identify, assess, and mitigate preventable harm as part of a patient safety learning laboratory, American Medical Informatics Association, Washington, DC
Hunt L, Benneyan J, Salem J (2018). Engaging students as levers in improvement work, Institute for Healthcare Improvement Annual Forum.
Nehls N, Salant T, Benneyan J, Aronson M, Sternberg S, Schiff G, Phillips R, Rivera M, O’Brian D, Dreilak M (2021). CLOSED: Closing loops by operationalizing systems engineering and design, Institute for Healthcare Improvement 2021 Annual Forum.
Neri P, Dalal A, Fuller T, Breuer D, Ergai A, Bates D, Benneyan J (2017), Experiences implementing a user-centered design process across a large patient safety learning laboratory, American Medical Informatics Association, Washington, DC.
​
Salant T, Benneyan J, Nehls N, Aronson M, Sternberg S, Schiff G, Phillips R, Rivera M, O’Brian D, Dreilak M (2021). CLOSED: Closing loops by operationalizing systems engineering and design, Silverman Symposium on Quality and Patient Safety.
Salem J, Berry J, Cox J, O’Neill M, Comstack C, Holler N, Crofton C, Singer S, Benneyan J (2018). Designing high reliability processes for coordinating perioperative care for medically complex children, Society for Health Systems Annual Conference.
Salem J, Benneyan J (2018). Designing high reliability processes for coordinating perioperative care for medically complex children, Human Factors in Healthcare annual symposium.
Smith C, Holmes A, Nehls N, Junod C, Benneyan J, Carmody M, Powers L, Kerwin J, Singer S (2018). Understanding transitions from hospital care to home care: An industrial engineering approach, Society for Health Systems Annual Conference.
Smith C, Nehls N, Schleis A, Jacoson M, Bates P, Bing-You R, Van der Kloot T, Ergai A, Benneyan J (2018), Workflow analysis of interprofessional medical education in the clinical learning environments to improve learning outcomes and patient safety, Human Factors in Healthcare annual symposium.
Ward E, Benneyan J, Hunt L (2018). Reaping rewards, overcoming barriers: Partnering with patients in care redesign, 20th National Patient Safety Annual Congress.