The Influence of Process Management: Uncovering the Impact of Real-Time Managerial Interventions via a Topic Modeling Approach

2021 ◽  
pp. 1-46
Author(s):  
Joshua Gyory ◽  
Kenneth Kotovsky ◽  
Jonathan Cagan
Keyword(s):  
Problem Solving ◽  
Real Time ◽  
Process Management ◽  
Topic Model ◽  
Design Team ◽  
Design Cognition ◽  
Team Members ◽  
Conceptual Engineering ◽  
Before And After ◽  
The Impact

Abstract Computationally studying team discourse can provide valuable, real-time insights into the state of design teams and design cognition during problem-solving. The particular experimental design, adopted from previous work by the authors, places one of the design team conditions under the guidance of a human process manager. In that work, teams under this process management outperformed the unmanaged teams in terms of their design performance. This opens the opportunity to not only model design discourse during problem solving, but more critically, to explore process manager interventions and their impact on design cognition. Utilizing this experimental framework, a topic model is trained on the discourse of human designers of both managed and unmanaged teams collaboratively solving a conceptual engineering design task. Results show that the two team conditions significantly differ in a number of the extracted topics, and in particular, those topics that most pertain to the manager interventions. A dynamic look during the design process reveals that the largest differences between the managed and unmanaged teams occur during the latter half of problem-solving. Furthermore, a before and after analysis of the topic-motivated interventions reveals that the process manager interventions significantly shift the topic mixture of the team members’ discourse immediately after intervening. Taken together, these results from this work not only corroborate the effect of the process manager interventions on design team discourse and cognition but provide promise for the computational detection and facilitation of design interventions based on real-time, discourse data.

A Topic Modeling Approach to Study the Impact of Manager Interventions on Design Team Cognition

2020 ◽  
Author(s):  
Joshua T. Gyory ◽  
Kenneth Kotovsky ◽  
Jonathan Cagan
Keyword(s):  
Problem Solving ◽  
Topic Modeling ◽  
Process Management ◽  
Topic Model ◽  
Design Team ◽  
Design Cognition ◽  
Team Cognition ◽  
Modeling Approach ◽  
Before And After ◽  
The Impact

Abstract In order to computationally study design cognition under design process management, this work utilizes a topic modeling approach to analyze design team discourse during problem-solving. The particular experimental design, from previous work by the authors, places one of the design team conditions under the guidance of a human process manager. In that work, teams under this guidance outperformed the unmanaged teams in terms of their design solutions. This opens the opportunity to not only model design discourse during problem solving, but also explore the impact of process manager interventions and their impact on design cognition. Utilizing this approach, a topic model is trained on discourse of human designers, for both managed and unmanaged teams, collaboratively solving a design problem. Results show that the two team conditions significantly differ in a number of the extracted topics, and in particular, those topics that most pertain to the manager interventions. Furthermore, a before and after analysis of the topic-motivated interventions, reveals that the process manager interventions significantly shift the topic mixture of the team members’ discourse toward that of the interventions immediately after they are provided. Together, these results not only corroborate the effect of the process manager interventions on design team discourse and cognition, but provide promise in the computational detection and facilitation of design interventions based on real-time discourse data.

Effect of Team Size on Problem-Solving and Solution Quality: An Empirical Study

2019 ◽  
Author(s):  
Seth Jacobs ◽  
Matthew Pfarr ◽  
Mohammad Fazelpour ◽  
Abdul Koroma ◽  
Tseday Mesfin
Keyword(s):  
Problem Solving ◽  
Empirical Study ◽  
Design Solution ◽  
Design Team ◽  
Design Teams ◽  
Team Size ◽  
Solution Quality ◽  
Team Members ◽  
Protocol Study ◽  
The Impact

Abstract The size of a team can affect how they tackle a design problem and solution quality. This paper presents a protocol study of the impact of team size on problem-solving and design solution quality. The protocols are coded with micro-strategies, and macro-strategies, and final solutions are scored using a rubric of meeting constraints, manufacturability, feasibility, and cost. The results show that the larger design team sizes analyze design solutions more frequently and propose solutions less than the smaller design teams. Among the three team sizes of 1, 3, and 5, the teams of three designers scored the best on final designs. These teams used a fair amount of both proposing solutions and analyzing solutions of micro-strategies. The teams of 5 designers use backtracking macro-strategies more frequent than teams of 3 and one because as the team size increases, more time is spent among team members to discuss previous ideas.

Human versus Artificial Intelligence: A Data-Driven Approach to Real-Time Process Management During Complex Engineering Design

2021 ◽  
pp. 1-38
Author(s):  
Joshua Gyory ◽  
Nicolas F Soria Zurita ◽  
Jay Martin ◽  
Corey Balon ◽  
Christopher McComb ◽  
...  
Keyword(s):  
Real Time ◽  
Design Process ◽  
Process Management ◽  
Superior Performance ◽  
Team Members ◽  
Engineering Teams ◽  
Complex Design ◽  
Data Driven Approach ◽  
Complex Engineering ◽  
Process Managers

Abstract Managing the design process of teams has been shown to considerably improve problem-solving behaviors and resulting final outcomes. Automating this activity presents significant opportunities in delivering interventions that dynamically adapt to the state of a team in order to reap the most impact. In this work, an Artificial Intelligent (AI) agent is created to manage the design process of engineering teams in real time, tracking features of teams' actions and communications during a complex design and path-planning task with multidisciplinary team members. Teams are also placed under the guidance of human process managers for comparison. Regarding outcomes, teams perform equally as well under both types of management, with trends towards even superior performance from the AI-managed teams. The managers' intervention strategies and team perceptions of those strategies are also explored, illuminating some intriguing similarities. Both the AI and human process managers focus largely on communication-based interventions, though differences start to emerge in the distribution of interventions across team roles. Furthermore, team members perceive the interventions from the both the AI and human manager as equally relevant and helpful, and believe the AI agent to be just as sensitive to the needs of the team. Thus, the overall results show that the AI manager agent introduced in this work is able to match the capabilities of humans, showing potential in automating the management of a complex design process.

scholarly journals 888. Improving outcomes with revised preoperative universal decolonization protocol

2020 ◽  
Vol 7 (Supplement_1) ◽  
pp. S479-S479
Author(s):  
Silvia I Gnass
Keyword(s):  
Surgical Procedures ◽  
Baseline Period ◽  
Surgical Team ◽  
Team Members ◽  
Surgical Infection ◽  
Before And After ◽  
Trial Basis ◽  
The Impact ◽  
Before And After Study ◽  
Patient Infection

Abstract Background In order to improve outcomes, including reduced surgical infection rate and costs, a revised universal preoperative decolonization protocol was implemented on a trial basis. Methods In a 12 month before and after study at a public teaching hospital in southern California, an alcohol based nasal antiseptic was introduced in place of nasal povidone iodine (PVI) for all surgical patients pre-operatively, paired with chlorhexidine (CHG) bathing which was already in place. All surgical procedures were included, the most common being cholecystectomy, cesarean section and hip fracture. The alcohol nasal antiseptic was selected to replace the PVI nasal antiseptic based on efficacy, staff preference and cost. At the same time, surgical team members began self-application of the alcohol nasal antiseptic each day prior to surgical procedures. This was not mandatory and compliance was not tracked, though informal feedback and observation revealed most surgical team members were applying the nasal antiseptic prior to cases daily. Results In comparison to the 6 month baseline period where there were 27 SSI in 1188 procedures, during the 6 month study period there were 10 SSI in 1253 procedures, representing a 63% reduction (p=.0162) for all types of procedures. We have observed a reduction of 17 SSIs in 2019, compared to the previous year, during the 6 months period. That means a saving of $589,420 during the same period. Conclusion Preoperative universal decolonization with alcohol based nasal antiseptic in place of nasal PVI, paired with CHG bathing, was effective in reducing SSI rate and associated costs. Further study is needed to measure and assess the impact of surgical team member nasal decolonization on patient infection risk and rate. Disclosures All Authors: No reported disclosures

Abstract 199: Hardwiring Institutional Learning Through Team-Based Problem-Solving in a Pediatric Cardiovascular Intensive Care Unit

2012 ◽  
Vol 5 (suppl_1) ◽  
Author(s):  
Eric A Williams ◽  
Lauren Ladwig ◽  
De Ann Nikolai ◽  
Gay Matthews ◽  
Jeanine M Graf
Keyword(s):  
Problem Solving ◽  
Real Time ◽  
Task Force ◽  
High Reliability ◽  
Ease Of Use ◽  
Practice Change ◽  
Frontline Staff ◽  
Pareto Chart ◽  
Exact Test ◽  
Before And After

Introduction: Our purpose was to develop a high reliability system (The SWARM) to empower frontline staff to share information regarding communication and process inefficiencies to rapidly develop solutions. Hypothesis: A systematic process for recognition, evaluation, and transparent feedback would increase the use of problem-solving. Methods: A multidisciplinary team, led by a physician and nurse, developed an email strategy to let frontline staff identify problems in real-time and solve and spread this knowledge throughout the CVICU and Heart Center. Problems submitted were outside the scope of institutional patient safety reporting. Unless immediate attention was required, the SWARM team evaluated entries weekly. Entries were categorized as: 1) Useful information to share with ICU staff; 2) More information needed to be obtained and personnel consulted; or 3) A multidisciplinary task force needed to generate consensus and evidence-based guidelines. Solutions were shared with ICU staff via direct email and a monthly newsletter. After 6 months ICU staff were surveyed with 5 questions regarding awareness, utility, and ease of use of the SWARM process. Staff were resurveyed after re-education and institution of leadership walk-rounds. Rates of SWARM entries before and after re-education were compared using Fisher’s Exact Test for proportions.*p<.05 Results: Of the total 65 SWARM entries, 9% were level 1, 86% were level 2 and 5% were level 3. Entries were categorized into problem types and are shown in the Pareto Chart as percent of total(See Chart). Pre- and post- survey results indicated that staff awareness increased from 70% to 91% and staff understanding of utility increased from 57% to 75%. After re-education, use in CVICU increased by 417% (0.7/week to 2.9/week)*. The increased rate of use of the SWARM system has been sustained for 8 weeks post education. The current rate is 1.8/week. The top two Solutions by category were Re-Education (62%) and Practice Change (17%). Conclusions: A system was developed to better manage quality through real-time awareness of our performance. The formal creation of a standardized context for information sharing has resulted in a significant increase in the performance of problem-solving. This multidisciplinary effort allows for minimizing unnecessary variation in how quality improvement efforts are identified and performed.

Only As Strong As the Strongest Link: The Impact of Individual Team Member Proficiency in Configuration Design

2020 ◽  
Author(s):  
Ethan Brownell ◽  
Jonathan Cagan ◽  
Kenneth Kotovsky
Keyword(s):  
Team Performance ◽  
Team Member ◽  
Positive Impact ◽  
Individual Characteristics ◽  
Configuration Design ◽  
Design Team ◽  
Markov Modeling ◽  
Team Members ◽  
Hidden Markov Modeling ◽  
The Impact

Abstract Prior research has demonstrated how the average characteristics of a team impact team performance. Individual characteristics of team members and individual team member behavior have been largely ignored, especially in the context of engineering design. In this work, a behavioral study was conducted to uncover whether the most or least proficient member of a configuration design team had a larger impact on overall performance. It was found that a configuration design team is most dependent on the proficiency of its most proficient member and results suggest that replacing the most proficient member with an even more proficient member can be expected to have a more positive impact than replacing any other member with a higher proficiency member of the same change in proficiency. The most proficient member had a significant positive effect on how quickly the team reached performance thresholds and that the other members of the team were not found to have the same positive impact throughout the design study. Behavioral heuristics were found using hidden Markov modeling to capture the differences in behavior and design strategy between different proficiency members. Results show that high proficiency and low proficiency team members exhibit different behavior, with the most proficient member’s behavior leading to topologically simpler designs and other members adopting their designs, leading to the most proficient member driving the team design and team performance.

Vancomycin Use: Room for Improvement Among Hospitalized Children

2016 ◽  
Vol 30 (3) ◽  
pp. 296-299 ◽  
Author(s):  
Jessica Gillon ◽  
Meng Xu ◽  
James Slaughter ◽  
M. Cecilia Di Pentima
Keyword(s):  
Real Time ◽  
Interrupted Time Series ◽  
Antibiotic Use ◽  
Hospitalized Children ◽  
Interrupted Time Series Analysis ◽  
Prospective Audit ◽  
Children's Hospitals ◽  
Children’S Hospitals ◽  
Before And After ◽  
The Impact

Introduction: The use of vancomycin is common among hospitalized children. We sought to evaluate the impact of prospective audit with real-time feedback on vancomycin use and pharmacy costs. Methods: Vancomycin use was evaluated at Monroe Carell Jr Children’s Hospital at Vanderbilt (MCJCHV) before and after the implementation of prospective audit with intervention and feedback to providers in 2012. Antibiotic use was compared to academic children’s hospitals with established antimicrobial stewardship programs (ASPs). Two similar pediatric academic institutions without an ASP were used as nonintervention controls. Analysis of monthly days of antibiotic therapy (DoT) per 1000 patient-days was performed by interrupted time series analysis. Results: Monthly vancomycin use decreased from 114 DoTs/1000 patient-days to 89 DoTs/1000 patient-days ( P < .0001). We did not find significant differences in the slope of change in vancomycin use between MCJCHV and institutions with ASPs either before or after the intervention ( P = .86 and P = .71, respectively). When compared to children’s hospitals without ASPs, the use of vancomycin was significantly lower at MCJCHV ( P < .001). Conclusion: The use of vancomycin at academic children’s hospitals with an ASP is declining. In our experience, prospective audit with real-time intervention and feedback to providers significantly reduced the use and costs associated with vancomycin.

scholarly journals 2084. Implementation of a Diagnostic Stewardship Algorithm by Bedside Nurses to Reduce Unnecessary Urinary Cultures in Hospitalized Adult Patients

2019 ◽  
Vol 6 (Supplement_2) ◽  
pp. S703-S704
Author(s):  
Valeria Fabre ◽  
Ashley Pleiss ◽  
Zoe Demko ◽  
Anna Sick-Samuels ◽  
Lauri Hicks ◽  
...  
Keyword(s):  
Asymptomatic Bacteriuria ◽  
Control Unit ◽  
Johns Hopkins Hospital ◽  
Post Intervention ◽  
Statistical Process ◽  
Team Members ◽  
Before And After ◽  
Algorithm Implementation ◽  
The Impact ◽  
Intervention Unit

Abstract Background Urine cultures (UCx) are often ordered in patients without symptoms of urinary tract infection. A pilot study was conducted to assess the impact of a nurse-driven UCx diagnostic stewardship intervention for adult inpatients. Methods We interviewed eight nurses to determine the feasibility of a nurse-driven UCx stewardship intervention. Based on their feedback, an algorithm with appropriate indications for UCx was developed (Figure 1) and approved by physicians and nurses for piloting on a 24-bed medicine unit at The Johns Hopkins Hospital. UCx orders/100 patient-days (PD) were trended with statistical process charts in the intervention and a control unit. Nurses used the algorithm to guide discussions with ordering providers and to suggest instances where UCx may be unnecessary (“intervention”). Nurses were educated on an antibiotic (abx) use safety and appropriate testing during live sessions prior to algorithm implementation. Two study team members reviewed all UCx ordered in the intervention unit 12 months before and 6 months after the intervention for appropriateness based on algorithm criteria. Feedback on UCx order appropriateness and case-based discussion were provided to nurses via in-person meetings post intervention. Data were compared using the χ 2 or the Mann–Whitney test as appropriate. The rate of UCx orders before and after the intervention were compared using a standard incident ratio (IRR). Results With algorithm implementation, the mean rate of UCx orders/100 PD decreased from 2.7 to 1.8 (39% decrease) in the intervention unit (IRR 0.61, 95% confidence intervals (CI) 0.45–0.82, P = .16). Mean UCx order rates in the control unit were 2.49 and 2.99, respectively (Figure 2). Characteristics of patients reviewed for appropriateness were similar between the two study periods: median age 63 (IQR 39, 74) vs. 56 (IQR 45, 76), female sex 65% vs. 61%, on hemodialysis 7% vs. 11%, urinary catheter present 20% vs. 29%. The proportion of inappropriate UCx decreased from 59% (98/165) to 50% (32/64) (P = 0.16). There were 8 and 1 cases of asymptomatic bacteriuria inappropriately treated in the pre- and post-intervention periods, respectively (42 and 7 abx days). Conclusion With the appropriate training and tools, nurses can steward UCx and reduce unnecessary testing and abx use Disclosures Sara E. Cosgrove, MD, MS, Basilea: Consultant; Theravance: Consultant.

Understanding the Context of Multidisciplinary Design: Establishing Ecological Validity in the Study of Design Problem Solving

1993 ◽  
Vol 37 (16) ◽  
pp. 1082-1086 ◽  
Author(s):  
Michael D. McNeese ◽  
Brian S. Zaff ◽  
Clifford E. Brown ◽  
Maryalice Citera ◽  
Jonathan Selvaraj
Keyword(s):  
Problem Solving ◽  
Human Factors ◽  
Concept Mapping ◽  
Design Process ◽  
Collaborative Design ◽  
Design Problem ◽  
Design Team ◽  
Design Teams ◽  
Team Members ◽  
Design Activities

The need to understand the design process in all its complexity is motivated by an interest in the development of tools and technologies that would be capable of aiding collaborative design teams. This development effort depends upon an understanding of design activities as they occur within a real world context. Observations of design activities that are made without direct communication with the design team members may fail to capture many of the subtler aspects of the process - aspects that are best understood when described by the design team members themselves. In order to supplement observational studies, this paper presents a case study in which a dialog with members of a variety of collaborative design teams was established in order to elicit information about the nature of collaborative design. A knowledge acquisition technique, concept mapping, was used to achieve an understanding of the role of human factors specialists within the collaborative design process specific to the Air Force's system acquisition program. Results highlight various findings about the nature of design problem solving such as the way different organizational settings influence human factors input in the design process/product. The paper discusses the usefulness of concept mapping to capture in-depth design knowledge and how this type of knowledge complements other approaches to understanding design.

Sign in / Sign up

Export Citation Format

Share Document