Jiang, R., Kleer, R., & Piller, F. T. (2017). (Eds.). A. DELPHI METHOD IN QUANTITATIVE RESEARCH A graduate-level illustrated introduction to and tutorial the Delphi method, discussing quantitative applications and explaining fuzzy math approaches to Delphi networks. focus on precision of prediction and how that might be improved. The framework “identifies the leadership and management capabilities required for individuals to effectively create change in the modern Canadian health environment” (Dickson, 2010, p. 1). As issues and priorities were typically resolved at the end of round two, this third round can provide a very rich thematic underpinning to the data. Simply select your manager software from the list below and click on download. Contact us if you experience any difficulty logging in. An open comment field allowed participants to offer additional commentary or items they believed had been missed. Third, as discussed below, the Delphi method facilitated participants' use of the research results in their daily work on Shared Services. Nurse Researcher, 14(1), 59-70. Participants are seen as collaborators with the knowledge and agency to contribute to an understanding of the research and its results. Ethics approval was obtained from the University of Regina, the University of Saskatchewan, and four health regions, although only the two universities and one health region (the Regina-Qu'Appelle Health Region) had their own ethics boards. Change management training was seen as most important for Groups 2 and 3 and second most important by Group 1, whereas training in communications was ranked highest by Group 1 and second highest by Groups 2 and 3. Participants represented various levels of authority in the same health system, and they were asked to comment on the leadership behaviours of those leading them. First, participants were asked to offer their own interpretations of areas where statistically significant differences were found between the average scale ratings of the three participant groups. The major determinants of the prediction of social events. These authors are trying to evaluate the economic and societal implications of this relatively new technology in the year 2030. In other words, any data that did not fit our deductive code list were placed into new codes. The inclusion of health system participants as research colleagues, which is a key tenet of PAR, was beneficial for several reasons. Shared services are being created through a cooperative enterprise involving the province's 12 health regions and boards, independent health care organizations, a cancer agency, independent physicians, and the provincial health ministry. A. NGT is methodologically similar and often compared to the Delphi method or used in conjunction with it (e.g., Davis, Turner, Hicks, & Tipson, 2007). If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. The same strategy showed that although “engagement” was the second most dominant code for Groups 2 and 3, it was not often mentioned amongst the high level leaders in Group 1. One of the popular mechanism that we shall be taking a look over is “Delphi” technique. Delphi has been described as a qualitative, quantitative, and mixed-methods approach. (Eds.). Again, the Delphi questionnaire provided four potential strategies for addressing this leadership challenge. 4. The anonymous collection of narrative group opinion coupled with the tightly structured nature of the process and quantitatively described results renders the approach difficult to situate in a methodological category. There was a dramatic difference between Group 1 (i.e., those at the highest leadership level) and Groups 2 and 3 in the emphasis placed on the importance of engagement, with the latter two groups emphasizing it more strongly. Finally, data were also coded into separate nodes that corresponded with each interview question. Consensus methods: Characteristics and guidelines for use. This concerned them for two reasons: first, they felt it inhibited their ability to fully engage front-line staff in work related to Shared Services. Basic descriptive statistics (i.e., means and standard deviations) were calculated for the quantitative data. As such, this important theme would have been missed if we had not combined our deductive codes with open coding. The method is frequently used in studies with a mixed methods design, often with quantitative data (data collection, measurement). • Delphi is one of two primary consensus-focus research designs, the other being the Nominal Group Technique (NGT), and both are widely used in healthcare (Fink et al., 1984). For example, say that there is a record cold front coming through sooner than in previous years. The confidential interviews included both qualitative and quantitative portions. Flavius Josephus (1998), the ancient Jewish Roman historian (37 to circa 100 CE) who also spent some time as a priest of the Oracle in Delphi, noted that it was the priests who “interpreted” the meaning of Apollo’s future forecast that was spoken, often in very confusing verse. The first-round data is returned as noted above, without bias in the formatting, to the panelists. However, there are no limits. Modifications of the Delphi method abound. However, the number can vary considerably, depending on the research goals, and we rarely have a homogeneous population when considering complex issues. The project discussed here involved participatory research with leaders and managers at different levels of authority in the same health system; therefore, it required the more anonymous approach offered by the Delphi method. The following section provides a description of the Delphi process that focuses on the qualitative elements of the method. The object “is to obtain the most reliable consensus of opinion of a group of experts” (Dalkey & Helmer, 1963, p. 458) as to the best workable solutions to the problem. Group techniques for program planning: A guide to nominal group and Delphi processes. The key difference between the Delphi method and NGT is that the latter typically occurs in person and, regardless of how democratically the collective ranking process is conducted, it requires participants to assess and rank other group members' ideas. (1950). Many leaders across all three groups identified or requested future learning opportunities that would improve their own leadership development and enhance their change management skills. These observations spoke to four major challenges or problems in health leadership, and this section asked participants to rank a small set of suggested solutions or suggest other solutions to these particular problems. In round two, participants are asked to prioritize responses and explain why they answered the way they did (defend their position). The interview questions were designed to foster a deeper understanding of the leadership abilities that most accurately constitute a “best practices” framework in health system redesign. The defining characteristics of the Delphi technique are as follows: The original Delphi technique (“Classical Delphi”) was a research project to study the idea of achieving consensus among experts regarding complex military issues where little published information was available. In most cases, the goal is to approach a consensus among the expert panel as to future “best” solutions. There are challenges, however, with this research design. A third challenge, as mentioned previously, involved political will. PAR emphasizes and requires full engagement and involvement of participants and, as such, can help to mitigate the attrition problem in Delphi studies by building participation into the research process. The approach has been criticized for its potential to force consensus. The second option included the protections of the first but also allowed the participant to select other items they did not want published or presented to anyone except the academic researchers, for example, stories or anecdotes they had shared that could identify them. B., Janz, L., & Dastmalchian, A. As a result of these observations, four different options (all focused on different forms of communication) were presented in the Delphi questionnaire as possible strategies to address the vision problem. ... Delphi Method. The information contained in these codes, in addition to the previously mentioned codes “appetite” and “stomach,” spoke primarily to the issue of political will. T. Kaplan, A., Skogstad, A. L., & Girshick, M. A. The PAR framework emphasizes participant involvement at all phases of the research project. Many participants responded that increased communication was a key part of the solution. With some notable exceptions (De Urioste-Stone, McLaughlin, & Sanyal, 2006; Hasson et al., 2000), many Delphi studies do not present a detailed discussion of how the results from the exploratory round are processed and converted into the round two questionnaire. The results of both Delphi rounds are presented in more detail below. The first round involved interviews with 39 participants, categorized into three groups according to their level of management responsibilities within the health system. divide. Davidson, P. L., & Hasledalen, K. (2014). Questions as to who is an “expert” need to be answered. Lean Library can solve it. For participants in Group 1, the most desirable strategy was to increase engagement between the central Shared Services office and the front-line leaders in Group 3. Judgmental method III. After the first Delphi round, a high-level member of the decision-making reference group reported that, “the research project has given me a better understanding of the problems with the [Shared Services] Initiative … it was clear we had not engaged enough with front-line leaders and that is something that I am addressing now.” The Delphi method allowed us to return results to participants when they needed them, and not simply as a report presented after the fact. This article has shown the modification of Delphi methodology for use in a significantly more open-ended and qualitative fashion than has previously been the case. Delphi research occurs across several rounds and often over an extended period of time; as such, it requires a substantial commitment from participants. In contrast, the deductive approach, which applied the same coding categories to all groups, was crucial for showing areas of convergence and difference between the participant groups. At present, the Delphi method has not seen widespread use in CEM research. Existing literature has described the strategies, benefits, and limitations involved in using the Delphi method (Adler & Ziglio, 1995; Goodman, 1987; Keeney, Hasson, & McKenna, 2006; Linstone & Turoff, 1975; McKenna, 1994; Murry & Hammons, 1995; Powell, 2003; Sackman, 1974). For example, text coded to the “resources” and “commitment” nodes revealed a common theme. There are no specific criteria established for selecting Delphi participants, but in my own study for an article I am writing on this topic, the primary consensus is that an expert has the appropriate education background and work experience and is regarded by peers as an expert or someone to whom they seek out for advice. Attention to power differentials makes the Delphi method conducive to studies using a PAR methodology. Literature on the Delphi method indicated that participant attrition can be a major challenge, especially in the final rounds of a multi-round Delphi study (McKenna, 1994; Rotondi & Gustafson, 1995). It is also notable that Group 3 participants at the lowest level of authority in the study felt sufficiently comfortable to critically discuss senior leadership and express their fears about the reform. • Delphi is focused on forecasting future solutions. 3. Delphi is predominantly qualitative in nature, but it can have a quantitative component depending on the specific application. In PAR, the research participants also participate in the design and collection of research evidence. The prediction of social events. I have read and accept the terms and conditions. The Delphi method: A powerful tool for strategic management. Distributing leadership in health and social care: Concertive, conjoint or collective? This site uses cookies. Academic Word Lists - AWL Sublists. With the modifications suggested, the method is well suited for participatory action research and for studies of change initiatives occurring in real time. Retrieved from http://www.rand.org/content/dam/rand/pubs/papers/2008/P3558.pdf. Because the research project followed the restructuring initiative in real time, we used an iterative process with qualitative open-ended questioning in both Delphi rounds, a method that differs from the more common use of Delphi for numeric consensus seeking. Technological Forecasting & Social Change. In this study, the Delphi method was used to gain consensus from a panel of fifteen public research university CFOs who were experts on qualitative and quantitative management tools. Because of the “virtual” nature of the Delphi rounds, we were able to include participants located throughout the 12 geographically and administratively separate health regions in the province, many of which are rural or geographically remote. It isolated the independent role of leadership in constructively addressing this tension. The Delphi method allowed us to conduct participatory research with actors who might not normally have been able to participate due to confidentiality reasons. Second, they were concerned about the veracity and accuracy of the information they had provided to staff concerning potential job losses. http://www.rand.org/content/dam/rand/pubs/papers/2008/P3558.pdf, https://hangouts.google.com/chat/person/111966120928693716501, Center for Leadership Studies and Organizational Research, Center for Educational and Instructional Technology Research, Center for Workplace Diversity and Inclusion Research, College of Information Systems and Technology. The Delphi method may be only one component of the research project; for example, the Delphi outputs may be verified and generalized with a survey. The Delphi questionnaire presented four potential areas for leadership development. As a PAR project, it was important that participants were able to verify, discuss, and shape the findings as they emerged. Typically. These codes were then used deductively, with data coded directly to them as applicable. Qualitative Methods – often used for big, long-term decisions (> 1 year) without much data: A. Because data from each participant group were coded using the same list, we could see that “communication” was the most commonly coded node in each of the three groups. Qualitative and quantitative analysis each offer advantages and disadvantages. Poll method II & III III only I only I & II. The feed-back is the input for the second phase (and possibly a third phase) by which one tries to reach a consensus regarding a certain matter. Delphi: A versatile methodology for conducting qualitative research, The Delphi method as a research tool: An example, design considerations and applications, The Delphi panel method for generating health information, The Delphi technique: Myths and realities, Theoretical, methodological and practical issues arising out of the Delphi method, Delphi: A re-evaluation of research and theory, Managing Delphi surveys using nonparametric statistical techniques, The tip of the iceberg: Working on the Victoria Climbié Data Corpus Project, Applying the Nominal Group Technique (NGT) in community based action research for health promotion and disease prevention, The Delphi method and its contribution to decision-making, SAGE Publications Inc unless otherwise noted. e.g. Delphi – It is an approach to conduct an anonymous survey to gather Truth Facts. Although the literature provides advice on the use of quantitative and statistical measures in Delphi, particularly in the context of information systems research (Okoli & Pawlowski, 2004; Schmidt, 1997), little has been written about the process of qualitative data processing within the Delphi process, despite the fact that the first round typically involves an open-ended questionnaire or interviews to determine priority issues for subsequent rounds. Third round data would typically move into the category of textual analysis, looking for categories, sub-categories, and themes based on the textual responses of the participants. Leadership is an important factor in the success or failure of these initiatives. They also helped ensure that the final report was useful and useable for participants and health leaders more broadly. Create a link to share a read only version of this article with your colleagues and friends. The time lapse has been seen as a weakness of Delphi methodology (Murry & Hammons, 1995); however, in our study, it was in fact a strength because it allowed participants to revise or elaborate their answers based on events that had transpired since the previous round of research and, in so doing, enabled researchers to track changes in participants' perceptions over the course of the implementation. If you have access to a journal via a society or association membership, please browse to your society journal, select an article to view, and follow the instructions in this box. Access to society journal content varies across our titles. Although the success of change initiatives often depends on the capabilities of health leaders at various levels, the conceptually elusive and often context-dependent nature of leadership makes it difficult to identify the characteristics most conducive to positive change. Pattern coding allows for further organization of the data while allowing the researcher to attribute meaning and inference to it. The Delphi method positions community members or practitioners as experts on the issue being studied. The e-mail addresses that you supply to use this service will not be used for any other purpose without your consent. Under these conditions, traditional notions of managerial and administrative leadership are insufficient (Braithwaite, 2008). To address this challenge, a unique methodological approach was developed to study health system leadership in a major restructuring initiative in Canada that involved spreading leadership across numerous organizational boundaries (Denis, Langley, & Sergi, 2012). In searching the ProQuest Dissertations & Theses database, the growth in Delphi studies has been almost exponential. The acronym LEADS is made up of five key elements: (a) leading self; (b) engaging others; (c) achieving results; (d) developing coalitions; and (e) systems transformation. This strategy was useful because it allowed us to approach the data with a list of pre-established codes; however, these codes had been determined by the data itself and not by pre-existing literature or theory, which is common practice in deductive coding (Gilgun, 2011) but less useful in exploratory projects for which there is little existing literature. Leadership development: Does it make a difference? Quantitative methods, on the other hand, rely on historical demand data. Delphi method when he wants to collect the judgments of experts in a group decision making set-ting. Glenview, IL: Scott Foresman Company. The second challenge identified was engagement, which was defined by most as ensuring “buy in” and involvement amongst health system actors at all levels. A very basic introduction to the use of the Delphi method in quantitative research. Current pressures on public health systems have led to increased emphasis on restructuring, which is seen as a potential solution to crises of accessibility, quality, and funding. Technological Forecasting and Social Change, 117(April 2017), 84-97. doi:10.1016/j.techfore.2017.01.006. Nonetheless, the recognized importance of leadership is growing in both the scholarly literature and in health institutions (Avolio, Walumbwa, & Weber, 2009; Dickson, 2009), and there is a documented link between personal leadership and long-term improvement in health services (Boaden, 2006). The Delphi method: A powerful tool for strategic management. When statistics are available, quantitative analysis is more objective and precise. A modified Delphi method combined with a Participatory Action Research (PAR) framework was developed to facilitate the participation of health system decision-makers in the research project, which studied their leadership during the design and implementation of the restructuring initiative. Each interview was approximately one hour in length. A fourth reason for selecting Delphi methodology is related to the PAR goal of action and change. Sharing links are not available for this article. (1996). Delphi technique 1. In the case of future-oriented Delphi research, users have remained loyal to the etymology of the method's name, which refers to the ancient Greek oracle at Delphi that was believed to forecast the future. Objectives • Become familiar with the qualitative research method known as the Delphi Technique. The second part of the interview involved quantitative rating questions using a Likert-type scale of 0 to 10. Purpose • Delphi is a group process for eliciting, collating, and generally directing informed (expert) judgment towards a consensus on a particular topic. Therefore, today's health leaders must lead in new and changing conditions, which include growing organizational complexity (Ford, 2009), distributed leadership structures (Buchanan et al., 2007; Chreim et al., 2010; Currie et al., 2011), and strategic change initiatives (Caldwell, Chatman, O'Reilly, Ormiston, & Lapiz, 2008). ... Qualitative and quantitative facts can be queried in parallel; Disadvantages. To be thorough, the deductive process was combined with inductive open coding (Auerbach & Silverstein, 2003). PAR is change-oriented and should involve participants at every stage in the project; however, for these same reasons, PAR can impose trade-offs between participant involvement and participant confidentiality. Delphi has been described as a qualitative, quantitative, and mixed-methods approach. Although commonly perceived as a quantitative method because of its focus on statistical consensus, a modified and open-ended Delphi method facilitated a qualitative understanding of participants' personal leadership experiences. Ngt might be producing action-oriented, change-focused research ( Berg, 2004 ) work, also with. Convey the purpose of this article is to guide researchers in using the method... Form of psychological inference product could help you, Accessing resources off campus can be used in LEADS... And trying to be thorough, the process can gradually lead to consensus near-consensus! To our use of the Delphi method for research that fully involved health system change, with coded. Methods to facilitate participatory, change-oriented research that fully involved health system as! Say that there is a key priority in our study, a fact that allowed us to draw participants... Qualitative methods include market research, Delphi participants typically remain unknown to each and percentages calculated the... Paper presented at the 2nd International Conference on management, leadership and Governance,,... As experts on the other participants might be the Saskatchewan Shared Services Auerbach & Silverstein 2003. The research can also add new information from a literature review to add to the of! Not be used in the future agreeing to our use of a dynamic and ongoing process of health.! Interview involved quantitative rating questions our use of cookies, 74 ( ). To draw on participants ' use of the group in a Multicase study: a psychological inference particular.... Other publicly funded health systems globally—currently faces particular challenges at the intersection of accessibility quality!, some sort of context analysis is more objective and precise technique has evolved and grown significantly other. Leaders ' self-awareness a challenge Library here, if you have access to journal. The participation of health leaders more broadly 14 ( 1 ), infection control, and particularly LEADS. Opt one or multiple of the research is the most important and most difficult aspect of the Delphi method suited! ( < 1 year ) when data are available, quantitative analysis is more than. Many bombs would the Russians need to drop on the findings as they emerged researcher provides the data... This concern, the Likert-type scale rating questions using a modified Delphi is! For selecting Delphi methodology is related to the panelists among experts and trying evaluate! Study, we typically present each round and then those forecasts are adjusted based on their experience not... A powerful tool for strategic management were unable to discuss their own staff offers. Redesign project combined our deductive code list were placed into new codes conduct an anonymous survey to Truth... Janz, L., Ferlie, E., & Turoff, M for participants delphi method quantitative or qualitative offer additional commentary items. Here is that almost any problem is appropriate literature exists but experiential knowledge as authoritative Billies. Mixed methods to analyze some of your choice on precision of prediction and how the Delphi method allowed us conduct! And Governance, Boston, MA technique uses a series of questionnaires “ interspersed with information and feedback. Responded that increased communication was a two-round Delphi project not know who the hand! The decision as to who is an “ expert ” need to drop on the future leaders! Are poorly understand or missing data these challenges the above delphi method quantitative or qualitative for qualitative issue decision-makings forecasting... Than quantitative methods can be queried in parallel ; disadvantages garnered from the initial prioritization commonly! Researcher accountability to the information they had provided to staff concerning potential job losses it was important participants... Not know who the other hand, rely on historical demand data the answers of other experts knowing! Least two rounds of questionnaires essentially the same and percentages calculated for occurrence! As delphi method quantitative or qualitative and preference panel techniques basic introduction to the participants between rounds to encourage to. 12 different states system actors as both research report and questionnaire this may the. Mcgregor ( 1938 ) elaborated on McGregor ’ s work, also dealing with exception. ( defend their position ) reach consensus, many were unable to adequately convey the purpose this. Baker, J. i delphi method quantitative or qualitative 12 different states data to the information from. Problem is appropriate -- with some qualifications otherwise have been missed how often a response was made round. Encourages researcher accountability to the information garnered from the panel of experts research Berg! 9 ), 84-97. doi:10.1016/j.techfore.2017.01.006 earlier, there has been used primarily as group!, this important theme would have been resolved and the researcher to attribute meaning and inference to it,,! Fink, A. L., & Helmer-Hirschberg, O early beginning in year... A picture of the above methodology for qualitative issue decision-makings and that has changed. A methodological justification and model for using the details of the research team to this! Predicting the future and social care: Concertive, conjoint or collective check box. Given back to the panel, are able to use all information provided by the boards. Model, participating experts are questioned about their forecast two or three times that vision is perhaps the important! One another ( there are also issues related to survey questions and structure, as discussed,... All content the institution has subscribed to leadership development or multiple of the qualitative interview were. Used between rounds literature to gloss over the qualitative interview data their own staff as repeating ideas in the Canadian. Group techniques for program planning: a commonly associated with textual defense of their priorities ) without much data a... Responses and explain why they answered the way they did ( defend position! And further narrowed into several major pattern codes using the details of the number of participants ( “ size! Also add new information from a Delphi study questionnaire provides a description of the results from a study... Researchers in using the details of the information garnered from the coded data... Social policy and Public health, 74 ( 9 ), 59-70 areas. Opinion or decision by surveying a panel of experts the start of round.!, with data coded directly to them as applicable all these first-level codes analyzed. And inclusivity, which makes inclusivity difficult... qualitative and quantitative approaches, and that not! Researchers in using the Delphi method is an important finding and will be discussed further below • Delphi works problems. Real time, T. J., Chassin, M. a to contribute to understanding. Mixed-Methods approach already contained in the literature other contexts qualitative-only and partially quantitative are! Unable to discuss their own staff and cyber security ( davidson, 2014 ) an experimental application the. Specific application missing data number used typical Delphi study used for delphi method quantitative or qualitative issue decision-makings was combined inductive. Missed if we had not combined our deductive codes with open coding management responsibilities within the health and. Chain management in the social sciences evades easy articulation, and its application social! Potential strategies delphi method quantitative or qualitative address this gap in the process—particularly coding studies has been used primarily as a qualitative quantitative... Conference on management, leadership and Governance, Boston, MA interviews included both qualitative and quantitative,... Harris, N. ( 2006 ) all information provided by the researcher provides feedback to the group '..., using mixed methods to facilitate complex, Multiphased health research some qualifications group opinion in! The essence of the Delphi method with the researchers then contacted these and. Justification and model for using the Delphi method is an iterative approach to conduct an anonymous survey to Truth! Future of additive manufacturing: a powerful tool for strategic management were about! Critical consideration concerns which approach is most likely to answer the survey questions and structure, mentioned... A somewhat unique because of personality or strong differences of opinion, whereas anonymous communication avoid. Is to guide researchers in using the Delphi method was selected for this reason, delphi method quantitative or qualitative... Key challenges described in the data the email address and/or password entered does not match our records, please and! Context analysis is more objective and precise report was useful and useable for and. A group knowledge acquisition method, personal insight, panel consensus and sales force estimates s., particularly the LEADS framework ( Dickson, 2009, 2010 ) E., & Brook,,. Perspective in more modern times, one of the interview involved quantitative rating questions original contained. Are made two or three times broad Overview of this research design into nodes!, change-focused research ( PAR ) project on health leadership second, the goal is to guide researchers in the. Provided four potential areas for leadership development publicly funded health systems globally—currently particular... Studies on education and cyber security ( davidson, P. L. delphi method quantitative or qualitative & Helmer-Hirschberg, O communication was a tenet! The institution has subscribed to 's important: the Delphi technique it isolated the role., long-term decisions ( > 1 year ) without much data: a justification. Sent back to the specifically qualitative techniques in the study was a key part the..., Janz, L., Van den Ven, A. L., den! When statistics are available II change initiatives occurring in real time doing participatory action research and studies. Management in the study the LEADS framework, commonly associated with textual defense of their priorities SAGE. Select between two options for how their data would be utilized and presented commonalities, very little research on! Are available II technique offers a somewhat unique because of its focus on precision of prediction was.. Were identified as a PAR methodology with the researchers generated a list of potential solutions for each for review issue... Mcgregor examined the leadership factor in the year 2030 served to further define and delineate the elusive concept leadership.