|
Proposed model to apply Soft Systems Methodology (SSM) within National Information Infrastructure (NII) and to incorporate Delphi into SSM.The incorporating of Delphi into SSM to prioritize the list of issues and applying SSM within the NII situation can be viewed as human activity systems. The following relevant systems and root are proposed:
CATWOE analysis: C = Customer = analyst and analysts supervisor A = Actors = analyst and Delphi participant T = transformation = less understand of SSM --à improved understand of SSM NII situation -----à improved NII situation SSM --------------à incorporated Delphi within SSM W = system thinking is a valuable methodology for improving the problem situation , O = owner = analyst and supervisor of analyst E = environmental constraints = resources, time, skills, desired, knowledge, money, methodology and technique constraints, NII culture. Figuree 1 (will be inserted later) Human activity system based on proposed model of applying SSM within NII situation. The required activities demanded by the above root definitions are shown in the conceptual model of figure 1. This conceptual model suggest that the analyst, in effort to carry SSM study within NII situation: (1) understand : SSM, Delphi, NII and case study (2) identify list of NII issues, (3) apply the Delphi questionnaire to prioritising NII issue, (4) select a relevant system and develop conceptual model, (5) compare conceptual model with the NII situation, (6) continue to debate desirable and feasible changes and (7) finally recommend action. Understand: SSM, Case Study, Delphi and NII Understand SSM (activity 1.1) Introduction to Soft Systems Methodology Checkland and Scholes (1990) contend that Soft Systems approach is a particularly productive methodology for studying any organized purposeful human activity. A set of such purposeful human activities can be termed as a system, in which the various activities are interrelated. Soft systems methodology (SSM) refers to such a set of activities as a human activity system. Checkland (1981) summarise a system characteristic which can be used as the basis for examining models of human activity to satisfy the following criteria:
Soft systems approach has been applied to various areas: public utilities, health, industry, agriculture, research, education etc (Watson and Smith, 1988; Checkland and Scholes, 1990). SSM has been also applied for government organization concerned with the development of computing and telecommunication in the State government service and Health Service at national level (Checkland and Scholes, 1990). Therefore SSM can also be argue as useful methodology to analyse information system policy at the national level. SSM is particularly good, because of the intellectual activity it involves of conceptual modelling, as a tool for self-analysis for the reflective information systems (IS) practitioner involved in projects at the national level. There are two version of methodology available: the classic 7 stage SSM and the developed form of SSM. The developed form of SSM examines two parallel streams of enquiry; logical and the cultural. The main difference, in terms of the logical stream, between this developed form and the earlier classic 7 stage approach, is that the step by step nature of the methodology is de-emphasized. And this developed form SSM puts much more emphasis on the stream of cultural enquiry. The methodology adopted here for analysing the process of making NII is the developed form SSM, since the culture plays a very important role in the process of creating NII. A brief history of Soft Systems Methodology Soft Systems methodology was developed by members of the Department of Systems at Lancaster University. Professor Peter Checkland and his associates undertook the research programme to apply engineering principles to managerial problems that were considered to be complex problem domains. Such problem domains cannot be formulated and defined precisely, often the problem could be considered to be an area of concern requiring attention. There are three reasons for this, firstly, the methodology is suitable to focus on complex problem domain, which cannot be formulated precisely. Secondly, the methodology enables the analyst to embark on a process of learning about the real situation being investigated. Thirdly, the methodology is an approval to aid the analyst in thinking and suggesting recommendation for further action to improve the problem situation. The user of SSM will always carry out both a logic driven stream of enquiry and a cultural driven stream of enquiry. Logic driven stream of enquiry There is not much difference between the logical stream of enquiry and the earlier classic seven stage approach. The seven stage approach is summarised in figure 2. The analyst enters the problem situation. Analyst identifies primary tasks and issues. Relevant systems are subjectively chosen and modelled. A conceptual model is constructed and compared with the real world. The difference between the models and the real world is identified. Changes, which are systematically desirable and culturally feasible, are identified so that purposeful, feasible and acceptable action to improve the situation can be taken.
Figure 2. Logic driven stream of enquiry. Clasic seven stage approach. Source: Checkland and Scholes (1990)
Cultural driven stream of enquiry The cultural stream of enquiry as perceived by Checkland consists of three analysis of the problem situation; the intervention analysis, the social system analysis and the political system analysis. Checkland and Scholes (1990) recommend that the ideas of roles, norms and values are a way of making sense of the social contact, but techniques from disciplines such as anthropology or sociology and other disciplines might also be appropriate (Checkland & Scholes, 1990; Lewis, 1994) Analysis of the intervention. This analysis involves identifying possible roles in the situation such as clients, problem solver and problem owner. The role of client is usually the most straightforward to identify as this is the person or persons who caused the study to happen. The role of problem solver is given to whoever wishes to bring about improvement in the problem situation through active participation in the intervention process. Consideration of the impact that analysis and the analysis process will have on the situation being studied is needed. The role of problem owner is allocated to those individuals who will benefit from improvements in problem situation. Checkland and Scholes (1990) gave an example, in SSM study of the problems of vice in the West End of London, some immediately obvious possible problem owners would be: Parliament, the courts, the general public, the police, their family, etc. There will be many possibilities for who might be the problem owner. In analysis of the intervention, it is essential to be clear about who is being taken to occupy the above three roles. There is often some benefit to exchange role as it places a different perspective on the problem. Analysis of the Social system The SSM model views a social system as a continually changing interaction between three elements: roles, norms and values. Each continually defines and redefines and is itself defined by the other two elements (Checkland and Scholes, 1990). A role is a social problem that is recognised as significant by those in the problem situation (Checkland and Scholes, 1990). Checkland and scholes (1990) describe that the role in situation is characterised by expected behaviours or norms. They mean also that values are those standards that the performance or behaviour of a role will be judged by. Hofstede (1980, 1991)) defines culture as the collective programming of the mind that distinguishes one group from another. Defined as a set of shared assumptions, culture represent the system of socially constructed meanings and preferences a group develops as it collectively negotiates environmental forces and the complexities of internal integration (Hofstede, 1980, 1991). Restated, national culture can be interpreted as a common frame of reference or logic by which member of a society view organisations, the environment, and their relations to one another. It can be also defined as the shared values of a particular group of people. According to Lachman (1983) and Triandis (1995), national culture reflects the core values and beliefs of individuals formed during childhood and reinforced throughout life. Since national culture influences the core values formed early in life, differences among organizational cultures within a country may be limited to the organizationss influence over peripheral values (Erez and earley, 1993). Recently, scholars have begun to explore the link between national culture and policy decisions and formation of the various NII (Garfield and Watson, 1998). In this analysis, it need to be understand that direct questions will probably receive a formal responses. The SSM analyst needs to be mentally open to every conversation, interview or perusal of documents, etc. The analyst needs to review the exchanges communication with regard to roles, norms and values. Analysis of the Political System In the stream of cultural analysis, the analyst accepts that any human situation will have a political dimension, which is the process by which differing interest reach accommodation (Checkland & Scholes, 1990). Accommodating those interests is the business of politics, and the concept will apply to a group or a company even a city or a nation state (Checkland & Scholes, 1990). The analyst will study in practice by asking how power is expressed in the situation. A vital part of understanding the nature of problem situation is understanding the politics and the political dimension that operate within it. Analysis of the Political System enriches the cultural appreciation built up in analysis of Social System and analysis of intervention. All three analyses complement the work of selecting, naming and modelling relevant human activity systems and work in parallel with the logic-driven stream of thinking. The two streams of thinking complement each other and should unfold over time. Understand Case study (activity 1.2) Introduction Different research strategies represent different ways of answering questions (Yin, 1994). Each research design has its own advantages and disadvantages. Although each strategy can be used to serve exploratory, explanatory or descriptive research, the strategy selected for a research topic depends on: the type of research questions to be answered; the investigators control over the events; and the type of phenomena that becomes the focus of the research (Yin, 1994). Many IS researcher have a preference for case study and action research as the most appropriate method for the investigating the Information System field. This research uses a case study based methodology which focuses on understanding the dynamic present within a given setting (Yin, 1994). Case study approach is a research method which emphasizes in-depth understanding and qualitative analysis which is very relevant for this situation. It is an appropriate research strategy when a how or why question is being asked about set of events in their given setting (Yin, 1994). Yin (1994) claims that case studies should use multiple sources of evidence to establish a chain of evidence. Case research can be used to provide description, to generate and test theory. One aim of this research is to test the suitability of incorporating Delphi technique into SSM. Suitability of research methods In summary, the case study was considered appropriate to meet the research objective because it answered the research question, and provided the opportunity to investigate the situation in their natural setting. Design of case study method The case study is defined by Eisenhart (1989) as "a research strategy which focuses on understanding the dynamics present within single settings" (pp 534). A case study can be either a single or a multiple case study (Yin, 1994). Even within a single case study, researchers can employ an embedded design or conduct multiple levels of analysis (Eisenhart, 1989; Yin, 1994). The decision about which type of case study research design to adopt needs to be made prior to the commencement of data collection. On the other hand, multiple case studies can be used to compare the similarities and differences between cases. The findings of multiple case studies therefore, are often regarded as more robust than a single case study (Herriott and Firestone, 1983, cited in Yin, 1994). Conducting multiple case studies can demand extensive resources, including time and funding. Moreover, the number of cases selected should be carefully considered to ensure that the research question could be sufficiently answered within the research timeframe. These cases should be considered as multiple experiments, and not just simply multiple samplings. Each case within multiple case studies should be regarded as an individual case or experiment, and the analysis need to follow cross-experiment rather than within-experiment design and logic (Yin, 1994). Multiple case study research design Yin (1994) provides a detailed overview of the multiple case study method (Figure 2). This general approach was used for the study. The first step, theory development, was conducted by reviewing the literature on NII and developing a conceptual framework of problem situation. The cases were then selected and the data collection protocol was designed using the conceptual framework. Once the case studies were conducted, they would be analised gradually and case reports would be written. This analysis was then continued with a cross-case analysis to reveal the similarities and differences between cases. (this figure under construction) ANALYSE A DEFINE AND DESIGN PREPARE, COLLECT, AND ANALYSE CONCLUDE Conduct 1st Write individual Draw case study case report cross-case conclusion
Select cases Modify Develop Conduct 2nd Write individual theory Theory case study case report
Design data Develop policy collection implications protocol
Conduct Write individual Write cross- remaining case report case analysis case study Figure 3. Multiple case study method Source: Yin, 1994. Understand Delphi (activity 1.3) SSM advocates the achievement of accomodation or at the very least consensus across stakeholder groups (root definitions) as the starting point of user driven involvement. Issues of definition are of particular importance where stakeholder groups are not incorporated, meet irregularly, or have to accommodate a complex of socio-emotional perspectives (Davenport and Travica, 1995). This research incorporate Delphi technique as techniques to collect issues of NII situation. Background Olaf Helmer and Norman Dalkey developed the Delphi process in the 1950s. Both were scientist at the Rand Corporation, the Delphi technique was originally as an iterative consensus building process for forecasting futures. Since then it has been deployed as a generic strategy for developing consensus and making group decisions in a variety of fields (Linstone & Turoff 1975; Turoff & Hiltz 1996). The nature of Delphi The straightforward nature of the Delphi technique by utilizing an iterative survey to gather information sounds very simple to do. There has been diverse definitions and opinions about the Delphi technique. Some of these misconceptions are expressed in statement such as the following (Turoff & Hiltz 1996): -It is the use of a survey to collect information -It is a method for predicting future events -It is the use of voting to reduce the need for long discussions -It is a method for generating a quick consensus by a group -It is the use of anonymity on the part of participants -It is a method for quantifying human judgment in a group setting Some of the above are true but a few have different meaning to the purpose of carrying out a research using Delphi technique. The following description describe the Delphi technique and its objective in more details:
The Delphi technique were commonly applied to group of a size, for example 20 to 50 individuals, that could not function well in a face-to-face environment for reasons such as communication bias, politic, diverse area of residence, etc. Delphi objective The objective of most Delphi applications is the reliable and creative exploration of ideas or the production of suitable information for decision-making (Ziglio, 1996). There are five main objectives of using Delphi technique in a research (Linstone & Turoff 1975).
From the work of Linstone and Turoff (1975), three considerations are important for Delphi applications to issues related to problem situation.
Pill (1971) and Goldschmidt (1975) mentions that there are two options when one is working on a problem under conditions of uncertainty determined by insufficient data and incomplete theory. The first option is to wait until we have and adequate theory based on tested scientific knowledge enabling us to adress the problem concerned. The second option is to make to make the most of what is, an unsatisfactory situation and to try to collect the relevant intuitive insights of experts and use their informed judgement as systematically as possible. It is within this second option that is suggested that Delphi method can be applied in the fields of policy with the aim of generating new insights and future scenarios, assessing the desirability and feasibilty of policy alternatives; and contributing to problem solving and informed decision-making. Result of Delphi studies The Delphi method represent an organised method for collecting views and information pertaining to a specific policy area. Ziglio (1996) point out that the results of a Delphi exercise can serve any one or any combination of the following purposes:
The Delphi process
Kaynak, Bloom and Leibold (1994) point out that a Delphi study involves a number of considerations, including the selection of panelists, the design of questionnaire, the provision of feedback and a decision on the number of rounds to be conducted. An interest group is typically assembled, either through correspondence or face-to-face discussion, to assess issue(s) of mutual concern (Kaynak, Bloom & Leibold, 1994). While the individuals in the group share a common interest (the subject of the research), they (usually) represent different points of view. The panel of experts should be selected by their knowledge of the subject under review. In order to ensure a wide range of ideas and views the selected experts should not be permitted to interact with one another during the mullet round process to avoid any form of bias. Everybody in a group will then be asked to give their comments regarding a particular set of issues. A facilitator will then calculate all the individual comments and produce a report documenting the response of the group. The individuals then have to compare what they said to the groups normative response as a basis for discussion. The discussion, again via remote or face to face conversation, is used to share, promote and challenge the different points of view (Linstone & Turoff, 1975). Once this is done, the participants, having the benefit of the previous discussion, anonymously comment on the issues again. This process continues until the group reaches consensus or stable disagreement (Linstone & Turoff, 1975). Design of Delphi questionnaires A questionnaires, compiled and adapted by the authors for the subject conditions and interest, as well as inputs from other relevant personnel should be used. Design of the questions should reflect the need for the targeted respondents to think about their answer , not just simply answer the questions. The first round questionnaires (Q1) should be mailed to a panel of selected experts following an introductory letter which briefly explain the Delphi method, the research objectives(s) and how respondents co-operation will be utilized (Ziglio, 1996). The Q1 poses the problem in broad terms and invites answers and comments. In most cases, stamped, self-addressed envelopes can be included to ensure high return rate. A follow-up letter can be sent to those who have not yet reply within certain period. Prior to sending questionnaire form researcher must agree on the minimum number of respondents to allow valid research (Ziglio, 1996). The replies to Q1 are summarised and used to construct a second questionnaire (Q2) (Ziglio, 1996). Q2 presents the result of Q1 and gives the respondents an opportunity to re-evaluate their original answers in the light of comprehensive feedback on the responses of the whole group. This whole interactive process can be repeated as many times as are judged appropriate with further questionnaire; Q3 or Q4, when issue can be clarified, areas of agreement and disagreement can be identified, and understanding of the priorities can be developed (Ziglio, 1996). Two phase of Delphi In Delphi technique application, Ziglio (1996) identified two phase.
Advantages of the Delphi Technique. Rotondi, A. & Gustafson, D (1996) point out that the advantage of Delphi technique could include any or all of the following
The Delphi technique can be used for as tools for improving data collection, generation of ideas, exploration of futures scenarios and informed decision-making in problem solving situation and policy areas. The Delphi can provide a very important tool for decision-makers facing uncertainty by: exploring the nature of particular problem situation or impact of policy; assessing its magnitude; and evaluating different possible ways of adressing it (Ziglio, 1996). The result of the Delphi technique can greatly assist policy makers to improve creativity in their decision making when accurate information is unavailable (Ziglio, 1996). Understand NII (activity 1.4) A nations infrastructure is defined by its basic communication and transport capacity. Porter (1990) points out that: " A nations industry depends on modern and improving infrastructure. This is particularly true in advanced transportation, logistic and telecommunications, all integral to introducing modern technologies and to competing in foreign markets." Many governments are refocusing their attention from traditional road and railway infrastructure development to electronic communication and other transport project (Gubaxani, 1990). Example of this phenomenon is national effort of SouthEast Asia countries to establish information super highway or a national information infrastructure (NII), which may change the way its citizen create and communicate knowledge. And the fact that other developed nations, such as USA, Japan, German and UK, are continue doing with creating of NII. For example, Japan have commited to the fiber-optic connection of every home, school and business by the year 2015. Building an advance national information technology infrastructure (NII) has been argued to provide competitive advantage for the countries that develop it and also for the companies that operate in those countries (Cats-Baril and Jelassi, 1994; Porter 1990). An NII is of national significant because it is potentially a very significant contributor to economic wealth in the emerging age of electronic commerce (Garfield and Watson, 1998). Establishing an NII requires political leaders to make key decisions on its structure and management (Gartfield and Watson, 1998). The push for Asian countries NII project is being led by their countries leader who point to the growing importance of the information processing sector of their countries economic. Researchers have attempted to explain the degree of success of national information technology projects by analyzing the culture, political and economic environment of those countries (Cats-Baril and Jelassi, 1994; Garfield and Watson, 1998) The study covers three IS projects at a national level in SouthEast Asian countries (Indonesia, Malaysia and Singapore). The selected countries have similar social and cultural backgrounds. The countries have highly regulated economic environments and strong government policies in their Information Technology field. Each of the three countries is gradually moving towards open and deregulated economic environment. The national IS project or National Information Infrastructure selected for studies are: Indonesia-Nusantara 21 administered by PT Telekom, Singapore-Singapore One administered by National Computer board, Malaysia-Multimedia Super Corridor administered by Multimedia Development Corporation. All of these IS projects are at the national level. Identify initial issues (activity 2.1 and activity 2.2: apply SSM to NII and identify list of NII issues) Data are gathered from a variety of sources; government NII policy documents, many of which are electronically available, archival records; newspaper and magazine articles, and primary and secondary interview sources (written records of interview). Specifically, written records of speeches made by government officials and state documents identifying NII policies. The focus of these activities is to find a starting point for identifying the set of issues of NII situation. Gather data of NII situation and apply Delphi for prioritizing list of NII issues Select participant and facilitator (activity 3.1 and activity 3.2) The Delphi expert and facilitator who will participate in this research will be selected from government institute or private company personnel who are involved in the NII. The participant expected to be diverse categories: government institute, education institute, Multi National Corporation, Domestic company etc. Prior to the first round of the Delphi study a list of stakeholder groups (organisations) in each of the three countries is compiled using the directory of organisation in country NII document. In order to involve organisations with a substansial interest in this research, analyst asks the contact persons of the organisations to nominate two key person in the organisation who would participate in the study. Criteria of selected panel:
Select the medium to be used to contact participant (activity 3.3) There will be three round of Delphi study. On Round One of Delphi, selected participant will be presented questionnaire in the E-mail format. On the Round Two of Delphi, facilitator will interview participants on their issues. On Round Three of Delphi, questionnaire will be presented in the E-mail. It will attempt to put questionnaire form and the result on website. Criteria of medium:
Structure the Delphi questionnaire (activity 3.4) The prototyping literatures and activity 2 provides list of issues for Round One of Delphi. A questionnaires (Q1), should be compiled and adapted by the analyst for the subject conditions and interest, as well as inputs from other relevant personnel should be used. Design of the questions should reflect the need for the targeted respondents to think about their answer, not just simply answer the questions. The Q1 poses the problem in broad terms and invites answers and comments. A follow-up E-mail can be sent to those who have not yet reply within certain period. The replies to Q1 are summarised and used to construct a second questionnaire (Q2). Q2 presents the result of Q1 and gives the respondents an opportunity to re-evaluate their original answers in the light of comprehensive feedback on the responses of the whole group. The replies to Q2 are summarised and used to construct a third questionnaire (Q3), issue can be clarified, areas of agreement and disagreement can be identified, and understanding of the priorities can be developed.. Send the questionnaire to participant (activity 3.5) There will be three round of Delphi study. On Round One of Delphi, questionnaire (Q1) will be sent to selected participant in the E-mail format. On Round Three of Delphi, questionnaire (Q3) also will be sent to selected participant in the E-mail format. Only two round from this three round Delphi study that the participant will receive questionnaire in E-mail format. Comment on issues (activity 3.6) The participant (stakeholder) is asked to comment on issues by answer the questionnaire (Q1 and Q3) and participate in the interview of Q2. Criteria for commenting on issue:
Analyse on comment (activity 3.7) Facilitator calculate responses from every Delphi Round and summarise the result, including mean rank of issues and the number of times an issue is marked unimportant. Facilitator ranks the list of issue in mean importance sequence. Report to participant (activity 3.8) Following the previous activity, an analysis of mean rank, percent of participants ranking an issue in the top of the list, indicates a sufficient level of consensus attained, will be reported to participants. The report will be included in the next questionnaire or interview. Interview participants on issues (activity 3.9) Round Two involves semi-structured interview with a participant from Round One. Round Two interviews is employed to provide support and clarification for the Round One findings and Round Two questionnaire (Q2). The Round Two begin the process of achieving agreement. Publish result of prioritized list of NII issues (activity 3.10) Analyst summarise and publish the final result of Delphi study , including mean rank of issues, percent of participant ranking an issue in the top of the list and the summary of all the issue is given during the study. Select a relevant system and develop conceptual model (activity 4) (5) compare conceptual model with the NII situation, (6) continue to debate desirable and feasible changes and (7) finally recommend action. At the end of Delphi study, the analyst can further continue with SSM analysis based on the list of issue from Delphi study. Relevant systems are subjectively identified and root definition of particular system could be derived from the outcome of the previous activity. A root definition clarifies two aspect of the area of concern for further analysis: first is identification of the NII problem situation, what requires to be addressed, and second it identifies the system in which the subsequent analysis will be done. The formulation of a root definition indicates that the expressed unstructured problem situation has been structured to enable systemic analysis. It is the root definition which adds structure to problem situation. It is possible to derive as many root definitions as there are particular weltanshauungs or actors. Having defined the roots of the model, the next stage is to illustrate it in a manner that shows the relationship between activities required of the system: via the Conceptual Model. The model will be in form of a drawing or diagram. The conceptual model shows those components (sub-systems or activities) which are logically necessary to achieve the transformation described in the root definition. Criteria of Conceptual Model:
Constructing a model will always means going through several drafts, and several levels of resolution. Once a first-resolution version model has been drawn, the process of decomposing begins: the system is divided into sub-systems, and root definitions and conceptual models are developed for each of them. In the process of hierarchical decomposition, it is important that lower-level systems describe how the activity described at the higher level is achieved. Once the analysts are satisfy with the conceptual model, it is time to move from the Systems World to the Real World again Compare conceptual model with the NII situation (activity 5) The comparison is carried out by taking the models across the Systems World/Real World line in the SSM stages diagram, and using them to compare what is desirable from a systems point of view with what is happening in real world. Mismatches between the systems model and the real world might indicate problems and/or where improvements could be made. The output from this stage is a list of system activities, the corresponding activities in the real world, and the differences. Changes which are systematically desirable and culturally feasible, are identified so that recommendation of action to improve the situation can be taken. Debate desirable and feasible changes (activity 6) There are three types of changes possible: Changes to structure, procedures and attitudes. The decisions about which changes to make are done after discussion or debate among the stakeholder of NII situation. The analyst should make it clear what type of change is required where/from who, and how/which structures, procedures or attitudes might be affected. Changes must be described in systems terms or logical terms and culturally feasible. Recommend of action to improve the NII problem situation (activity 7) After debate of desirable and feasible changes has been completed, a recommend of action is taken stating what should be done and how it should be done to improve the NII problem situation. Define efficacy, efficiency and effectiveness, Monitor activities and take control action (activity 8,9 and 10) Activity concerned with deciding what control action to take With criteria :
Activity concerned with collecting information about the system. Activity concerned with taking control action References Bennet, P. On linking approaches to decision-aiding: issues and prospects. Journal of the operational research society, 1985, 36, 659-670. Checkland, P. and Scholes., J., Soft System Methodology in Action. Wiley, Chichester, 1990 Flood, R., and Jackson, M., Creative Problem Solving. Wiley, London, 1991 Flood, R., Solving Problem Solving. Wiley. Chicester, 1995 Jackson, M., Beyond a system of system methodologies. Journal of the operational society, 1990, 41, 657-668 Kreher, H., Critique of two contributions to soft systems methodology. European Journal of Information Systems,1993, 2,304-308. Lewis, P., Information Systems Development. Pitman Publishing, London, 1994 Mingers, J. and Taylor, S., The use of Soft Systems Methodology in practice. Journal of the operational research Society, 1992, 43, 321-332. Mingers, J. and Brocklesby, J., Multimethodology: Towards a Framework for Mixing Methodologies. Omega, 1997, 25, 489-509. Ormerod, R., Putting soft OR methods to works: information system startegy development at sainsburys. Journal of the Operational Research Society, 1995, 46, 277-293. Ormerod, R., On Nature of OR-entering the fray. Journal of the Operational Research Society, 1996, 47, 1-17 Ormerod, R., Combining management consultancy and research. Omega, 1996, 24, 1-12. Ormerod, R., The design of organizational intervention, Omega, 1997, 25, 415-435. Savage, A. and Mingers, J., A framework for linking Soft Systems methodology
Watson, R., and Smith, R., Application of the Lancaster Soft Sytems Methodology in Australia. Journal of Applied systems Analysis, 1988, 15, 3-26. Yin, R.K., Case Study Research: Design and Methods. Sage Publication, London, 1994
Bukley, C. (1995). Delphi: A methodology for preferences more than predictions, Library Management. (16)7: 16-19. Dalkey, N.C. and Helmer, O. (1963) An experimental application of the Delphi Method to the use of experts. Management Science. 9, pp. 458-467. with references to Buckley (1995) Delbecq, A.L., VandeVen, A.H. and Gustafson, D.H. (1975). Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes, Scott-Foresman & Co. Goldschmidt, P. (1975) Scientific inquiry or political critique? Remarks on Delphi assessment, expert opinion, forecasting and group process by H. Sackman. Technological Forecasting and Social Change. 7, pp.195-213. with references to Ziglio (1996) Kaynak, E., Bloom, J. & Leibold, M.(1994) Using the Delphi technique to predict future tourism potential, Marketing and Intelligence & Planning. (12)7: 18-29. Linstone, H.A. & Turoff, M. (1975) The Delphi Method: Techniques and applications. Addison-Wesley Publishing Company Inc. Pill, J. (1971) The Delphi Method: substance, contexts, a critique and an annotated bibliography. Socio-Economic Planning Sciences. 5, pp. 57-71. with references to Ziglio (1996) Principia Cybernetica Web. Delphi Method. http://pespmc1.vub.ac.be/ASC/DELPHI_METHO.html (April 20th, 1998) Rotondi, A. & Gustafson, D. (1996). Theoritical, Metodological and Practical Issues Arising out of the Delphi Model, in M. Adler and E. Ziglio, Gazing into the Oracle: The Delphi method and its application to social policy and public health. London, Jessica Kingsley Publishers, pp. 34-55. Turoff, M. and Hiltz, S.R. (1995). Computer-Based Delphi Processes, in M. Adler and E. Ziglio, Gazing into the Oracle: The Delphi method and its application to social policy and public health. London, Jessica Kingsley Publishers, pp. 56-88. Ziglio, E. (1996). The Delphi Method and its Contribution to Decision-Making, in M. Adler and E. Ziglio, Gazing into the Oracle: The Delphi method and its application to social policy and public health. London, Jessica Kingsley Publishers, pp. 3-33.
For More Information Contact: |
Send mail to o.b.bintoro@massey.ac.nz
with questions or comments about this web site.
|