Where do you want to go?

Part A: Soundness of Design In acknowledging the similarity of purpose of both quantitative and qualitative research, I will proceed to describe the soundness of my methodology. To do so, I will again compartmentalize my discussion into two sections; 1.) assumptions of the qualitative approach, and 2.) feasibility of the design.

Section 1: Assumptions of the Qualitative Approach The qualitative approach must submit itself to the same types of rigor as traditional research, however, because the paradigms are inherently different, a new conceptualization of that rigor is necessary (Lincoln, Guba, 1985). Regardless of which approach is taken, internal validity remains "truth value". External validity is still applicability or generalizability. Reliability denotes consistency, and objectivity is seen as neutrality. The conventional semantics of rigor, however, may induce a reviewer of qualitative research to judge it in conventional terms. Herein lies the controversy. Lincoln and Guba offer another set of terms. "The four terms credibility, transferability, dependability and confirmability were established as the naturalists’ equivalents for…internal validity, external validity, reliability and objectivity" (1988, p.3). The terms were adopted for the single purpose of making clear "the inappropriateness of the conventional terms when applied to naturalistism and to provide a more logical and derivative relation to the naturalistic axioms" (Lincoln, Guba, 1985, p.301).

(I was unable to transfer a table into HTML so the following is a bit off)

Conventional Paradigm <--> MEANING <---> Emergent Paradigm

Internal Validity <--> "TRUTH VALUE" <--> Credibility

External Validity <--> GENERALIZABILITY <--> Transferability

Reliability <--> CONSISTENTCY <--> Dependability

Objectivity <--> NEUTRALITY <--> Confirmability

Credibility’s goal is to demonstrate that inquiry was conducted in a way which ensures the subject was accurately described. An inaccurate description is the single-most important threat to validity. The "truth value" of a depiction can only be decided by the one depicted. For each interview conducted, I will be providing a "portrayal" of the interview to the interviewee. While a full verbatim transcript will be kept for coding and analysis, the portrayal is more a general impression of how the respondent answered. Key points will be brought up and summarized for the subject to comment on. This is not to be confused with interpretation. That will remain the sole responsibility of this researcher. The portrayal is more on the order of authentication by the interviewee.

Validity [in qualitative research] is not a commodity that can be purchased with techniques…rather, validity is like integrity, character, and quality, to be assessed relative to purposes and circumstances (Brinberg and McGrath, 1985, p.13).

Transferability is the second term that needs clarification. In the conventional sense, generalizability is the degree an account of a particular situation can be extended to other situations. It is embodied in the context-free and time-free laws that apply to human behavior. This is in itself a controversial proposition for social scientists. Noted author and statistician Lee Cronbach poses the question "Should social science aspire to reduce behavior to laws?" (Cronbach, 1975, p.212). He cut away significant underpinnings for conventional paradigms when he points out that the generalizations that result from tested hypotheses "decay" with time (Wehlage, 1981). Transferability on the other hand means the extent to which a case "facilitate the drawing of inferences by the reader that may have applicability in his or her own context or situation" (Lincoln et al, 1985, p.20). Transference takes place between contexts A and B. If B is sufficiently like A on factors that possesses, then the inquiry may be found to be significant and transferable. The saliency of those factors would vary from inquiry to inquiry.

This is not to imply that qualitative transferability lacks rigor. Just as quantitative research defines or operationalizes variables for consideration, good qualitative research provides the contexts or elements by which comparisons are made by A and B. In the case of this study, the frameworks of agenda-setting theory, decision-making models, and the implications of bureaucratic organization are structures set up explicitly for comparison. These are pre-set parameters, however, the nature of qualitative research is one of constant discovery. This is not a "controlled" experiment. Other elements are allowed in as the research proceeds. Indeed, the researcher would be remiss to ignore or dismiss alternative frameworks.

Maxwell (1992) brings both concepts of internal validity (credibility) and external validity (transferability) together under the terms, internal generalization and external generalization. While the latter conforms with the above description, he sees the former, internal generalization, as the more pertinent issue. By internal generalization, he means the ability to apply inferences from a small sample group to the whole group. This is somewhat akin to controlling for representativeness of sample, but whereas representativeness is considered a priori in the planning of study design, internal validity/generalizability occurs during the interpretation of the data. More on this as I later discuss verifying results in the data analysis section.

Reliability denotes consistency. In the conventional sense, this deals with reproducibility of the study. While appearing deceptively simple, there are many dimensions to reliability; inter-observer reliability, intra-observer reliability and test-retest reliability are just some of the different aspects. Basically the measurement of reliability rests on the ratio between the variability between individuals to the variability of the scores. This can be accomplished by various statistical methods such as Cronbach’s alpha and Kuder-Richardson. "However, they do not take into account any variation from day to day or from observer to observer, and thus lead to an optimistic interpretation of the true reliability of the test" (Streiner, Norman, 1995, p.7).

"Qualitative research does not pretend to be replicable" (Marshall, Rossman, 1989, p.148). The nature of my study has been described earlier as one where the elements are fluid through time. Any attempt to measure a snapshot in time would prove to be erroneous in both depiction and explanation. First, qualitative research by its nature cannot be replicated exactly because the real world changes. This is not to deny its importance though. Again, while recognizing this as a general axiom, I will seek to provide a framework for replicability.

Replicability applies not only to data, but also to the processes that I will use. It should apply to data "so that we can whether our measures are reliable, but [also] to the entire reasoning process used in producing conclusions" (King et al, 194). Remembering that inferences are made from data, a thorough documentation of bibliography and footnotes (secondary data) is necessary so that a subsequent researcher can retrieve this data and make his/her own inferences. As in quantitative research, this provides access to the data set. Even more important is the primary data that is collected. Tapes, verbatim transcriptions and documents used to make inferences from should be made available. Both of these can be construed as the raw data.

Equally important is the methodology. While the collection and analysis of data through interviews may appear to be less than rigorous, I will provide a set of rules such as coding and sorting that will be adhered to. I will elaborate more on this as I discuss it in the analysis section. The structure of an a priori coding scheme will be used so that a future researcher or reviewer can duplicate the process. I will, however, reserve the right to make adjustments and additions to that scheme as research is underway. As the process of collection and analysis are simultaneous in the qualitative paradigm, good research may open up new insights and unanticipated ideas and concepts (Lofland, 1977). To account for this, a detailed research log will be kept during the duration of the project. In this way, field decisions for altering strategies or substantive focus, the rationale for new directions, and new insights will be documented (Marshall et al, 1989). From these artifacts, a reviewer can accept or challenge the directions that I have taken.