Type of studies

There are three types of studies utilized in the process of scientific research, corresponding roughly to the three major purposes of science. The first type is descriptive in which the researcher attempts to document what is actually occuring. The study may be either qualitative (descriptions in words) or quantitative (descriptions in numbers). The researcher has no control over the phenomena of study, but simply records what is observed or reported.

The second type of study is referred to as correlational. In this study, the research relates the level of one variable to a corresponding level of another variable in an attempt to discover any relationships between them. The purpose of this type of study is to predict the level of one variable by knowing the level of a second variable. The researcher has only moderate, if any, control over the variables in this type of study.

The third type of study is referred to as experimental. In this type of study, the researcher manipulates the level of one variable and observes the corresponding change, if any, in the level of another variable. The purpose of this type of study is to determine if there is a causal relationship between the two variables.

Experimental studies

An experiment consists of assigning participants to at least two groups, i.e. experimental and control, and administering or manipulating an independent variable in the experimental group while holding conditions constant and equivalent for the control group. The groups (assumed to be equivalent in all respects initially) are then compared on the dependent variable (performance or change in behavior) to determine the likelihood or probability that the independent variable caused these changes in the experimental group. Note the phrase "likelihood or probability" is used since the experimenter is NEVER absolutely certain that the independent variable was the actual "cause" of the observed changes. Alternative explanations exist. Among the most obvious is the possibility that the two groups differed prior to the administration of the independent variable.

There are three fundamental ways that such an occurrence may be ruled out. Subject assignment to groups can increase the likelihood that there are no important initial differences between the experimental and control groups. The subjects may be randomly assigned to the groups thus distributing individual differences evenly across the groups or they may be assigned on the basis of matching on important factors such as age, ability, gender, SES and others. It has been argued that random assignment, particularly where the groups are large, is a more powerful technique and more likely to assure nonsystematic differences between groups, i.e. important extraneous factors are randomly distributed in both groups.

The second method of equating groups consists of statistical accommodations applied to groups consisting of different individuals. Initial group differences may be assessed by pretesting and, if found to be different on the dependent variable prior to applying the independent variable, then using the analysis of covariance to test the differences between groups on the posttest data (same measure) provides a statistical adjustment in the final results based upon initial group differences. The statistical procedure has been characterized as asking the question "What would the data be like if they weren't like they are?" Such a procedure is superior to examining gain scores and is most often used where random assignment of subjects to groups is not possible because the groups are intact already, e.g. educational groupings,or the study is addressing the effect of the treatment on so-called "classification variables" which cannot be manipulated, e.g. gender,  and the like.

A third way in which initial group differences may be avoided is achieved by controlling or eliminating them through the experimental design. In the definition of an experiment stated previously, called a two group design, it is assumed that different individuals are assigned to the two groups. Using the same individuals for the experimental and control groups assures group comparability and importantly decreases error variance which results in a clearer picture of the impact of the independent variable. If individuals are to be used twice, then exposure to either condition initially, i.e. order of conditions, could change the results such that order alone could be the causative agent rather than the independent variable. Therefore, the experimental design provides for counterbalancing the order that the subjects receive the conditions.

For example, half of the subjects receive the control condition first followed by the experimental condition while the other half receive the experimental condition and then the control. Whatever the potential effect of having a particular order of conditions has on the outcome is canceled or offset by combining the results for both experimental subgroups (E then C; C then E) and both control subgroups. However, sequence of conditions could interact with the independent variable in a complex manner to produce misleading results.

Research Designs

A excellent overview of experimental and quasi-experimental research designs is provided by Dawson (1997). In general, "true" experiments involve the manipulation of variables and the assignment of subjects in a manner which assures group comparability. They are preferred over other approaches since there are few threats to either internal or external validity, i.e. the validity of the experimental conclusions. Quasi-experiments are efforts to assess the impact of variables over which the experimenter does not have control and/or assessing the effect of a variable under circumstances where subject assignment to groups is not controlled. The accompanying tables developed by Campbell and Stanley (1963) illustrate the differences among "preexperimental" designs which are at best, demonstrations of possible effects of a variable, and true experimental and quasi-experimental designs.

Two quasi-experimental designs widely used in developmental and educational psychology are the longitudinal and cross-sectional designs. Both approaches have their advantages and disadvantages. The longitudinal, a favorite among early developmentalists, is expensive, time consuming, and subject to bias in sample selection, attrition of subjects and experimenters, testing, history, statistical regression, and instrumentation problems. Its strength derives from the singular capability of tracing the development of unique traits across time within the same individuals. The cross-sectional approach avoids many of the problems of the longitudinal, but is subject to risk from the cohort effect, particularly where the age differences among groups are large, e.g. 10 years. This, of course, allows extraneous factors to creep into the data which falsely suggest change due to development rather than cultural events. In an effort to either control for or assess the extent of the above possibilities associated with these designs (principally history vs. cohort effects), the following designs have emerged.

Cohort-Sequential Design. The cohort-sequential design combines two or more longitudinal designs, each of which covers the same age groups but at different points in time. This design offers an opportunity not only to assess age changes, but to measure potential cohort effects and the interaction between age and cohort effects. Table 2.2 illustrates a comparison of 60- and 70-year olds with the initial data gathered beginning at two time points (1960 & 1970) and the subjects retested after 10 years elapsed. Testing mean differences between rows provides a measure of cohort effects, i.e. did it matter which decade one was born in? Testing column means reveals age effects-the effect desired without contamination from extraneous factor. The interaction between rows and columns, if significant, indicates that comparisons for one age were affected by cohort factors while those for another were not. This study would require 20 years to conduct- a definite disadvantage. Additionally, the history threat is also attendant to this design as in the simple longitudinal approach. Table 2.2* illustrates an incomplete Cohort-sequential design which may better clarify the manner in which samples are tested.

Table 2.2* Cohort-sequential Design Allowing Cohort and Age Tests

Time-Sequential Design. The time-sequential approach, depicted in Table 2.3, is likened to the cross-sectional method.

Cross-Sectional Design. Two or more cross-sectional studies are conducted each beginning, in this instance, 10 years apart and each using 60- and 70-year-subjects; the four cells are comprised of different subjects. History effects may be measured in addition to age effects and the possible interaction between the two variables as well. The interpretation of row and column main effects as well as the row X column interaction is conceptually the same as for the previous design. This study requires only 10 years to complete but has the disadvantage associated with the cohort risk. Cross-sequential designs have both longitudinal and cross-sectional features. Age effects are not separated from either cohort or history effects; rather cohort and history effects are separated from each other. Two or more age-different cohorts are tested at least on two occasions, say 10 years apart. Table 2.4 illustrates the design.

Table 2.4 Cross-sequential Design

The logic of testing row (cohort) effects would be assess the effect of extraneous events on the dependent variable however with different mean ages across the two rows, no conclusions are possible with respect to cohort effects. Similarly, column means reflect history effects but they too are confounded by having individuals of different mean age within each column, i.e. column 1 mean age-65 vs. column 2 mean age=75.

Schaie's "Most Efficient" Design. Finally, there is Schaie's "most efficient" design which combines all the features of the first three approaches. Figure 2.1 graphically demonstrates the facets of this most complex approach. The most important feature is that one can test age, history and cohort effects although the picture is complicated by the possibility that all three factors may show strong effects. That, of course, leads to rather weak conclusions.

The most important aspect of this presentation for developmental and educational psychology are the threats to the validity (both internal and external) of experimental conclusions since developmental and educational researchers are usually interested in studying classification variables or intact groups such that quasi-experimental designs are often involved.

Resources:

 

|