As you use statistics in the classroom or read educational journals, it isimportant to remember that there may be some disadvantages to usingstatistics.

Statistics are limited by the accuracy of data. If inaccurate data are analyzed,then inaccurate statistics will result.

Statistics are only as reliable as thepeople who use them, and are of no use if one does not understand how tointerpret them.

It is critical to aware of the sources of the data generated, and the typeof statistic used to analyze the data. Without this information the resultspresented may be meaningless

This section will give you a start at looking critically at statistics and experimental design.

 

Variables

Independent, Dependent, and Confounding

A variable is a quantity that changes. You may remember talking about independent and dependent variables when you were learning how to read graphs in school. The independent variable was the x-axis (horizontal) label and the dependent variable was the y-axis (vertical) label.

There is another way to determine the independent and dependent variables. The independent variable is the quantity that you, as the experimenter,control. The value of the independent variable depends only on what you determine that it should be. The dependent variable is the unknown. Itis what the experimenter measures. It is the quantity that changes when you change the independent variable. A common research goal is to determine the relationship between two variables.

 

 

Example

Suppose you wanted to know whether tall people are better basketball players than short people. You gather a sample of people of various heights and ask them to shoot baskets from the foul line. The number of baskets they make out of ten shots will be your measure of basketball ability.

The variables in this example are HEIGHT and NUMBER OF BASKETS. The height of the people in your sample cannot be changed- by choosing these particular people, you have determined the values of the independent variable.

The number of baskets your subjects will make is unknown. Number of baskets is the dependent variable.

It is very important that a statistician not make inferences or predictions based on statistics, if other factors can influence results. For example, a teacher could not make a good prediction of future test results, based on past statistics, if half the class will be up all night before the next test protesting an austerity budget. If a student scores ten points below the mean on a test a statistician cannot interpret that as lack of study, since many factors could account for a low score. These factors are called confounding variables. A confounding variable is a variable the experimenter did not take into consideration and which could account for the observed relationship between the independent and dependent variables.

Going back to our basketball example above, supppose you found that tall people did make better basketball players than short people. A reason for this could be that the tall people in your sample had more practice with foul shots than the short people in your sample. AMOUNT OF PRACTICE is a confounding variable.

Quatitative data are anything that can be expressed as a number, or quantified. Examples of quantitative data are scores on achievement tests,number of hours of study, or weight of a subject. These data may be represented by ordinal, interval or ratio scales and lend themselves to most statistical manipulation.

Qualitative data cannot be expressed as a number. Data that represent nominal scales such as gender, socio economic status, religious preference are usually considered to be qualitative data.

Both types of data are valid types of measurement, and both are used in education journals. Only quantitative data can be analyzed statistically, and thus more rigorous assessments of the data are possible.

Sampling is the process researchers use to determine who should participate in their study. A sample should be representative of the population. For example, TV shows are rated by the Nielsen company. Nielsen does not check out every home in America (the population) to see what people are watching. Rather, a sample of American homes is selected to represent the entire population. In order for the Nielsen ratings to really reflect America's TV viewing habits, all types of Americans must be represented in the sample, and in the same proportion that they exist in the population. Choosing Nielsen families from a list of credit card holders, for example,automatically excludes all people who do not have credit cards, and is not representative of the population.

Sample size is an important factor in statistics. The larger the sample size, the more likely it is that the results are reliable. A small sample size is an indicator that the results may not apply to the population as the experimenter intended.

A sample should also be drawn at random, to avoid any intentional or unintentional bias in choosing the subjects. A random sample means that each person in the population has an equal chance of being chosen for participationin the sample.

These two terms, reliability and validity, are often used interchangeably when they are not related to statistics. When critical reader soft statistics use these terms, however, they refer to different properties of the statistical or experimental method.

Reliability is another term for consistency. If one person takes the same personality test several times and always receives the same results, the test is reliable.

A test is valid if it measures what it is supposed to measure. If the results of the personality test claimed that a very shy person was in fact outgoing, the test would be invalid.

Reliability and validity are independent of each other. A measurement maybe valid but not reliable, or reliable but not valid. Suppose your bathroom scale was reset to read 10 pound lighter. The weight it reads will be reliable(the same every time you step on it) but will not be valid, since it is not reading your actual weight.