Sampling

My review of literature relating to merit pay in education is only a fraction complete. I’ve already taken a few wrong turns on my way to different grocery stores, all in the name of looking for the right fruit.

I am settling now on the concept of teacher organizational citizenship behavior (OCB) and how it is impacted by merit pay compensation. I intend to look further at research already available regarding the components, causes and moderators of OCB, in teachers and other professionals. I also will review the efficacy of using existing instruments to measure OCB in teachers. At this point though, I hypothesize that merit pay compensation programs will moderate the OCB of public high-school teachers in the United States.

I believe that the interaction between merit pay and teacher OCB will eventually be best explored through a mixed-methods analysis taking both a quantitative and qualitative approach to the problem. However, I propose to start with a quantitative study that compares the level of OCB in teachers who are compensated via a merit pay system to teachers who are compensated via a fixed salary schedule. I will discuss and debate the various sampling procedures here as if I am undertaking that particular research. In this case, my sampling frame will need to be much narrower than my target population of all public U.S. high school teachers because I must account for other environmental and organizational culture factors that may moderate OCB in the experimental and the control groups. Also, specifically for the experimental group, I will need to evaluate the design of the compensation plan to determine whether any extra-role behaviors actually become in-role behaviors according to the performance targets established by the merit pay plan.

The sampling procedure that I use will depend largely on the availability and size of a sampling frame to which I can gain access. If I can find a sizable school district that has a large enough population of high school teachers who are compensated via merit pay as well as a population of teachers in the same schools who are not being compensated with performance incentives, I may be able to utilize some probability sampling procedures. Assuming such a survey population is available, I could use a simple random or systematic procedure to randomly select subjects for the sample. The risk in taking these approaches is that I may not get a representative sample, i.e. male and female teachers or teachers with varying experience or education, which will bias the measurement OCB and thus, my results. To solve for this, again assuming that the sample frame is large enough, I would first identify and divide the population into different subgroups related to the dependent variable – OCB. For example, I would identify subgroups by gender, years of education and/or experience, subject taught, school taught in, etc. for each of the experimental and control populations. I could then take a proportional stratified sample from these samples or if I don’t have enough participants represented in each subgroup, I could take a disproportional stratified sample and then weight the results to reflect the whole population.

Using a cluster sample from within the district might be more appropriate, that is I could find one particular high school within the district that has teachers who are compensated with merit pay and those who are not. Although limiting the sample frame in this way may reduce the validity of the results, it may help to control for some of the environmental and organizational culture factors that moderate OCB.

In the case that I am not able to gain access to a school district large enough to support probability sampling, I will need to rely on nonprobability sampling. Using a convenience sample procedure, I would be identify a particular high school which has instituted a merit pay plan for some but not all of its teachers and then measure and analyze the OCB of teachers in that particular school. In the unexpected case that I find one such high school that meets these criteria yet is also big enough to afford me the chance to sample certain subgroups, I could effect a quota sample and select participants who represent the subgroups mentioned earlier, like gender and teaching experience.

If I were to pursue a mixed-methods analysis of merit pay and teacher OCB, I could proceed to use some purposeful sampling procedures like typical or extreme case sampling. In each of these, I would define and identify the “average” (for typical case) or “atypical” (for extreme case) high school teacher from a particular school division or high school and measure the OCB exhibited by those who are and are not compensated with merit pay. If I find both the “average” and the “atypical” teacher, I can effect a maximum variation sample.

In the case of this research proposal, a snowball sample seems inappropriate. I believe that there would be too many different cultural and environmental factors at play in such a network sample which would render the OCB measurements and conclusions invalid. In the same way though, I think that to control for these same factors, an element of critical case sampling will be valuable in this study. That is, I think the study will ultimately need to identify one particular school district or even one particular high school from which an experimental and a control group of teachers can be drawn.

If I use nonprobability sampling procedures and an element of critical case sampling as currently planned, I know that I won’t be able to generalize any discoveries about the relationship between teacher OCB and merit pay to the larger US high school teacher population. However, I still think that the results will be significant for the school district that is sampled and that the data obtained will inform us about how merit pay and teacher OCB interact. At the least, it will suggest complementary future studies.