Sunday, February 27, 2011

Beware the causes!

I have read two particular studies in the past few days, one for this class and another for a Guided Study in International Human Resources, in which the authors "point blank" warned against making assumptions about causality based on the data they collected. Both were designed to assess correlations between variables so I’ll first discuss the dangers of implied causation in correlational studies as they were highlighted by our textbook author. First, Dr. McMillan points out that even if a positive or negative or predictive relationship between two or more variables is proved, the causality of that relationship should not be assumed because a correlation is not designed to provide evidence about causal direction.

This particular problem was illustrated in an article about the factors that influence the work role adjustment of an expatriate manager in Japan. In his study, Black (1988) hypothesized that the level of family’s adjustment is correlated to the manager’s adjustment but he backed away from trying to assess causal direction because the family’s adjustment (or lack thereof) could be caused by the manager’s adjustment (or lack thereof) just as easily as the manager’s adjustment could be caused by the family’s adjustment.

The second study that mentioned causality as a limitation is one that I am reading for my research proposal, about the relationships between several individual and organizational characteristics and organizational citizenship behaviors (OCB) in schools. At the end of the article, the authors (Somech & Ron, 2007) suggest that since they used a cross-sectional survey for their study, they were not able to assess the extent of a causal relationship between the variables and the OCB that they otherwise were able to show were correlated.

The other warning that Dr. McMillan issues about inferring causation from correlational studies is more so a warning about taking actions based on observed correlations. Again, going back to the studies I’ve picked as examples, if one were to deploy 100% of available resources to address and improve the family’s adjustment to the international assignment when it is actually the manager’s issues that are causing the family’s disorder then the organization is misallocating its resources. And based on the OCB study, if it were determined that OCB precede the evidence of organizational characteristics, policies that focus resources on replicating the related organizational characteristics will be misaligned.

This assumption of causation is equally a problem in comparative studies. Even if a study shows that there are significant similarities or differences between groups when certain variables are assessed, it does not mean that the groups are similar or different because of the independent variable. To illustrate this, I’ll use an aspect of my own planned research proposal. If I were to execute a comparative study only and I determined that the OCB reported by teachers who receive merit pay incentives is lower than those teachers who do not receive merit pay, it would be very unwise to stop there with a conclusion that merit pay reduces teacher OCB. The lower OCB observation could very well be related to a different characteristic or variable that I have not even included in my study. My conclusion then should be to propose research that digs deeper into other causes or implications of OCB and how those might also interact with merit incentives.

Citations for the studies used as examples above are:

Black, J. S. (1988). Work role transitions: A study of American expatriate managers in Japan. Journal of International Business Studies, 19(2), pp. 277-294.

Somech, A., & Ifat, R. (2007). Promoting organizational citizenship behavior in schools: The impact of individual and organizational characteristics. Educational Administration Quarterly, 43(1), 38-66. doi:10.1177/0013161X06291254

1 comment:

  1. Two things are really insightful here:
    When you write, "since they used a cross-sectional survey for their study, they were not able to assess the extent of a causal relationship between the variables and the OCB that they otherwise were able to show were correlated" you have pinpointed the essential role of the research design in allowing (or not) one to draw conclusions.
    The other arises when you write, "The lower OCB observation could very well be related to a different characteristic or variable that I have not even included in my study," since that acknowledges the reality of alternate explanations (threats to internal validity). Great explanations!

    ReplyDelete