The False God of Statistics

After reading Kevin Watson’s recent posts about small groups, I found myself once again opening up the Call to Action documents to investigate their treatment of small groups.

And I was reminded why reading the Call to Actions documents is bad for my soul.

Here is my simple challenge for the bishops and other leaders of the denomination. I’d like anyone using this report as a catalyst for change in the UMC to answer the following questions:

1) What is factor analysis?

2) What is the difference between confirmatory factor analysis and exploratory factor analysis?

3) What is an eigenvalue? What is a scree plot?

4) What is multiple linear regression analysis?

5) What is multicollinearity?

6) Can a correlation prove causation?

7) What does the term “statistically significant” mean?

Does a person have to be a statistical expert to use statistical data? No. But it helps to understand the basics – which I’m pretty certain are not covered in seminary.

This is made worse when the use of the term “drivers” in the Towers Watson document indicates that the people who wrote the report do not know the answer to question 6. (Or worse, they know and do not care.)

The way most people speak about the CTA data is almost idolatrous. It is as if the data in the report are magical totems that provide solutions to our problems. What they actually provide – as far as I can tell – is the authority of technical mastery. The data and the numbers sound impressive, especially to those who do not know much about statistics. They grant the leaders of the church authority that they seem not to have based on their position. (This may explain why the Call to Action report itself is nearly devoid of theological content. Statistics, not theology, are the currency of credibility in our culture.)

We as a denomination do not need statistical hocus pocus to tell us that getting people engaged in real small groups is good. We do not need Towers Watson to remind us that effective lay leadership is important to having a healthy church. There is not a single “aha” moment in the Call to Action data. (I’d love to hear any finding or suggestion of this process that came a surprise to the people who have been guiding it.)

The report’s primary usefulness in practice has been rhetorical. It allows leaders of the denomination to point to the “the numbers” and say they demand we do such and so. The fact that we must manufacture authority in this manner is a sad commentary on the state of things.