Tyranny of Metrics

by

Jerry Muller's book, "The Tyranny of Metrics," examines how fixating on numbers creates a number of problems. The author states:, 

This book is not about the evils of measuring. It is about the unintended negative consequences of trying to substitute standardized measures of performance for personal judgment based on experience. The problem is not measurement, but excessive measurement and inappropriate measurement—not metrics, but metric fixation.

T

he book reads as a cautionary tale for the Church. The more the Church reads reports that "our numbers are in decline" the tighter we cling to metrics. As the Church faces a legitimacy and relevancy crisis in the culture, there is a temptation to fixate on metrics as a way to show legitimacy and relevancy. 

As the author says, metrics are not evil. It is difficult to diagnose a problem if there are not some measurements we can look at over time to make adjustments. It was once said to me that the numbers we look at in the church are similar to the numbers a doctor looks at when you go in for a check up. These "check up" numbers do not tell you everything about your health, but they are a starting point. If your breathing is consistently slow it could mean you are a super healthy marathoner but it could also mean you are close to death.

It is the fixation on metrics that is creeping into the Church that is a cause for concern. 

So what does metric fixation look like? Muller describes it in this way:

  • The belief that it is possible and desirable to replace judgment, acquired by personal experience and talent, with numerical indicators of comparative performance based upon standardized data (metrics)
  • The belief that making such metrics public (transparent) assures that institutions are actually carrying out their purposes (accountability)
  • The belief that the best way to motivate people within these organizations is by attaching rewards and penalties to their measured performance, rewards that are either monetary (pay-for-performance) or reputational (rankings)

Paradoxically using metrics as the means to gain a sense of clarity of a situation and using metrics as the primary measure of success creates misaligned incentives. For instance, the surgeon who wants to have more patients may talk about how many successful surgeries they have preformed. However, what she/he fails to communicate is that the they only take cases that have a high probably of success to begin with. Thus the "successful' surgeon may not who you want for your complicated surgery. This example might be called, "creaming" the numbers. That is the surgeon counts only the "cream of the crop." The patient and the surgeon have misaligned incentives. 

Muller points out a series of reoccurring flaws in using metrics:

  1. Measuring the things that are easy to measure (such as people in worship) 
  2. Measuring inputs (such as money) over outcomes (such as a transformed life)
  3. Creaming (counting only the best)
  4. Lowering standards (calling a gathering a worship, then add those numbers to weekly worship total)
  5. Omitting or distorting data (such as double counting a person who attends more than worship hour on Sunday)
  6. Cheating (such as when the preacher adds to the numbers because "it felt like there were more people there...")

The reality is metric fixation is killing clergy and creating cultures where churches are dominantly assessed through what is easily counted rather than through the Spirit. How we overcome metric fixation is a difficult but not impossible process. Metrics are only a small picture of reality, but because it is a number it weighted heavier. Metrics give the impression of concreteness and accessibility to situations that are ambiguous and complex. There is a desire to simplify the complexities of the world giving us a false sense of control and understanding. 

Fixating on metrics means that when a church provides their "check up" numbers, we forget that sometimes the heartbeat of the church looks good because there is a pacemaker modifying the heartbeat. Just because a church looks strong in the metrics might mean they are "juicing" in subtle (and even purely motivated) ways. Just because a church looks weak in the metrics does not mean that the church is dying or failing. 

And even if it is dying, the Church of Jesus Christ is not dead for too long because Sunday is coming.