What Do We Measure and Why? Questions About the Uses of Measurement

Print Share on LinkedIn More

Sidewalk

We live in a culture that is crazy about numbers. We seek standardization, we revere precision, and we strive for control. The very ancient and dominant belief of Western culture is that numbers are what is real. If you can number it, you make it real. Once made real, it’s yours to manage and control. We increasingly depend on numbers to gauge how we are doing on virtually everything. We ascertain our health with numbers: How many calories or grams should I eat? What’s my cholesterol reading? We assess one another with numbers: What’s your I.Q.? What’s your GPA? Your Emotional Intelligence? And, of course, we judge organizational viability only with numbers: What’s the customer satisfaction rating? Inventory turns? ROI? What about the P/E ratio?

It is numbers and only numbers that define and make visible what is real. This is the “hard stuff,” the real world of management–graphs, charts, indices, ratios. Everyone knows that “you can only manage what you can measure.” The work of modern managers is to interpret and manipulate these numeric views of reality. The desire to be good managers has compelled many people to become earnest students of measurement. But are measures and numbers the right pursuit? Do the right measures make for better managers? Do they make for stellar organizations?

As we look into the future of measurement, let us pause for a moment and question this number mania. We’d like you to consider this question: What are the problems in organizations for which we assume measures are the solution?

Presumably, most managers want reliable, high-quality work. They want commitment, focus, teamwork, learning, and quality. They want people to pay attention to those things that contribute to performance.

If you agree that these are the general attributes and behaviors you’re seeking, we wonder whether, in your experience, you have been able to find measures that sustain these strong and important behaviors over time. Or if you haven’t succeeded in finding them yet, are you still hopeful that you will find the right measures? Do you still believe in the power of measures to elicit these performance qualities?

We believe that these behaviors are never produced by measurement. Rather they are performance capabilities that emerge as people feel connected to their work and to each other. They are capacities that emerge as colleagues develop a shared sense of what they hope to create together and as they operate in an environment where everyone feels welcome to contribute to that shared hope. Each of these qualities and behaviors–commitment, learning, teamwork, quality, innovation–is a choice that a person makes. Depending on how connected they feel to the organization or team, people choose to pay attention, to take responsibility, to innovate, to learn, and to share their learning. People can’t be punished or paid into these behaviors. They are either contributed or withheld by individuals, as they choose whether and how they will work with us.

But to look at prevailing organizational practice, most managers consistently seem to choose measurement as the route to these capabilities. They agonize to find the right reward that can be tied to the right measure. How long have we searched for the rewards that will lead to better teamwork or to more innovation? And haven’t we learned yet that measures or rewards work only as a short-term incentive, if they work at all? Ironically, the longer we try to encourage these behaviors through measurement and reward, the more damage we do to the quality of our relationships and the more we trivialize the meaning of work. Far too many organizations have lost their way on the path to quality because they have burdened themselves with unending measures to achieve their goals. How many employees have become experts at playing “the numbers game” to satisfy bosses rather than experts at doing their jobs? The path of measurement can lead us dangerously far from the organizational qualities and behaviors that we require.

But measurement is critical. Measurement can provide something that is essential to sustenance and growth: feedback. All living things thrive on feedback. We have to know what is going on around us, how our actions affect others, how the environment is changing, how we’re changing. If we don’t have access to this kind of information, we can’t adapt or grow. Without feedback, we shrivel into routines and develop hard shells that keep newness out. We don’t survive for long.

In any living system, feedback differs from measurement in several significant ways:

  • Feedback is self-generated. Individuals or systems notice only whatever they determine is important them. They ignore everything else.
  • Feedback depends on context. Critical information is being generated right now. Failing to notice the “now,” or getting stuck in past assumptions, is very dangerous.
  • Feedback changes. What individuals or systems choose to notice will change depending on the past, the present, and the future. Looking for information only within rigid categories leads to blindness, which is also dangerous.
  • New and surprising information can get in. The boundaries are permeable.
  • Feedback is life-sustaining. It provides essential information about how to sustain ourselves, but it also indicates when adaptation and growth are necessary.
  • Feedback supports movement toward fitness. Through the constant exchange of feedback, individuals and their environments can co-evolve toward mutual sustainability.

As we reflect on the capacities that feedback can provide, it seems we are seeking many similar attributes in our organizations. But we haven’t replicated the same processes, and therefore we can’t achieve the same outcomes. There are some critical distinctions between feedback and measurement, as shown in the lists below.

If we understand the critical role played by feedback in living systems, and contemplate these distinctions, we could develop measurement processes that support the behaviors and capacities we require, those that enhance the vitality and adaptability of the organization. To create measures that more resemble feedback, we suggest the following questions. We use them as design criteria for any measure or measurement process:

Who gets to create the measures? Measures are meaningful and important only when generated by the people doing the work. Any group can benefit from others’ experience and from expert advice, but the final measures must be the creation of the people intimately involved with the outcome. People only support what they create, and those closest to the work know a great deal about what is significant to measure.

How will we measure our measures? How can we ensure that measures are useful and current? How will we know when they are obsolete? How will we keep abreast of changes in context that warrant new measures? Who will look for the unintended consequences that accompany any process and feed that information back to us?

Are we designing measures that are permeable rather than rigid? Are the measures open enough? Do they invite in newness and surprise? Do they encourage people to look in new places or to see with new eyes?

Will these measures create information that increases our capacity to develop, to grow into the purpose of this organization? Will this particular information help individuals, teams, and the entire organization grow in the right direction? Will this information help us to deepen and expand the meaning of our work?

What measures will inform us about critical capacities: commitment, learning, teamwork, quality, and innovation? How will we measure these essential behaviors without destroying them through the assessment process? Do these measures honor and support the relationships and meaning-rich environments that give rise to these behaviors?

Although the answers to these questions may reveal a daunting job ahead, we assure you it is not difficult to implement the necessary changes. These changes do, however, require extraordinary levels of participation–defining and using measures becomes everyone’s responsibility. We’ve known teams, manufacturing plants, and service organizations where everyone knew that measurement was critical to their success and therefore embraced the task with great enthusiasm and creativity. They were aggressive about seeking information from any source that might contribute to those purposes that had been defined as most important to their organization–such purposes as safety, team-based organization, or social responsibility. Their process was creative and experimental, and the measures they developed were often nontraditional. These people stretched and struggled to find ways to measure qualitative aspects of work. They developed unique and complex multivariate formulas that would work for a while and then need to be replaced by new formulas. They understood that the right measurements gave them access to the information they needed to prosper and grow. But what was “right information” kept changing. And in contrast to most organizations, measurement felt alive and vital in these work environments. It wasn’t a constraint or dead weight; rather it helped people accomplish what they wanted to accomplish. It provided feedback, the information necessary for them to adapt and thrive.

Being in these workplaces, we also learned that measurement needs to serve the deepest purposes of work. It is only when we connect at the level of purpose that we willingly offer ourselves to the organization. When we have connected to the possibilities of what we might create together, then we want to gather information that will help us be better contributors. But in too many organizations, just the opposite happens. The measures define what is meaningful rather than the greater meaning of the work defining the measures. As the focus narrows, people disconnect from any larger purpose and do only what is required of them. They become focused on meeting the petty requirements of measurement and, eventually, they die on the job. They have been cut off from the very wellspring of purpose that motivates them to do good work.

If we look closely at our experience in the past few years, it is clear that as a management culture, we have succeeded at developing finer and more sophisticated measures. But has this sophistication at managing by the numbers led to the levels of performance or commitment we’ve been seeking? And if we have achieved good results in these areas, was it only because we discovered the right measures, or can we credit something else going on in the organization?

We would like to dethrone measurement from its godly position, to reveal the false god it has been. We want instead to offer measurement a new job–that of helpful servant. We want to use measurement to give us the kind and quality of feedback that supports and welcomes people to step forward with their desire to contribute, to learn, and to achieve. The uses of measurement must come from a deeper place of understanding–the understanding that the real capacity of an organization arises when colleagues willingly struggle together in the common work that they love.

What do YOU think is important to measure?


 


Reprinted with permission from Journal of Performance Management. 1999. New York: Research Institute of America Group (January) pp 24-27.


Margaret Wheatley and Myron Kellner-Rogers are consultants to an unusually broad array of organizations, from the U.S. Army to local schools, from global corporations to religious orders. Together they lead the work of the Berkana Institute, a nonprofit research foundation, and are partners in the educational and consulting firm Kellner-Rogers & Wheatley Inc. They are co-authors of A Simpler Way (1996, Berett-Koehler Publishers). Dr. Wheatley’s earlier book, Leadership and the New Science (1992, Barett-Koehler) was recently named one of the 10 best management books of all time.


 

Feedback Measurement
Context-dependent One size fits all
Self-determined; the system chooses what to notice Imposed; criteria are established externally
Information accepted from anywhere Information from fixed categories only
System creates own meaning Meaning is predetermined
Newness, surprise are essential Predictability, routine are valued
Focus on adaptability and growth Focus on stability and control
Meaning evolves Meaning is static
System adapts to and with its environment System adapts to the measures
  • Greg

    This is a very important article. I teach science on the university level and I tell my students that you can measure things till the end of time and not learn anything. The question is what are you going to measure and why. There is a qualitative aspect to even the most empirical quantitative endeavors and we often forget that about science … and management.

  • Ben Warner

    I’m concerned about the assumptions in this article, especially as reprinted in the NPQ. Do we really believe that nonprofits have enthroned measurement in a godly position? That we are too dependent on measures and too focused on ROI? Are nonprofit managers and directors going to read this and suppose that they should be de-emphasizing outcome measures and performance metrics?

    In my experience, nonprofits are still behind the curve in measurement. Because ROI is much more complex to calculate in a field that is not profit-focused, we too often think about program costs and outcomes in separate, convoluted circles. We still rely much too much on anecdote and photograph to tell our stories, and our biggest problems in internal management are being too empathetic, not too numbers-driven.

    Yes, measurement can be taken too far. But I don’t think, nor have I seen the research that would suggest, that nonprofits have reached that point. Instead, I fear we’re still far away from implementing adequate performance measurement systems, within which the desired flexibility can operate. No for-profit industry really ever forgets about the bottom line; unfortunately, many nonprofits too often have left “bottom line” too amorphous to drive internal systemic reforms. At least that’s been my experience in the nonprofits I’ve consulted with — I’d appreciate knowing if the field is really at the point suggested in the article.