Over the past decade, there has been a dramatic increase in demands for evaluation within nonprofit organizations. No longer viewed as a process that only large, well-established, well-funded programs need to undertake, evaluation is now the coin of the realm–a virtual necessity for every new program proposal or request for funding renewal.
The evaluation field has also matured and diversified during this period. It now values “process” or “implementation” evaluations–activities to understand program operations–in addition to “impact” or “summative” evaluations–investigations to determine program effects. Rather than insist on the most scientifically elegant and rigorous designs for all evaluations, it credits the use of a range of research designs and methods for collecting necessary information, and it considers a variety of purposes for undertaking evaluation as legitimate and worthwhile.
One of the most important recent advancements in evaluation is the growing emphasis on making evaluation results immediately useful. “Utilization-focused” evaluation, promoted by Michael Patton, C. Lynn Usher, and others, “begins with the premise that evaluations should be judged by their utility and actual use” (Patton, p.20) as opposed to, for example, their adherence to the strict requirements of traditional scientific inquiry. To be useful, an evaluation must be accessible to program directors, staff, participants, and other stakeholders, and it must translate criticism and recommendations into realistic strategies for organizational change.
While it makes sense that all organizations can use evaluations to become more effective, the history and experience of evaluation as external and, apparently, judgmental is difficult to overcome. Most organizations are already doing their best to “own” the need for evaluation, but obstacles to its practice still persist. We must develop expertise and build a culture in which evaluation is viewed primarily as an opportunity to learn about ourselves. To do so, evaluation must be internally elected and highly participatory, not externally imposed by funding agencies and experts. Until this transition from external to internal is made, the learning curve for organizational self-evaluation will be steep and resistance to it will be firm.
In this article, we hope to advise program directors on how to become more active, effective consumers of evaluation, by describing how to present feedback–including criticism–and how to introduce the effective organizational change efforts that emerge from evaluation findings.
All of us like to think that good planning and careful phrasing are the keys to successful change. However, the blank, angry, and confused stares that often greet our efforts belie this orderly expectation. So, too, the resistance generated by even the most sensible or necessary organizational changes. Why does this occur?
Criticism and change are challenging–they question the very ideas, values, plans, commitments, efforts, and relationships that sustain people, and therefore generate a great deal of opposition. Some people resist criticism and change because they make us feel misunderstood. Others feel tested, judged, manipulated, patronized, or bullied. Still others, who believe in the merit and good intent of the evaluators, don’t know what to do with the feedback and end up feeling lost or inadequate.
Yet we do change. We even welcome the opportunity when we are ready. Readiness is derived from the Greek word arariskein which means “fitting” or “joining” or “being arranged for use.” Experience has taught us that certain approaches to criticism and change simply work better in particular organizational climates at particular times. And this is the heart of it: the alignment of interactions–the fit between the intervention and the problem, the style in which the intervention is introduced, and the way in which the people are addressed–is the key variable in determining success. We call this arrangement for use readiness.
This idea of readiness is as old as time itself. Traditional teachers, for example, often wait years before they think students are ready to receive their wisdom. Nowadays, and across many disciplines, the importance of intervening when the time is right is pivotal in theories of change. In crisis theory, for example, the openness and urgency of crises are said to create opportunities for change. The educational theorist, Eleanor Duckworth, emphasizes identifying and capitalizing on “teachable moments.” Evolutionary and systems theorists, such as Gregory Bateson and Irvin Lazlo, assert that “systems in disequilibrium are vulnerable to change,” often random and unpredictable, but, with forethought, open to planned interventions.
Building on these insights, we are proposing a new concept of readiness that offers program directors a much broader range of options for introducing change and–to complement these options–we offer a wide array of interventions that match well with various “states” of readiness.
Consider the following example: In the midst of a process-oriented evaluation at Agency A, it becomes clear that clients are dissatisfied with the services offered in a particular program site, or by a certain combination of personnel. The program director acknowledges that it is critical to feed back this information to the staff, but she can’t imagine that now is the right time, that the staff is ready to hear the news. After all, the unit has just experienced some serious assaults–a valued worker has resigned, a major service contract has not been renewed, and so forth. A deeper understanding of the various kinds of readiness that organizations experience and the types of information that can be most easily shared in each, as well as a range of strategies to impart evaluation results and initiate change, would serve this program director well. Let’s begin by defining the three major states of organizational readiness.
Readiness exists in three different states, each of which requires specific kinds of interventions. There are responsive states of readiness, such as curiosity, receptiveness, and determination, which are best served by information, advice, and guidance. There are unstable states of readiness, like confusion, anxiety, and crisis, which need to be reframed as integral aspects of the change process, then cultivated as the seedbed of creative thought and action. And then there are forays, changes in progress that either have not come to fruition or are not yet sufficiently recognized to exert a strong influence on the whole organization.
When the relationship between director and staff is built essentially on trust and credibility, and when staff feel effective in their practice and express a desire to learn from an evaluation, we begin with straightforward approaches to feedback and change. And we begin our survey of organizational readiness by trying to identify responsive states of readiness. Even within this responsive mood, there are a variety of “states,” arrayed along a continuum from modest interest to an active desire to learn to a determination to change.
When people are curious, but not actively seeking solutions to problems, for example, it is best to offer information and a variety of ways to understand a problem, but not to push for change. Future scenario planning is a form of strategic planning that assumes an uncertain future. It hypothesizes several paths to effectiveness and helps organizations be alert and aware rather than committed to a particular plan. This intervention is ideal for expanding the field of vision.
When receptive, people are open-minded. They have identified a problem but have not yet found the solution they are seeking. In this case, it helps to narrow the field by presenting to the organization a few strategies that have clear recommendations about how to choose among them. Also offer pros and cons as well as preferences backed by experience.
When there is urgency–for instance, if funding is threatened or community support dramatically decreases–people generally feel a strong need to do something quickly and to seek help. In this case, make clear, decisive suggestions and recommend specific solutions to the problem. At this stage, a variety of suggestions will only frustrate.
Determined people have identified a problem, know that a specific solution is required, and are committed to action. In this case, provide specific technical assistance–such as a specific, effective, way to implement a program–and then get out of the way. The determined staff will make immediate use of the assistance and carry forth on their own.
Often, organizations are thrown into states of disequilibrium when anticipating, or receiving, the results of evaluations; program participants are confused, anxious, resentful, suspicious, even angry. Conventional wisdom suggests that the director and evaluator should work hard to mitigate this disequilibrium–they should help the program get back into balance. We argue that this unstable state of readiness presents many opportunities for making creative use of evaluation findings. We explain this conclusion by discussing two of the responses mentioned above–confusion and anxiety.
Staff at Program X have just learned that their long-practiced approach to intervening with substance-abusing teenagers has not resulted in a decrease in drug use among the town’s teen population. They can’t believe it–it can’t be true! And if it is true, how can they continue to feel proud about, and effective in, their work? Resistance to feedback often begins with this kind of confusion. What can a director do? Try naming and affirming the confusion. Frame it as natural and as a source of potential energy and creativity. Instead of trying to quickly quell the anxiety this confusion causes, sustain or amplify it. Get people together and give them permission to wonder out loud what is going on and how they might do things differently. This discussion often unearths curiosity–which creates an opportunity to offer suggestions–and creative ideas. Program directors can then support the “grassroots” response, instead of forcing the issue of change upon the staff.
Anxiety about change–fear for one’s job or one’s ability to adapt–turns people inward, away from colleagues and collaboration and from realistic program planning. To get anywhere in an anxious climate, we must name it–not ignore or deny it. It helps to draw out both the individual and collective elements–what people fear for themselves and for the organization–in order to see the connection. During times of high anxiety, it is also important to provide structure. For example, you might plan an off-site meeting, saying, in essence, “We’ll talk together until we feel clear about the problem