Over the past decade, there has been a dramatic increase in demands for evaluation within nonprofit organizations. No longer viewed as a process that only large, well-established, well-funded programs need to undertake, evaluation is now the coin of the realm–a virtual necessity for every new program proposal or request for funding renewal.
The evaluation field has also matured and diversified during this period. It now values “process” or “implementation” evaluations–activities to understand program operations–in addition to “impact” or “summative” evaluations–investigations to determine program effects. Rather than insist on the most scientifically elegant and rigorous designs for all evaluations, it credits the use of a range of research designs and methods for collecting necessary information, and it considers a variety of purposes for undertaking evaluation as legitimate and worthwhile.
One of the most important recent advancements in evaluation is the growing emphasis on making evaluation results immediately useful. “Utilization-focused” evaluation, promoted by Michael Patton, C. Lynn Usher, and others, “begins with the premise that evaluations should be judged by their utility and actual use” (Patton, p.20) as opposed to, for example, their adherence to the strict requirements of traditional scientific inquiry. To be useful, an evaluation must be accessible to program directors, staff, participants, and other stakeholders, and it must translate criticism and recommendations into realistic strategies for organizational change.
While it makes sense that all organizations can use evaluations to become more effective, the history and experience of evaluation as external and, apparently, judgmental is difficult to overcome. Most organizations are already doing their best to “own” the need for evaluation, but obstacles to its practice still persist. We must develop expertise and build a culture in which evaluation is viewed primarily as an opportunity to learn about ourselves. To do so, evaluation must be internally elected and highly participatory, not externally imposed by funding agencies and experts. Until this transition from external to internal is made, the learning curve for organizational self-evaluation will be steep and resistance to it will be firm.
In this article, we hope to advise program directors on how to become more active, effective consumers of evaluation, by describing how to present feedback–including criticism–and how to introduce the effective organizational change efforts that emerge from evaluation findings.
All of us like to think that good planning and careful phrasing are the keys to successful change. However, the blank, angry, and confused stares that often greet our efforts belie this orderly expectation. So, too, the resistance generated by even the most sensible or necessary organizational changes. Why does this occur?
Criticism and change are challenging–they question the very ideas, values, plans, commitments, efforts, and relationships that sustain people, and therefore generate a great deal of opposition. Some people resist criticism and change because they make us feel misunderstood. Others feel tested, judged, manipulated, patronized, or bullied. Still others, who believe in the merit and good intent of the evaluators, don’t know what to do with the feedback and end up feeling lost or inadequate.
Yet we do change. We even welcome the opportunity when we are ready. Readiness is derived from the Greek word arariskein which means “fitting” or “joining” or “being arranged for use.” Experience has taught us that certain approaches to criticism and change simply work better in particular organizational climates at particular times. And this is the heart of it: the alignment of interactions–the fit between the intervention and the problem, the style in which the intervention is introduced, and the way in which the people are addressed–is the key variable in determining success. We call this arrangement for use readiness.
This idea of readiness is as old as time itself. Traditional teachers, for example, often wait years before they think students are ready to receive their wisdom. Nowadays, and across many disciplines, the importance of intervening when the time is right is pivotal in theories of change. In crisis theory, for example, the openness and urgency of crises are said to create opportunities for change. The educational theorist, Eleanor Duckworth, emphasizes identifying and capitalizing on “teachable moments.” Evolutionary and systems theorists, such as Gregory Bateson and Irvin Lazlo, assert that “systems in disequilibrium are vulnerable to change,” often random and unpredictable, but, with forethought, open to planned interventions.
Building on these insights, we are proposing a new concept of readiness that offers program directors a much broader range of options for introducing change and–to complement these options–we offer a wide array of interventions that match well with various “states” of readiness.
Consider the following example: In the midst of a process-oriented evaluation at Agency A, it becomes clear that clients are dissatisfied with the services offered in a particular program site, or by a certain combination of personnel. The program director acknowledges that it is critical to feed back this information to the staff, but she can’t imagine that now is the right time, that the staff is ready to hear the news. After all, the unit has just experienced some serious assaults–a valued worker has resigned, a major service contract has not been renewed, and so forth. A deeper understanding of the various kinds of readiness that organizations experience and the types of information that can be most easily shared in each, as well as a range of strategies to impart evaluation results and initiate change, would serve this program director well. Let’s begin by defining the three major states of organizational readiness.
Readiness exists in three different states, each of which requires specific kinds of interventions. There are responsive states of readiness, such as curiosity, receptiveness, and determination, which are best served by information, advice, and guidance. There are unstable states of readiness, like confusion, anxiety, and crisis, which need to be reframed as integral aspects of the change process, then cultivated as the seedbed of creative thought and action. And then there are forays, changes in progress that either have not come to fruition or are not yet sufficiently recognized to exert a strong influence on the whole organization.
When the relationship between director and staff is built essentially on trust and credibility, and when staff feel effective in their practice and express a desire to learn from an evaluation, we begin with straightforward approaches to feedback and change. And we begin our survey of organizational readiness by trying to identify responsive states of readiness. Even within this responsive mood, there are a variety of “states,” arrayed along a continuum from modest interest to an active desire to learn to a determination to change.
When people are curious, but not actively seeking solutions to problems, for example, it is best to offer information and a variety of ways to understand a problem, but not to push for change. Future scenario planning is a form of strategic planning that assumes an uncertain future. It hypothesizes several paths to effectiveness and helps organizations be alert and aware rather than committed to a particular plan. This intervention is ideal for expanding the field of vision.
When receptive, people are open-minded. They have identified a problem but have not yet found the solution they are seeking. In this case, it helps to narrow the field by presenting to the organization a few strategies that have clear recommendations about how to choose among them. Also offer pros and cons as well as preferences backed by experience.
When there is urgency–for instance, if funding is threatened or community support dramatically decreases–people generally feel a strong need to do something quickly and to seek help. In this case, make clear, decisive suggestions and recommend specific solutions to the problem. At this stage, a variety of suggestions will only frustrate.
Determined people have identified a problem, know that a specific solution is required, and are committed to action. In this case, provide specific technical assistance–such as a specific, effective, way to implement a program–and then get out of the way. The determined staff will make immediate use of the assistance and carry forth on their own.
Sign up for our free newsletters
Subscribe to NPQ's newsletters to have our top stories delivered directly to your inbox.
By signing up, you agree to our privacy policy and terms of use, and to receive messages from NPQ and our partners.
Often, organizations are thrown into states of disequilibrium when anticipating, or receiving, the results of evaluations; program participants are confused, anxious, resentful, suspicious, even angry. Conventional wisdom suggests that the director and evaluator should work hard to mitigate this disequilibrium–they should help the program get back into balance. We argue that this unstable state of readiness presents many opportunities for making creative use of evaluation findings. We explain this conclusion by discussing two of the responses mentioned above–confusion and anxiety.
Staff at Program X have just learned that their long-practiced approach to intervening with substance-abusing teenagers has not resulted in a decrease in drug use among the town’s teen population. They can’t believe it–it can’t be true! And if it is true, how can they continue to feel proud about, and effective in, their work? Resistance to feedback often begins with this kind of confusion. What can a director do? Try naming and affirming the confusion. Frame it as natural and as a source of potential energy and creativity. Instead of trying to quickly quell the anxiety this confusion causes, sustain or amplify it. Get people together and give them permission to wonder out loud what is going on and how they might do things differently. This discussion often unearths curiosity–which creates an opportunity to offer suggestions–and creative ideas. Program directors can then support the “grassroots” response, instead of forcing the issue of change upon the staff.
Anxiety about change–fear for one’s job or one’s ability to adapt–turns people inward, away from colleagues and collaboration and from realistic program planning. To get anywhere in an anxious climate, we must name it–not ignore or deny it. It helps to draw out both the individual and collective elements–what people fear for themselves and for the organization–in order to see the connection. During times of high anxiety, it is also important to provide structure. For example, you might plan an off-site meeting, saying, in essence, “We’ll talk together until we feel clear about the problem and have designed a strategy to address it that we can all support.”
Often even the most dismal evaluation results highlight some aspect of the program–one site, one team, progress toward achieving one particular objective–that appears more effective than the rest. Even rigidly or ineffectively organized programs produce some small successes–exceptions to the rule. These blips on the screen are critical to the organizational change process, and directors must learn to identify, support, and expand on them. When identifying these forays, we look primarily for two types of phenomena:
* People who clearly, strongly, and of their own initiative, move in the direction in which the organization has chosen to move.
* Innovative activities that, with a little encouragement and direction, can be leveraged so that they influence the larger organization.
Once forays have been identified or generated (as in a pilot project), it is time to take advantage of their tremendous potential in the larger system. There are many ways to leverage forays. Among them are:
- Increase the foray’s domain. If a small group is successfully managing a project, for example, increase the population for which they are responsible.
- Replicate the foray. Leverage a foray by replicating it in other areas of an organization. For example, determine why one program site is successful while others have failed, then introduce the successful methods at the other sites, adapting them, of course, to the unique circumstances that exist at each site.
- Increase the influence and authority of the foray leaders. When supporting forays, it makes sense to identify and highlight leaders whose innovative skills should be given a broader range. In the glow of this triumph, it can be a natural and fairly easy step to augment their influence and authority.
- Leverage the unintended consequences (and surprises) of the foray’s development. Change rarely proceeds exactly as one intends. Problems arise, but so do unanticipated opportunities. Being alert to these opportunities may provide alternative methods for organizations to achieve their strategic ends.
A countywide curbside recycling program, considered successful in its first few years of operation, has been unable to increase the number of households participating for several years running. In addition, there are pockets in the county where participation has been consistently low. This current evaluation noted, however, that one neighborhood in an “underactive” community is posting unexpectedly high rates of participation. A wise program director would gather information on that foray with the hope of extending its success.
No matter how skillful we are at offering feedback, no matter how attentive we are to states of readiness, no matter how inclusive and respectful we are in our strategic planning and implementation, we will be met with some level of resistance. Change and resistance go together, hand in glove. Each is natural, pervasive, and universal. We must recognize that while resistance is unavoidable, it is not bad. It is a fact of organizational life and as such, it must be managed, not avoided.
Resistance wears many faces: outright refusal, denial, skepticism, lethargy, incompetence, pessimism, and helplessness. Sometimes people resist by questioning the competence, credentials, or motives of the evaluators or organization leaders, or by engaging in behind-the-scenes lobbying. At others times, they become secretive, entering a bunker-like mode until the siege of change passes, and ignoring the need for action entirely.
Living systems thrive when their need for stability is appropriately balanced by their need for change. To effectively manage resistance, it must be understood as a system’s response to regain the equilibrium that has been disrupted by change. Viewed from this perspective then, resistance is simply feedback about those disruptions. And this feedback is useful because it points out how the relationships required to implement change are poorly aligned with each other and with the goals of the change project. Managing resistance means using the information it provides to realign those relationships in the service of achieving goals.
Poorly managed, resistance can be costly. Well managed, it permits you to execute projects and implement change efforts with a minimum of difficulty. Let’s consider the data from a social service program’s evaluation recommending that its eligibility requirements be redefined to better meet the service needs of the target population. This finding was unanticipated, and is likely to be bitterly opposed by a vocal minority of participants–perhaps those above the income cut-off who might lose certain services. The program director’s choices are not dichotomous: either implement the necessary change now, and in the process roll over a resistant minority, or give up this possibility for improving the program. Instead, she must develop a management plan for working through the resistance.
Here are some possibilities for managing the resistance:
- Anticipate resistance. If evaluation leads to change, resistance will follow–if not at the beginning then at some point in the change process. Try not to dread, ignore, or punish it. Regularly scan the horizon, like a sailor keeping watch over an unpredictable sea.
- Explore the meaning of resistance. Don’t try to determine the meaning by yourself. Question others who are involved. Consider such questions as: From which conflicts of interest does it emerge? Which partnerships are misfiring? Which relationships need recalibrating? The better you become at exploring and articulating the reasons for resistance, the less others will persist with passive, indirect responses. The more those in conflict come to a shared idea of the meaning of resistance, the better chance they have of resolving the difficulties. In fact, the very act of mutual inquiry is the first, major step toward building a partnership capable of solving the problems that led to the resistance in the first place.
- Validate the resistance, thus empowering those who resist. Let people know, in essence, that given the circumstances, their resistance makes sense. You might say that if in their position you, too, would resist. Beneath the manifest resistance, there is important information about how and how not to mobilize people behind a project. By acknowledging the hidden or deeper meanings of resistance, we accomplish two things: we let people know they are understood, and we encourage them to be direct so that in the future we can enter a productive conversation more quickly.
- Form a partnership to solve the problem pointed out by the resistance. When your efforts to “join” the resistance are sufficiently authentic, a partnership is formed with those who resist. The partnership turns inward, toward understanding the difficulties within the project teams. This effort to reach a common understanding is sometimes enough to break the problem’s hold; allowing the team to get on with the business of coming up with a solution or completing the project.
The process of planning, learning, and adjusting one’s course is continuous and conscious in effective organizations. Program directors must develop their own capacities to understand and use evaluation. Ideally as a program is being planned, you should incorporate evaluation so the process of feedback and change become a familiar process. When corrections are necessary, consider the fit of your ideas with the readiness of staff to make use of the ideas. Finally, by anticipating, understanding, and validating resistance, you join and rejoin your staff to effectively align yourselves in support of project achievement.
Patton, Michael Q. 1996. Utilization-Focused Evaluation, third edition. Thousand Oaks, CA: Sage Publications.
Barry Dym, Ph.D. is an organizational development consultant and psychologist whose practice is focused on managing organizational transitions and the planning and implementation of large-scale change. He was a co-founder of the Family Institute of Cambridge, a leading postgraduate training institute for couple and family therapists, and a lecturer at Harvard Medical School. More recently, he formed a consulting group called WorkWise Research and Consulting. He is the author of the book, Readiness and Change in Couple Therapy (1995), and of numerous articles in OD Practitioner. His clients range from entrepreneurial start-ups, family businesses, and nonprofits to complex organizations such as Honeywell and Digital Equipment Corporation.
Francine Jacobs, Ed.D., is an associate professor at Tufts University, with a joint appointment in the Departments of Child Development and Urban and Environmental Policy (where she is currently chair). Her teaching and research interests are in child and family policy and in program evaluation, particularly of community-based organizations. Her Five-Tiered Approach to Evaluation, first explicated in her 1988 co-edited volume (with Heather Weiss), Evaluating Family Programs, has become a popular method for conducting evaluation within these agencies. Fran has consulted widely to nonprofit and governmental programs at the local, state, and national levels.
