When was the last time you were in a nonprofit staff or board meeting where the agenda included a discussion of the implications or lessons from a specific research study?

Despite thousands of evaluations in human services, countless foundation and government reporting regimens, an “outcomes” movement, small armies of evaluation consultants, large parts of the nonprofit sector engage in very little discussion of contemporary research within organization managements, boards or supporters. Is research avoidance something inherent in our nonprofit corporate cultures?

Perhaps, we are chasing after so much all the time (funding, our board members, clients, the source of the leak in the community room) we just don’t have the time to seek out and read studies in all of the areas that might be pertinent to the way we work. Our staffs spend their time on urgent daily tasks to meet contracts or the basic requirements of our work and our boards are mostly concerned with the budget and keeping regulatory wolves from the door. Who can afford the time?

But, who can afford not to spend the time? The best organizations have a natural hunger to know about how they are doing, how others are doing things and with what results, and what variables might be affecting their work now and in the future.

There are a number of types of research that should interact in any good organization:

  • Informal but continuous and documented inquiry by everyone about community/constituent issues.
  • Systematic surveying and research on community/constituent issues, opinions, needs and desires.
  • Research about emerging trends in—and influences on—your environment.
  • Research about the research in your field, programmatic models being tried elsewhere in similar fields, ideas floated, policy being proposed, outcomes achieved and the variables believed to have affected those outcomes.
  • Research about financial models used in similar organizations.
  • Research about resources.
  • Evaluation of individual programs and outcomes measurement.
  • Evaluation of organizational fit to the environment and organizational performance against mission.
  • Research on management and governance ideas that might help your organization to function better.

Benefits of a Research Orientation

Recently I have had the opportunity to work with a group of diverse organizations in Memphis as they have engaged in a three year capacity building program. They include a Christian college, a rural senior services group, a community development organization, an aids organization and a massive agency that serves mentally disabled adults. One of the more surprising findings from that project was that each of the seven organizations involved benefited greatly from eliminating programs. This was the result of a careful critical examination of each organization’s overall effectiveness relative to its mission, its community, and other players in the field. One school eliminated a high school program to focus on younger students where interventions promise more success. Another organization that had been doing direct housing development repositioned itself as a partner to the entire housing development field because there were products that all CDC’s in the area needed that did not need to be replicated from neighborhood to neighborhood. We might see these as natural progressions in fine tuning focus unless we look at the history of the organizations. A number of them had been accumulating programs for years while their reputations in their communities waned and their financial positions worsened.

When these organizations took the time to research their positions including the models that were working in their fields, what they were most valued for, and a host of other questions, they were suddenly able to make decisions that freed them to focus more acutely. Simply evaluating their individual programs for funding sources and accreditation agencies had not helped them to make this leap. In fact, evaluation not connected to overall agency effectiveness had often weighed them down rather than freed them up.

Stakeholder Engagement and Democracy

Some of the organizations in Memphis have drawn both staff and board members together to struggle with larger questions of focus and they have also actively drawn in their constituencies and partners. This has resulted in more than a better strategic plan. It has resulted in a well directed and broad based dynamism.

In describing the results on stakeholders of their questioning and planning processes, they use the word “confidence” over and over. In describing the results on their organizations the use the word “focus.” Yet they question everything.

Their boards know how to weigh in, their staffs are more self-directed and they provide each other with feedback. These are not tightly controlled situations. They are situations into which people bring their intellects, spirit, and energy. Their reputations in the communities they serve and raise funds from are greatly improved and they have advocates everywhere so when external threats arise, they do not stand alone.

Later in this issue of NPQ you will read an interview with Betsy Santiago. When I first met Betsy she was barely out of her teens. She was and is still a very respectful person, not prone to asserting her opinion at every turn. But she was willing to risk being trained to be a part of a statewide research team on responses to homelessness at the program and foundation levels. She did a lot of listening and eventually gave voice to what she had heard, participating on boards, testifying to the legislature and helping to develop programs. Fifteen years later she still uses her integrated inquiry and advocacy skills among the homeless AIDS patients she works with to feed her own and other people’s understanding of what they are seeing as a collective group and what needs to be done.

This type of conscious respectful engagement with people experiencing the reality of what you work on is not, unfortunately, seen as core to the practice of a lot of organizations. It should be the starting place for all other research and planning. Without it, you can wander far from a worthwhile path and even into very destructive territory.

The Way Questions Are Posed: Definitional Power

Early in the battered women’s movement there was struggle to define what constituted “success” in interventions. Once convinced that battering was a dangerous crime, many funders wanted to see a particular result—that the woman leaves the situation—and this was often what was proposed as the proxy for effectiveness. Many of the shelters and coalitions that constituted the base from which policy was pushed and services provided argued back. Who was in the best position to make a judgement about the woman’s future?

There were assumptions and political orientations behind each position. The position of the battered women’s movement at that time was that women should be given the freedom, safe space, and legal protections from violence to make their own informed decisions. This required that police, court and other systems should be responsive and informed—which was not yet true. (You’d still be hard pressed to stake your life on it). Within that context, shelters believed that what was required of them was to respect the woman’s wisdom, further informing it and providing avenues for escape.

It was not until years later that research proved that the point of departure—when a woman leaves her batterer—is her most potentially fatal moment. The women working in the shelters had an instinct for the risks involved and for right practice. The systems were not sufficient to protect women. How could a shelter advise her to stay or go?

Further research by Susan Schechter, Tom Adams, Ann Jones, and others looked at the characteristics of batterers who eventually killed their partners. Out of this, risk assessments were developed so that advocates could help women to gauge their own situations for imminent danger. Among other things the assessments pointed out was that threats of suicide by the batterer correlated to higher rates of homicide. This was valuable information to give to the woman confronted with threats of suicide by a pleading, “abandoned” batterer.

In short, the way questions were posed and the way people practiced was changed by research over time. Little of what was found came out of shelter specific evaluations but the people working in those programs had a sense of the need for caution about any one best way to advise women. This caution was born from living and watching the reality of many women over time. Simple but painful experience shared among the loose-knit national network of shelters informed the larger research questions.

That whole movement has a few stalwart women and men to thank for their early dedication to the careful research that mostly bore out—but sometimes contradicted—the assumptions of that field. Susan Schechter, in particular, who recently passed away, often talked about the personal pain involved in tracing the factors leading up to a domestic homicide. She interviewed the batterers in prison and the families of the victims. She looked at the history of failed attempts at seeking help. And then she had to fight with some of the battered women’s advocates who were resistant to new information that altered their paradigms—trying to keep them in inquiry mode. Thankfully, this orientation lives on in the “Building Comprehensive Solutions” initiative within the battered women’s movement.

The Problem of True Believers

This sector is full of true believers. They bring energy and the sometimes single minded passion that drives change in society but some of the most painful missteps we have seen in this sector and the public policy pushed by it are the result of groups refusing to question their own most basic assumptions. This is excruciatingly human. Those who originally lay a path to reform know they must convince others of the rightness of the approach and eventually stepping out of line is seen as a betrayal of the intention—even if the original plan and rationale fails in practice.

Thus, long and now impossibly tangled legacies have been left by well meaning reformers with ideas that made perfect sense to them at the time. The Quakers’ involvement in the development of penitentiaries as a reform comes to mind. Their efforts were focused on relinquishing flogging and other brutal corporal punishments in favor of a strict isolation that would encourage penance. The isolation was not only from the outside world but from other prisoners and any discourse. The walls were thick and high. Bibles were the only reading material. When moved from one place to another the prisoners would have bags placed over their heads. This was to give prisoners time to converse with God. Many chose instead to become mentally ill if they were not already. Go figure.

No matter that penitentiaries have never worked to produce the hoped for outcomes, the basic concept persists after centuries and, in fact, has been resurrected with a vengeance in an explosion of privately run high security institutions, even down to the hub and spokes architectural design originally associated with penitentiary. The unanticipated consequences: we have built the U.S. prison system from something conceptualized to keep criminality contained to a small disordered segment of the population, into the largest prison system in the world. Now regularly seen as a good economic development strategy, the penitentiary has become truly massive and a core component of our market economy.

Not a good outcome. But to give the Quakers credit, they have stayed involved in criminal justice reform.

The essence of any scientific method is to try not only to prove but to disprove your hypotheses and in reasonable time. Of course this can be difficult when you have built monuments to the idea being promoted such as in the case of penitentiaries.

The organizational story of Youth Villages on the next pages illustrates the rich outcomes that can be achieved from an openness to having one’s programmatic model proved inadequate to the whole task at hand.

Conclusion

From what I have observed empirically over the years, organizations that mix all of the types of research listed at the beginning of this article excel. But I would like to suggest that there is a logical sequence in terms of importance and timing. This is reflected in their order as written. As you may note, the first two types of research have to do with truly understanding the needs and ideas and strengths of constituents, individually and collectively. This helps us to set direction and ask the right questions. All else flows from there.

People’s stories generally speak to us on many levels that we can not hear immediately. One woman told us she was sent to our shelter by another shelter three counties down the line because they were full. She told us that she was not able to finish school. That she was abused physically in her marriage. That her husband had a gun. That he was very jealous. That her husband was an officer in the Navy and that leaving him was frowned upon as disloyal among the other women. That she was most scared about the health of her infant and that she worried about contacting the military hospital where the baby’s doctor was for fear that her husband would find out where she was. That she was worried about her dog because her husband said he would kill it if she left. She told us she wanted to return to her family’s reservation for a few days to bury her baby’s umbilical cord with his ancestors before the baby got sicker. She wondered whether it was more responsible to leave or to stay.

Every single aspect of this story was meaningful in terms of what we now know about battering and what to do about it. We had to hear this one story and then combine with others who had heard tens of thousands of other stories to know the import that should be given to each of these facts beyond the immediate import to this woman and her children. And then we had to ask more questions and on and on. Eventually we had the beginnings of a cumulative picture and the beginnings of a responsible approach to systems change and direct service at least as things were then.

If we had decided that four of these facts were important and recorded only them and built our theories of practice around them, we would have been misguided. If we had asked only for information we thought was pertinent, we would have fallen short. If we expected for women’s experiences to remain somewhat constant despite the changes we saw in our communities we would have fallen behind.

The best organizations retain an understanding of the importance of each story. Youth Villages with its $80 million budget has a well developed research staff of 6.5. Sarah Hurley who heads that unit talks about the fact that program staff circulate multiple daily e-mail advisories that tell the story of each child who enters the system. “I don’t get the chance to look at them all,” she said, “but we all read some. Individual stories will always have a big impact here.”

Conversations with constituents is the foundation of good practice. This foundational practice should anchor everything including research and evaluation.

About the Author

Ruth McCambridge is editor-in-chief of the Nonprofit Quarterly