I first began working in organized philanthropy more than eighteen years ago. Like many other program staff, I came from an academic background and had been trained as a researcher. I had also worked as an evaluator. While I understood that my new job was to make grants, I also assumed that an important part of the mission of foundations was to support and share promising practices with the field—and that evaluation had an important role to play in that process.

If you polled foundation staff today, I think that many would still agree with that assumption, at least in principle. If we are to maximize our impact, it’s our responsibility to evaluate our work and to use that information to improve our grantmaking. We should also share those findings with our colleagues and the field at large. Yet if you asked most foundation staff how satisfied they are with their respective organizations’ current efforts in evaluation, I’d guess that not many would say they are “very satisfied.”

There’s a lot of contemporary rhetoric about accountability and transparency, much of it stimulated by high tech “new money” with ambitious expectations regarding measurable outcomes, metrics, and return on investment. A good deal has been written about what foundations should be doing to rigorously evaluate their work, but a lot less has been publicly shared about what we’ve actually accomplished. Why? It’s my sense that, for most of us in philanthropy, how best to evaluate our work remains an unresolved dilemma.

I believe that as a field we’re deeply conflicted about evaluation. As with most other significant internal conflicts, a lengthy course of therapy is probably warranted (perhaps augmented by medication). Or at the least, perhaps we might benefit from a more honest public conversation on this topic. In the words of Joan Rivers, that noted evaluator of celebrity fashion, “Can we talk?”

Let’s begin with board members. They’re the ones who ostensibly “set the bar” for a foundation’s expectations for accountability. And if you polled foundation trustees, I would imagine that most would rate evaluation as a highly important activity for their organization to engage in. But when you probe further and ask how much of the foundation’s limited resources should be devoted to evaluation, I think you’d get a decidedly more mixed response. Probe even deeper and ask for examples where evaluation has been a significant factor in shaping their foundation’s strategy or influencing major funding decisions, and I think most would be hard pressed to come up with more than a handful.

Most foundation staff members I know are ambivalent at best about evaluation. Time spent on internal evaluation is seen as time taken away from other important “front end” activities such as meeting with prospective grantseekers, proposal analysis, and keeping up with developments in their respective fields. There’s also an instinctive distrust of external evaluators, who are not infrequently seen as potentially undermining their professional autonomy.

If successful grantmaking is largely about relationships, inserting a (potentially insensitive) outsider into the dialogue can be counterproductive from the viewpoint of both funder and grantee. Program officers’ prior experiences with formal evaluations also tend to be less than terrific. Reports often arrive too late in the decision making process and are too dense and/or too ambiguous to be useful. Moreover, they feel their grantees have legitimate concerns about the intrusive demands of evaluators and the fact that they rarely receive valuable information (or any information at all) back in return for their cooperation.

A fairly small number of foundations employ a director of evaluation. Given the factors already discussed, those individuals occupy a particularly challenging position within their organizations. It’s not unusual for board members to have widely varying (and often unrealistic) expectations for evaluation. Other program staff may not necessarily see the “evaluation person” as a colleague; indeed, they may be viewed as a potential adversary. That leaves the evaluation director dependent on the strong, consistent support of the CEO to retain their legitimacy. How often in the dark of night have these stalwart souls fantasized themselves in the role of Jack Nicholson’s grizzled veteran in the film A Few Good Men? You may remember the pivotal courtroom moment, when he confronts Tom Cruise’s sanctimonious young prosecutor with the immortal line “You want the truth? You can’t handle the truth!”

Grantees, meanwhile, are trapped. They know they must embrace the notion of evaluation and accountability if they are to receive funding. But consider for a moment their dilemma. Most nonprofits have multiple funders, each of which imposes its own expectations for evaluation. The time that organizations must devote to filling out reports, refining logic models according to the requirements of individual funders, and submitting to interviews, surveys and visits by evaluators can add up to a major (uncompensated) burden. There is also the cost of having to ask their clients to cooperate in the same process. The smaller the organization, the more significant this expenditure of effort becomes.

More often than not, evaluation is a taboo subject in the dialogue between funders and funded. It’s very difficult (and risky) for grantees to complain. After all, what does it say about them if they appear reluctant to have the quality of their work assessed? But quietly, when offered the opportunity to speak “off the record,” they frequently question the value of all this investment of time and effort. Does it really make things better? Are there commensurate tangible benefits for their work?

Evaluators may be the most frustrated of the lot. They often find themselves greeted with suspicion on all sides. Foundation staff, grantees and community residents can question the “diversion” of potential grant dollars into evaluation. If the evaluators are white (and most are) and working in communities of color, they are also frequently resented and challenged to demonstrate their cultural sensitivity and proficiency—an uphill struggle, to say the least. They are typically squeezed on one side to keep costs down yet confronted on the other with unrealistic expectations regarding attribution, precision, clarity, etc.

When it comes to reporting results, evaluators confront their biggest challenge. A lot can be perceived to be riding on their findings, yet the data are rarely unambiguous. And, certainly, it’s only natural for funders and grantees that have put a lot of money and effort into a project to want to see positive outcomes. It’s the rare practicing evaluator who hasn’t experienced pressure (usually quite subtle) from funders and/or communities to soften the impact of potentially negative evaluation findings. Perhaps even more frequently, they must deal with the widespread perception among key stakeholders that a project is successful. That can result in what Fay Twersky calls “ceremonial evaluation” which has in too many cases become an empty yet dogmatic data ritual—requesting it, collecting it, submitting it and blessing it, without ever using it!

Evaluators must walk a very difficult line. They’re dependent on their foundation partners for a substantial portion of their livelihood. The integrity of their work is also of paramount importance. Yet how many partners want to pay repeatedly for bad news? On the flip side, evaluators privately fret that their work too often goes unused. The old saw about reports ending up “on the shelf” is not just a stereotype, and it’s not just because some reports may be difficult to read. Sometimes the submission of a final report is even greeted with indifference. It may be that the person who commissioned the work has since moved on or that institutional priorities have changed. Perhaps most frustrating for the evaluator is how little of their work is ultimately shared with the field. How are we to develop knowledge if so many evaluation reports end up in foundation filing cabinets?

As a field, we are investing millions of dollars a year in evaluation. And, don’t get me wrong, I believe we receive substantial value in return for that investment. There are countless examples of successful partnerships among funders, grantees and evaluators that have enriched the work of all three. But it’s my hunch that those experiences are more the exception than the rule. While many foundations may not recognize themselves in the scenarios I’ve sketched out above, I think most would privately acknowledge at least some ambivalence about the way we currently evaluate our work.

There are a number of aspects of the culture of foundations that contribute to that ambivalence, among them:

• The pressure to keep staffing lean and administrative expenses low works against the kind of reflective internal culture that supports the thoughtful pursuit of evaluation.

• The constant drive to “get money out the door” keeps foundations looking forward and creates disincentives to spend time and resources looking backward at the results of prior grants.

• An emphasis on novelty and innovation feeds an institutional restlessness and impatience that leads to frequent changes in priorities (and a subsequent lack of interest in learning from past experience).

• The private nature of most foundation deliberations dictates against true transparency. What are the incentives for a foundation to publicly disclose that a favored project (particularly one that was initially funded with significant fanfare) has fallen short of expectations? How many institutional egos are sufficiently mature and well developed enough to candidly address lessons learned from their work in a public forum?

So where does that leave us? I would encourage every foundation that is less than totally satisfied with its current approach to evaluation to have the guts to tackle this question in a format that can provide for some honest exchanges. Just as I believe that there’s no single most strategic way to do grantmaking, I also believe there is no “best” way to do evaluation. So this conversation will undoubtedly be different in each foundation that chooses to take it on. But I’d like to offer a few suggestions to prime the pump.

First, I encourage foundations to try to view evaluation through the eyes of their grantees. One good way to do so is to put a group of trustees and grantees (or former grantees) together for a conversation with no other staff in the room. Guarantee the grantees anonymity in return for the truth. Let them lay out the multiple demands they face for evaluation and the limited resources that they have to respond to them. Ask them what evaluation information would be most useful for them to improve their work (isn’t that the ultimate reason we evaluate?) and try to work backwards from that. Ask them how the foundation might help them learn how to evaluate their own work more effectively. The resulting dialogue might go a long way toward more reasonable, shared expectations on the part of those who are ultimately accountable for the way the foundation’s dollars are spent.

Another breakthrough intervention would be to bring foundation staff, evaluators and grantees together for a candid conversation about their own worst and best practices in evaluation—and their own contributions to each. The foundation folks would have to lead off with a truthful assessment of their own actions to stimulate a comparable degree of revelation among the other participants. But if such a dialogue could take place, it could help reduce defensiveness and begin to lay the pathway toward really improving the effectiveness of evaluation.

Then, contemplate ways in which your foundation might lessen the evaluation burden you currently place on your grantees yet get better results for all. Is there a way that multiple funders might accept the same grant report, to prevent overworked staff from spending all their time filling out forms? Are there other ways funders might join forces in supporting a single external evaluation that all could share? At the very least, foundations should examine their current practices to make sure that grantees and their clients are adequately compensated for any time they are asked to dedicate to evaluation. It’s quite simply the right and respectful thing to do.

Finally, let me suggest some questions to stimulate your own internal dialogue:

What is our culture? If we’re a “fast twitch” impatient organization, does that present a fundamental mismatch with the more deliberate, ruminative and reflective culture that has traditionally characterized evaluation? What style of evaluation makes sense for us and our grantees?

What’s our standard of evidence? Is a balanced well-told story sufficient? What kinds of metrics really make sense for our style of grantmaking? Do we hold all our grantees to the same standards of scrutiny?

What’s reasonable to expect of our grantees? Are our evaluation efforts proportional to the magnitude and length of our funding? Are grantees and their clients adequately compensated for the time they are asked to devote to evaluation activities? Will they receive data that will help them improve their work?

What can we honestly expect to achieve with our grant dollars? Are we the sole funder or one of many funders of the work we support? What’s a realistic increment of change for the dollars and other resources we are investing?

I’ve been intentionally provocative in this piece and have overstated our challenges and understated our successes in order to encourage a more honest conversation about evaluation. I continue to believe that evaluation should be an important component of the work of foundations. But I also believe that much of what we’re currently doing is not as effective as it could be, particularly when viewed through the eyes of our grantees. The dollars and effort we invest in evaluation are too precious to waste, and we have too much to gain by doing it better.

Tom David is an advisor to foundations and other social benefit organizations on strategy, evaluation and organizational learning, based in Menlo Park, CA. He is beginning to build a Web site at www.tdavid.net.