This is the last of three articles on ABFE’s (Association of Black Foundation Executives) 2022 conference last month, focused on emerging concepts and practices. The first is “Experiments in Liberatory Leadership,” the second “The Emergence of Black Power Funds.”

Evaluation approaches and practices are contested sites of power. This is not surprising, since they are the processes by which we assign value to things.

In the nonprofit sector, evaluation is shaped by the needs of philanthropy, as philanthropists drive the field with their ability to pay and demands for proof of the soundness of their investments. Evaluation is also a site for knowledge creation and strategic direction setting, as the reports that result are used to articulate an issue and make the case for efforts and funding.

When I wrote “The Leechers,” an article about knowledge and resource extraction in the sector, many evaluators of color wrote to share stories about the extent of knowledge extraction in the evaluation field. They told of evaluators of color leaving the field, frustrated with increased gatekeeping in relationship to the widespread movement for racial justice and the field’s demands for equity-focused evaluation, which often leads to white evaluator “allies” extracting, competing, and protecting their privileged positions in the field and market.

So, it is exciting to see evaluators of color innovate the field in new, more potent directions, away from the philanthropic gaze.

In a session titled, “A Black Feminist Framework for Research Evaluation as a Tool for Black-led Social Change,” Dr. Sydney McKinney, Executive Director of the National Black Women’s Justice Institute—which focuses on ending the criminalization and systemic punishment of Black women and girls—demonstrated how to use research as a tool for social change.

Dr. McKinney started by stating that rather than taking a stance of objectivity, NBWJI positions itself as an organization of researcher activists. The research it leads looks at how Black women and girls experience the police, focusing on not just fatalities, but the way everyday interactions are harmful. The Institute seeks to understand:

  1. What pushes Black women and girls into the criminal justice system?
  2. What do they need?
  3. What do they do after?

NBWJI began its work on Black women and girls because even though they are “overrepresented among those incarcerated at the federal, state and local levels,” there is very little data on this specific demographic group. Women are included in the criminal system’s data, but they are not segmented. NBWJI researchers go into the data sets that exist when it can, but since it is not engaging in conventional approaches and is instead centering Black women and girls and asking different questions, partnerships are important.

The Institute partners with Black-led organizations, working with those who are impacted to define its questions, create surveys, and integrate an ongoing relationship with data. Kenya Welsh, Director of Development and Strategic Partnerships, shared that the work begins with a pre-engagement needs assessment. NBJWI also looks for opportunities that are trauma informed, meaning that partners are skilled, or at least committed to becoming skilled, at designing with the assumption that the work may include people with a history of trauma. This allows staff to engage participants without retraumatizing them.

NBJWI projects integrate data design, collection, interpretation, and implementation into organizational project models in a seamless cycle of improvement.

For example, the Institute has a partnership with The Mentoring Center in Oakland, CA. The executive director, Celsa Snead, joined the session to share how such a partnership works. The Center runs EMERGE, an innovative school reentry program for girls aged 16 to 18 “who are returning to school from a condition of confinement or incarceration.” The overarching goal is “‘confinement to a college and career’ pathway for young women.” The specific goals are:

  1. Repair girls’ relationships with school
  2. Provide opportunities for accelerated credit recovery toward their completion of a high school diploma
  3. Facilitate their enrollment in an institution of higher learning and/or permanent employment
  4. Create a trauma-responsive environment that advances social and emotional learning and healing

This is a pilot program, and Snead said, “We wanted to make sure we are doing it correctly.” The program is exploring what it means to educate Black girls. It wants to expand the model, so the model has to be strong. Having the Institute as an original partner has significant benefits, Snead notes.

The Institute informed both program design and evaluation by accessing the girls’ voices. Thus, the evaluation was considered a key component of the work from the beginning. Snead said, “We felt good about the recommendations to make the program better. They were precise, tangible, and clear. Just the process helped us learn. We already started implementing what we learned to make improvements.”

How exactly did the evaluation process help? EMERGE made improvements to the model in both the curriculum and the school day. The evaluation also confirmed what program staff thought: that mental health care is needed, as well as other supports. The evaluation was also an opportunity to reflect on the work the program had done in terms of outcomes and impact.

Welsh said, “Talking starts the process. People want to talk. They want to be heard, validated, included. And they are more comfortable talking to someone that looks like them.”

Everyone who completes a survey is offered mental health support. Dr. McKinney said, “Paying people for their time is important, but so is mental health.”

However, “There’s not a lot of uptake from participants on the mental health services, so we have to figure out how to do it better,” Dr. McKinney said. She suspects that there is work to be done in getting participants to prioritize their mental health.

Snead said that while evaluation is critical in social change work, “a luxury that should not be a luxury,” it is important to find the right evaluator. However, while funders encourage and even require evaluation, Snead said, “It is rarely the case that evaluation is embedded in the program and supported by philanthropy. That’s a mistake, particularly with direct services.”

There are so many ways to leverage evaluations, but they require partnerships. Current practice is requests for proposals (RFP’s). There is no co-creation. Funders fund a person who gathers and analyzes the data. There is no continuous learning or internal staff. Dr. McKinney said, “We need to invest in the how, not just impact. How do you know you have impact if you don’t have the right model?”

Dr. McKinney said, “Funders could also do more with the public. Right now, most evaluations result in internal reports. Investing in data visualization is also critical. The work is to figure out how to get that information out more effectively.”

Further, Black research organizations tend to work with small budgets. Dr. McKinney said, “We have to ask for what we need.”

Dr. McKinney concludes, “A Black feminist approach to evaluation means that we own our bias, but our bias is supporting people and projects to be successful. The goal is to do the work so that organizations don’t need us. The evaluation is to build capacity.”

Thus, it appears that a Black feminist approach to evaluation states its bias up front: the betterment of conditions for a particular subordinated demographic group. Further, since the demographic group is subordinated, this approach centers group members and seeks to understand conditions from their perspective. And, while the work may be issue focused, it is also intersectional, seeking to understand the way different forces act on the issue. It is also trauma informed, with an understanding that oppression causes trauma and a commitment to, not only not contributing to trauma, but alleviating it. Finally, it is partnered, always tied to real action in the world.