Nonprofits Need to Invest in Evaluation
I am nearing the end of a series of articles on investments that nonprofits need to make to improve their effectiveness and impact. The premise of the series is that effective nonprofits need to do more than deliver programs and services to address a community need.Every time the media covers a nonprofit organization that fails to deliver on its promise to the community it casts a shadow on the entire sector. Simply saying that you are delivering effective programs is not enough. When funders are starting to ask, “what is your impact,” simply counting clients or food distributed is becoming less and less credible. More is demanded today. By strategically investing in your capacity your nonprofit can shift from doing good to creating an impact. Such investments are measured by the return that the investment produces and that brings us to the focus of this article. Nonprofits need to invest in evaluation. Period.
In the last couple of articles of this series, I have been referencing the convenience sample data contained in the 2016 Northwest Nonprofit Capacity Report. The messiness of the survey and self-reported data is, in my view, emblematic of the messiness in way many nonprofits measure and evaluate their effectiveness overall. For example, when asked on the survey to rate how effective they were at accomplishing their mission, the nonprofit responses averaged near the top of a 10-point scale, –“a great amount.” The high marks represent a healthy self-esteem but in the very next paragraph of the report, we read,
“Interestingly, in contrast to the high rankings for mission achievement, organizations ranked themselves lower in their use of data and program evaluation.… Ratings concerning the use of data to inform strategy and to promote continuous improvement were mid-range this year. In short, our analysis shows that relatively few organizations use evaluation findings for something other than reporting to their boards of directors. The…survey findings suggest that data and evaluation take a backseat to seemingly more immediate and pressing components of mission achievement” (p. 5 of the full report).
Buried later in the report is the fact that 15% of nonprofits even have a written theory of change. Arguably, impact is effectively demonstrated when evaluated against your theory of change (more here). So even while ranking themselves as “low” in using evaluation data and confessing that few have a theory of change (a basic evaluation tool), the respondents still rated themselves as highly effective in achieving their mission. Huh?
It is my opinion that the inability of a nonprofit to demonstrate its mission achievement through evaluation, while at the same time believing that they are achieving their mission, is a case of the “emperor having no clothes.” Further, beyond embarrassing oneself in public, it is ultimately a risk to the health and long term sustainability of the nonprofit organization to not invest in evaluation. So how does a nonprofit begin to authentically invest in evaluation? I would suggest the following ideas:
1. Start with your theory of change: You first have to understand what you are trying to accomplish before you measure it. Your nonprofit exists to meet some compelling community need. How you are trying to meet that need is critical to the DNA or your organization. A theory of change is a the written tool to document the relationship between what you do and the community impact you make. Without developing a clear understanding of your theory of change, evaluation is meaningless, despite what other might think. Invest the 2-4 hours in creating this tool. Get help if you need to but for goodness sake just get it done.
2. Match your evaluation to your stage of program development: Nonprofit program evaluation often develops along a well-worn path. At the most basic level, a program must demonstrate some face validity – something is happening and it looks like our intervention may be contributing to the change. From there, a program needs to work out the kinks so that the program is delivered with consistency and quality. Once consistency and quality are achieved, outcomes can be measured and, as the program matures, so does the evaluation sophistication and design. It is in the latter stages where attributing results begins to take shape. Matching your evaluation is an important first step.
3. Take the Long View: Recognizing that evaluation progresses from rudimentary to sophisticated requires one to take a long view of evaluation. I worked with an organization where it took almost two years to get access to school data tied to the educational intervention. It took as long to build a data collection capacity, and still more time to identify evaluation partners who could bring expertise and objectivity to the evaluation process. Data from initial evaluations is strong and, in a couple more years, this nonprofit will have findings that will withstand the rigor of peer review. In this way, evaluation will help inform the nonprofit’s growth and expansion. Quality evaluation takes time.
4. Build Partnerships: Evaluation requires a unique skill set that typically is not native to many nonprofit employees. Also, evaluations benefit from external eyes that have less at stake in the outcomes. For these reasons, evaluation becomes a team sport. Evaluation is stronger when combines internal and external resources and expertise.
5. Build an Evaluation Culture: A commitment to evaluation needs to permeate the culture of a nonprofit organization. If a nonprofit reduces evaluation to satisfying a funder requirement or compartmentalizes evaluation into the box of “extras we do after all the rest of the work gets done,” then evaluation will be lifeless and impotent. Conversely, if a nonprofit drives is actions from a true data-driven decision making model then evaluation can be transformative to an organization.
This series of articles is designed to promote leading edge nonprofit management practices and the strategic investments that support such leadership. Evaluation is one investment that leaders need to make. Evaluation requires time, money, and staffing but, for nonprofits seeking to make a difference, it is a nonnegotiable cost. Those nonprofits that invest in evaluation are those who typically are closer to the leading edge than those who do not. So step back and ask yourself not “are you effective in achieving your mission” but “can you demonstrate it?”
As always your thoughts are welcome.
~Mark
photo credit Luis Llerena
