Let's end the pro-forma accountability exercises and create something more meaningful
This Center for Effective Philanthropy blog on how to make reporting requirements more useful for both grantmakers and grantees is a year old. But, as with all good commentary on the nonprofit sector, it's still completely relevant.
If you’ve worked for a nonprofit you’ve been there: dragging yourself over the finish line to complete a funding report, trying to turn a spreadsheet with incomplete, inconclusive data into a cohesive narrative of your multiple successes. If you’re a grantmaker you no doubt have a stack of reports that you don’t have time to fully digest; the ones you do get to are as clear as mud. Blog author Jessica Bearman highlights the somewhat horrifying statistics that only “about half” of funding reports are used for internal decision-making, and only about one-quarter shared with the field.
How can we make reporting a useful exercise instead of a check-the-box requirement that doesn’t benefit anyone? Bearman provides some guidelines. They include grantmakers being very clear about what they want to learn; figuring out the best format in which to gain this knowledge; and determining ahead of time how they will follow up with grantees.
Bearman is agnostic regarding what should be reported. I am not. Many grantmakers currently require annual reports that focus on program effectiveness and expense accounting. These approaches are not very useful. A new approach is needed.
Human service (e.g., education, and healthcare) nonprofits all rely on the interactions between their staff and constituents to achieve results. A given organization has probably manualized its model; possibly staff are trained to deliver this model with quality and consistency. Even if these are in place, implementation involves interactions between dozens, hundreds - sometimes thousands - of people, all making daily, individual choices. It is these people - staff, volunteers and the constituents who are the focus of services - who will be the driving force behind any outcomes and impact produced by a particular program or intervention. Yet implementation is not usually well understood by either the grantmaker or the organization doing the work. What’s done to achieve results exists in the proverbial black box. Current reporting requirements – largely focused on accountability - leave this black box unopened and programmatic work underspecified.
If nonprofits and their multiple stakeholders are to discover and replicate successes, make a compelling argument for why and how their work has positive outcomes and impact, understand adaptations and obstacles - even define exactly what their program or intervention is - the black box of implementation needs to be opened and explored. Organizations with successful, evidence-based programs, such as Pathways to Housing and Nurse-Family Partnership, focus as much on careful implementation as evaluation of outcomes and impact.
Grantmakers can help more organizations open the black box through reports that facilitate grantees’ learning about their work. Questions that prompt organizations to dig into implementation will help both grantees and grantmakers. By deepening understanding of how work takes place in practice – rather than as outlined in a logic model - programs and interventions can be strengthened. With greater understanding of implementation, the connection between a particular program and outcomes, as well as the cost of running the program and what’s needed to build sufficient organizational capacity, can also become less opaque.
Nonprofits don’t need to start data collection efforts from scratch. Surveys have shown the majority of NPOs collect data around staff performance, customer satisfaction, and/or program activities. Quality data collection and data-informed improvement efforts are, however, much less prevalent. By asking targeted questions - and supporting organizations to answer them - grantmakers can transform data usage and grant reporting from compliance exercises into opportunities for learning and growth.
Here are some questions I ask when working with organizations around implementation and opening the black box.
- What do you know about your constituents – who they are, what other services they receive, how they participate in your program, how they experience it?
- How do you and your staff understand how work takes place “in the field”? How do you know if/how goals are being met? With what quality? How do you use this information to improve your work?
- What variations have you noticed in how your model operates within and across sites? Why do they occur?
- Where is your implementation strongest? Where are the greatest challenges?
- What do you not know about your work?
- What methods do you use to gather information from program staff and constituents? Who gathers, analyzes and reports this data and in what form? What systems do you have in place to look at the data you collect? Who participates in them? How do you act on what you learn?
What’s your experience been in this field? Have you attempted to open the black box? Nonprofit staff, are you comfortable talking to grantmakers about implementation? Have you gotten support to do this work? Grantmakers, have you looked with your grantees at their implementation? How do you build trust with grantees to explore their work more closely? We’re eager to get a conversation started!