Measurement - The Great Dilemma

A recurring theme in the last few months is the challenge and need for nonprofit and ministry organizations to find a decisive way to self-evaluate. By extension, Catalyst and other grantmakers need criteria to evaluate not only applications but also the outcomes of projects after they receive funding.There are a few particular difficulties in this:-Reluctance: Nonprofit leaders are largely involved in matters and issues that are driven by compassion rather than efficiency. Many times they are resistant to the "corporate" emphasis on numerical evaluation. In my years at camps I saw the anger that was prompted when someone from the board asked how many campers had made faith commitments during a particular program. Staff found that offensive and felt it diminished the nature of relationships to something merely transactional and manipulative.-Ambiguity: Social services and spiritual projects are notoriously difficult to quantify. How do you measure the benefits of a relationship? Rarely is there a single point of emphasis and the people involved may all have different value and priority for the varied outcomes.-Tools: Cultural change doesn't lend itself to a simple bar graph. There is a shortage of recognizable methods for identifying and communicating the kinds of outputs and outcomes we're interested in supporting.-Objectivity: Nonprofit workers are almost always passionate about their work. (We wouldn't partner with any that aren't). Their clientele are understandably grateful for the most part for what is being accomplished. It is a lot to ask of either of those groups to provide a relatively unbiased perspective.-Hope: As may be the case in other fields (but I suspect is exaggerated in these ones), nonprofit and ministry people are optimists. They look for the signs of life in even the most desperate situations. It's a necessary prerequisite of much off what they do, but when applied to evaluation is clearly distorted.-Narrative: When numbers are hard to generate or interpret we rely on stories. Funding for charities has always been based more on tugging the heart strings than swaying the intellect. The traditional pitch of "a sob story and a slideshow" is deeply entrenched and typically effective. Anyone can come up with at least one compelling account of someone who's life is being bettered from their efforts.Despite all of these impediments there are efforts being made widely to develop useful and relevant ways of measuring the results of nonprofits. As I wrote about previously, Jim Collins has produced a monograph of Good to Great aimed specifically at the social sector where he argues for the necessity of determining standards of evaluation that are measurable. It is a very live discussion among the professionals I met at the recent PIGS conference as well.Prior to the start of Catalyst I was involved in starting a new church in our community. We were under the authority and support of the church where I had been staff for several years; and I reported to the leadership there. When after more than a year our new congregation wasn't significantly growing the leadership began to question the wisdom in continuing. Of course I resisted. I could see the sparks of potential and the impact we were having on the few people who were involved. Ultimately the decision was made to close the new church. It was difficult for all involved (new congregation, myself, and the leaders of the larger church), and was made all the more difficult because there was no standard of measure by which to evaluate what was happening.I admit that I find the process of determining objective measurement criteria for matters of spirituality and social justice to be both daunting and dangerous. Obviously we don't want to reduce the efforts of our partners to spreadsheet entries. At the same time, I have become increasingly aware that with those criteria established and agreed upon there is a freedom to pursue a vision with greater confidence that you have defined your purpose and won't be dissuaded by the inevitable swings of energy and enthusiasm.I am eager to work with our partners to figure out how to farily and helpfully evaluate their honourable efforts.