Don’t Tell The Donor

Benchmarking With A Warped Stick

By “A Fundraiser …”

A lot of nonprofit fundraisers seem to love benchmarking studies.

I really can’t blame them. Fundraisers are always judged by their numbers, after all. If we are going to be compared what we raised during the same period in previous years it only makes sense to understand the context of giving to other similar organizations.

Anytime your fundraising program experiences large increases or decreases, it’s helpful to understand if the change is being driven by broader external factors or by specific issues to your audience and mission. Benchmarking studies can help identify trends in key indicators and give fundraisers the context we need in order to understand our own performance better and plan for the future.

Unfortunately, not all benchmarking studies are created equal. Some studies are nothing more than lazy half-assed analysis from vendors hawking thinly veiled sales pitches. Other well-meaning benchmarks often use questionable methodology. A flawed approach can produce misleading conclusions.

It’s therefore critical for fundraisers to share their feedback and reactions to what the benchmark studies are showing. This article compares the methodology of three of the most cited (and debated) benchmarking studies making their way around the blogosphere today. 
Perhaps the most well known fundraising benchmark study is the annual report on philanthropy published by the Giving USA Foundation as a public service initiative of the Trust for Philanthropy of the American Association of Fundraising Counsel (AAFRC). These are the people who received a lot of media attention by estimating that Americans donated nearly $300 billion to charity during 2006.

Giving USA’s annual “estimates” are based on original surveys of organizations and econometric studies using tax data, government estimates for economic indicators, and information from other research institutions. Giving USA uses data estimates from the Internal Revenue Service, the U.S. Department of the Treasury, the Bureau of Economic Analysis, The Foundation Center, INDEPENDENT SECTOR, the Council for Aid to Education, the National Center for Charitable Statistics at the Urban Institute, and the National Council of Churches of Christ.

While the Giving USA study should be commended for trying to report estimates for the percentage of change in giving to subsectors (health, arts, education, religion, etc.) there also seem to be some flaws. First, in compiling the study, the Center on Philanthropy at Indiana University relies on self-reported surveys from more than 1,300 organizations. This self-selection from those that send in completed forms might not represent a fair, randomized cross section. The report also counts money given to foundations as well as grants the foundations make to nonprofits and other groups. Therefore, some money appears to be counted twice.

In contrast to the Giving USA study, the Target Analytics Index of National Fundraising Performance attempts to benchmark the performance of 70 large national organizations. Rather than trusting self-reported surveys, Target asks fundraisers to submit their raw data to compare trends in key fundraising indicators every quarter. Individual participants are offered detailed statistics and organizers publish a summary of their findings.
The most recent report showed ongoing and persistent donor declines during the past two years, which was temporarily offset by increases in revenue per donor. When adjusted for inflation, the benchmarking study shows that real index revenue declined a median -2.8% from 2006 to 2007.

There is significant debate of whether the observable trends that are found in national organizations can (or should) be extrapolated or applied to the thousands of local fundraising operations. Furthermore, the findings measure revenue directly without accounting for changes in strategic spending that can impact changes in the scope of fundraising programs.
Finally, an eCRM software and services company named Convio recently publicized its second annual Online Marketing Nonprofit Benchmark Index Study. The report covers metrics like traffic, email file growth, and online giving for organizations to measure their progress against. These guys even tried to market the report by recording two interviews to discuss the findings on YouTube. 

Unfortunately, its methodology is limited to the clients they serve and arguably it fails to offer broader industry trends beyond their 400 clients. You can’t fault the sales team for wanting to use a study like this as a lead generating white paper… so, if you don’t mind registering – you can download the report here.

There are dozens of other benchmarks that have been presented at conferences so far this year measuring many different aspects of nonprofit fundraising. As I mentioned at the beginning… some are more geared toward making a sales pitch, while others truly attempt to generate debate and conversation within our industry.

No single report can be expected to tell the full story, so development directors should be warned not to get lazy and attempt to copy and paste a summary they find online into their next Board report.

On more than one occasion in my fundraising career, I’ve been in a fundraising department that bucks the overall trend quiet dramatically. Reading through these benchmark studies can be boring and painful, but on every occasion, understanding why my organization varied from the “norm” proved to be a very helpful exercise.

*Editor’s Note: Don’t Tell The Donor is one of the hottest blogs in the sector. It’s written anonymously because the author is well known in the sector and he/she/its bosses wouldn’t be pleased. Be assured, The NonProfit Times knows the author’s identity, at least enough to write the check. You’re going to have to trust us.