The following are Bill Schambra's remarks to the Wallace Foundation on January 12, 2012:
You've taken a step this evening—having a notorious critic of one of your core values appear before you to present his case—that is extraordinarily rare in philanthropy, and you’re to be commended for that.
My meager commendation, you may find, is your only reward for having me here this evening, but I hope that’s not the case.
I take my bearings this evening from a quote from your Chairman’s Letter from 2004. As she put it, “The [Wallace] Foundation came to see its key resource as knowledge rather than dollars. Developing and sharing effective ideas and practices in our chosen fields is our most important stock in trade—far more than just giving away money.”
This desire to be loved for your mind rather than just your money is in fact the central characteristic of modern American institutional philanthropy, beginning with the Rockefeller, Carnegie, and Russell Sage foundations over a century ago.
Its distinctive characteristic would be to harness the new physical and social sciences to get to the root causes of problems, as opposed to decrepit, feeble charity, which merely put band-aids on problems.
This focus on the development and dispersion of knowledge was evident in their grantmaking.
It focused on building up the modern research university, with its emphasis on the physical and social sciences; on the modernization of professions like medicine, public heath, and social work, through the introduction of science into their work and building codes of professional conduct based on it; and on constructing new institutions like think tanks and civic research bureaus, to begin to reshape public policy on the basis of science.
It’s difficult to overstate the role that measurement was to play in philanthropy’s plans for America. Statistics, numbers, metrics were to be gathered en masse as the best way to probe social problems to their roots.
The virtue of numbers is that they seem to be unassailably objective, neutral, nonpartisan. Figures don’t lie, they believed, only later coming to realize that liars sure can figure.
Objective, nonpartisan numbers provided the political leaders of the time—the progressives especially—with an ideal way to dampen the raging political conflicts of the early twentieth century.
The politics of the day seemed to be hopelessly distorted by the narrow, parochial claims of political factions or by dangerously divisive and seemingly irreconcilable ideologies.
But everyone, they believed, could rally around measurements, which were to be gathered without regard to partisan preference, and which transcended interests and ideologies, pointing toward solutions that were indisputably in the objective public interest.
As the prominent Progressive (and founder of the New Republic) Herbert Croly put it, in modern, complex society, the "cohesive element" would be "the completest social record," which could be assembled only by social science experts "using social knowledge in the interest of valid social purposes."
We were assured by the Progressives that numbers could be gathered in a way that everyone agreed upon, and that over time, they would add up to real knowledge about society, that is, accurate descriptions of the way people predictably behave under specified circumstances.
A widely accepted metric web would spread over the face of society, guiding social research into root causes, enabling ready statistical comparisons of outcomes from a variety of conditions, pulling together agents of social welfare in unified, comprehensive approaches to problems, and, most important, deflating and defeating the groundless, unscientific snake-oil remedies peddled by mere partisan politicians.
Now, the first several decades of the twentieth century provided an ideal institutional setting for the development of this metric paradise. A closely knit cadre of massive progressive foundations dominated the landscape of public policy, dictating and coordinating the scope of policy research at universities and think tanks, unchallenged by other significant actors, this being well before the emergence of large governments.
But it’s clear today that we have come nowhere near the ideal of “Metric Nation,” and institutional circumstances are such that we are, if anything, moving ever farther away from it.
Instead of a handful of large foundations calling the shots for American philanthropy, we have tens of thousands of foundations. Notwithstanding incessant calls for collaboration and coordination, these countless foundations pursue an endless variety of purposes, very few of them related to the development of social knowledge.
Where once the nation’s leading universities and research institutions were critically dependent on foundations for funding and so for guidance, today some of them possess endowments that dwarf those of most foundations, and pursue academic programs without heed to philanthropy’s concerns.
Where once philanthropy was the leading player in national social policy, today government at all levels has grown enormously and dominates the landscape of virtually every public policy issue.
Government in turn provides substantial support for the social service nonprofits and cultural and arts institutions that once depended on foundations. Most non-government funding comes not from foundations, but from fees for service or individual contributors.
Finally, and perhaps most important, few today have faith in a unified, coherent, rational, scientific approach to public policy capable of producing universally measurable and comparable outcomes.
It’s not that measurement has become any less prevalent. Indeed, numbers saturate our news reporting, policy recommendations, and political speeches.
It’s just that no one believes any more that the numbers will of themselves resolve or dampen down our political disagreements, or settle policy questions.
We now know that liars indeed sure can figure—that both sides of any political dispute are able to generate reams of social science evidence, proving indisputably that a given proposal will either rescue or ruin the republic.
Policy research institutions are now understood to be not neutral arbiters of abstract metrics but highly partisan numbers factories for diverse political agendas. Numbers don’t settle political disputes—they have just become potent weapons in their prosecution.
Indeed, ironically, if measurement was valued initially precisely because it would settle the most important and divisive policy questions before us, today we know that those are exactly the questions it is least able to settle. The more salient and controversial an issue the less likely it is to generate a set of widely accepted metrics.
We may be able to conclude empirically that, say, serving wine at the Isabella Steward Gardner Museum will attract more young adults than if you don’t, though I suspect one could have deduced that without too much study.
But how about the larger issues that constitute much of our social welfare politics today and upon which definitive data would be incredibly useful?
Does school choice work? Is Teach for America a good way to recruit teachers? How is welfare reform doing? Does Head Start produce lasting gains? What’s up with global climate change? What caused the recent great recession?
All of these questions, once thought imminently amenable to measureable answers, and urgently in need of such, given their divisiveness, instead generate furious storms of diametrically opposed but metrics based answers.
Measurement hasn’t tamed politics. Politics has seduced measurement.
A foundation like Wallace must then pursue its knowledge agenda under circumstances that are severely constraining and delimiting.
Given the collapse of any hope for an all-embracing, widely accepted social science metric, Wallace and other “knowledge-generating foundations” can no longer trust that their findings will fit neatly into larger frameworks containing widely accepted findings, because these simply haven’t emerged.
There are, rather, hundreds of different frameworks for collecting outcomes data, with thousands of consultants, each able to prove empirically that his or her approach is the best way to generate accurate readings.
There are tens of thousands of foundations selecting issues to study and measure based not on a shared, coherent, or encompassing research agenda but rather on factors ranging from careful investigation to a donor’s quirky and ill-considered intentions to whatever cause a board member may have discovered at last week’s cocktail party.
Measurements galore may emerge from this state of affairs, but only rarely and accidentally will they add up to a useful general proposition about public affairs applicable to more than a severely limited set of circumstances.
And even if measurable propositions are generated, they no longer bear the authority that science and objectivity once bestowed on metrics.
Published reports of measureable results might once have commanded respectful attention from and possibly replication by other social agencies.
But today, such numbers are likely to be viewed not as objective or neutral guides to social policy but rather as metric-tipped political missiles, to be regarded with suspicion and immediately countered by numbers proving the diametrically opposed case.
Conservatives, for instance, have not failed to notice that most foundation studies tend to follow a certain well-trodden path.
No matter the problem, the solution is inevitably a larger dose of the appropriate social services applied at ever earlier points and covering ever wider areas of human development; delivered by an expanded cadre of credentialed providers, who themselves urgently require ever more sophisticated professional training; and rewarded subsequently by ever better pay and benefits—all of which, of course, add up to more encompassing and expensive government.
However accurate by some arcane statistical standard such studies may be, their utter predictability and uniformity cannot fail to arouse skepticism among those opposed to expansion of the welfare state, and hence a certain cynicism about measurement itself.
Meanwhile, consider the daunting obstacle course to be run by nonprofit groups that seek funding from the “knowledge-generating” foundations under such bewildering circumstances.
Most nonprofits have developed programs based on the immediate, day-to-day encounter with the specific, idiosyncratic communities they serve, hence exhibiting the almost infinite variety of those communities.
But to win funding from a knowledge-generating foundation, the nonprofit must shoehorn its real-world work into the abstract, unfamiliar professional jargon to which data accumulators resort when they wish to generalize across (that is, to make disappear) the varieties of particular experiences.
The ability of a nonprofit to attract funding from multiple sources is, of course, an essential ingredient for a successful grant request. So that means it must recast its programs into as many different languages and metric frameworks as the foundations from which it seeks funding.
And this vast tangle appears before you add in the very substantial burden of measuring and reporting imposed by government funding.
Under such circumstances, only the largest and most established nonprofits—that is, those that are already deeply enmeshed within the unsatisfactory status quo—are likely to have the ability to deal with metrics-based funders.
Why have foundations persisted in their metrics mania in the face of these problems? Well, that splendid philanthropic independence, that detachment from the pushes and pulls of the real world that allegedly enable boldness and creativity, also permit foundations to evade the evidence of their own failures.
They are surrounded by consultants, academics, and professional experts whose livelihoods depend on selling and applying measurement tools, and by grantees who cannot afford to complain that the reporting requirements of a grant are unreasonably burdensome.
Occasionally, reality unmistakably intrudes—a much-courted district superintendent resigns, a supportive mayor is defeated, budgets are ruthlessly slashed across the board—and suddenly the project that was generating such encouraging numbers vanishes in an instant.
This should be the moment when foundations realize that metrics, no matter how promising, do very little to sway policy decisions. Instead, they tell themselves that were it not for this one little election or unfavorable school board vote or budget crisis, the project would have worked wonderfully.
Foundations simply cannot face that fact that the old politics of interest and ideology—which numbers were meant to tame—in fact still rages beyond the tiny, fragile metric oases so painstakingly and evanescently carved out of the howling political wilderness.
Most foundations will never hear the voice you’ve heard tonight—a voice introduced to you courageously and generously by your own evaluation staff.
Ed Pauly and his colleagues have thereby suggested for us the best way to judge the progress of one’s programs.
It is by inviting and cultivating robust and open discussion and disagreement among a variety of points of view, whether armed with numbers or not.
Because this is challenging, uncomfortable, and, given their insulation, unnecessary, most foundations will simply stay within their comfort zone and avoid, if not discourage, such debate. Congratulations to you all for choosing otherwise.
Will Miller, President of the Wallace Foundation, responds to Schambra's remarks:
Table Talk: The Other Half of the Conversation
Bill Schambra is a sharp and eloquent critic of The Wallace Foundation’s approach to philanthropy. So when we invited him to our January board dinner, we knew that while we’d set the table, Bill would serve the food for thought.
Let’s start by conceding that we savored some of Bill’s observations. We’ll come back to the portions we left on the plate.
To understand Bill’s critique, you have to understand how Wallace, with a number of other foundations, “does” philanthropy. We use our money to test ideas with potential for driving widespread, beneficial change. One example: We believe if principals receive the right training and support, they could help turn around the nation’s failing schools. Therefore, we fund efforts to develop ways to improve school leadership. We then study those efforts to see what works and what doesn’t, and share widely what we’ve learned.
We readily concede that this “evidence-based approach,” as we call it, has risks. Bill correctly notes, for example, that think tanks, which foundations like ours regularly rely on to conduct research, are often seen “as highly partisan number factories.” He is correct that the power imbalance between grantor and grantee means “grantees can ill afford to complain.” We agree, too, that foundation emphasis on “metrics” can force potential grantees to “shoehorn their work into metrics buckets.”
Bill fails to note, however, that these risks can be mitigated. Skepticism about think tanks? Wallace goes out of its way to work with researchers known for independence, objectivity and credibility, and we publish all evaluations, no matter what they find. Grantees afraid to speak truth to power? We join hundreds of foundations in asking our grantees to take part in a biennial, anonymity-guaranteed survey (by the respected Center for Effective Philanthropy) that gives us candid feedback on how our behavior stacks up against that of our peers – which enables us to work on fixing problems. Metrics mania distorting nonprofit endeavors? We select grantees already committed to the areas we are funding, so our grants do not torque them off mission.
I could go on, but there’s a bigger point to be made: Our approach can accomplish a lot of good. Just ask the scholars who have documented an American history replete with examples of the key role foundations played in building the evidence that inspired the spread of innovations such as:
* The Green Revolution (Rockefeller Foundation),
* The 911 emergency response system (Robert Wood Johnson Foundation),
* Sesame Street (Carnegie Corporation and the Ford Foundation).
In the world of policy, there are many other cases – early childhood education and welfare reform, to name two – where foundation-funded evidence was crucial.
Bill suggests that foundation-generated “evidence” leads invariably to government expansion. Really? In our work at Wallace, we’ve seen how evidence helped prompt the reallocation of tax dollars from lower- to higher-quality after-school programs for disadvantaged children. Moreover, evidence can put the brakes on overconfident claims of effectiveness that Bill rightly criticizes – and give foundations and grantees a better chance of helping society identify workable solutions as well as what falls short.
In the public sphere, we know evidence alone is insufficient. Politics, local experience, and solid evidence work together – raising new questions, uncovering unnoticed problems, highlighting failures and successes, and taking into account community context and needs. Throwing up our hands in frustration that measurement is messy could lead to the dangerous conclusion that facts don’t matter in democratic deliberation. They should and do.
There’s another issue. If strategy and metrics are futile, how is a foundation like Wallace, with its original donors long gone, to decide what grants to make? It’s one thing to back an initiative that seems to make sense, but for which there is little or no evidence, when you’re spending your own money or, at least, acting on clear instructions from the founders. It’s another when you are entrusted with other people’s money and asked to exercise judgment. We believe good stewardship compels us to act with clear strategic logic, ensure our work is aligned with our goals, and measure whether or not we are achieving them.
Breaking bread with a self-described “notorious” critic of your approach is an act of faith – that both sides can listen and learn from each other, that debate sharpens thinking. In my experience, civil conversation almost always bears this fruit. One small example: The day after our dinner, the board reviewed a draft values statement and replaced the word “metrics” with the word “evidence.” Why? Bill’s comments had reminded us not all evidence comes from measurement. At the same time, our dinner dialogue reaffirmed our belief that one of the most potent ways a foundation can contribute to progress is to help forge new ideas in the crucible of real-world experience.
Will Miller is president of The Wallace Foundation in New York City.