Try My Elixir
Evaluation directors could learn a thing or two about how to "sell" the evaluation "product." Here are ten ways to communicate the value of evaluation to grantmaking colleagues.
There's a reason evaluation directors are about as welcome as a hair in your soup.
Knowing our work is not integrated effectively into the business of philanthropy, we yield to the professional temptation to raise the volume about our underused brainpower and the lack of appreciation for our work. We are missionaries trying to convert those not baptized into the faith. On top of that, our products are often too late to be relevant or too complex to help much.
I know because I'm one of those evaluation directors. In a grantmaking process that seems like it would move smoothly but for my input, my job can be sort of like poking sticks into a kid's bicycle spokes.
So when a trustee of the foundation I work for cautioned against "worshiping at the altar of evaluation," I figured there had to be a better way.
There is. It requires evaluation directors to quit spouting off about how much we know. Instead, evaluation directorslike any good elixir peddlermust show that the merchandise adds value to the work our colleagues regard as a priority. Like it or not, that means meeting the needs of our customers, not the evaluation directors themselves.
It's a fatal mistake for evaluation directors to assume anyone else cares much about how we do our jobs. Our work has no value, per se. It is only valuable to the extent it advances the mission of our organizations.
The only way we can get anyone else to care about our work is to reveal in a pleasing manner how our so-called elixir will improve their lives or their chances for professional success.
That's hard to do. But it's not impossible. It means evaluation directors must push statistical techniques into the background and become highly skilled communicators and facilitators.
Imagine if the inventor of a remarkable kitchen device refused to belittle himself by hiring a TV pitchman to package and market what the creator and his peers see as a noble and useful tool chefs everywhere should use. Why should the truth need so much explaining?
That's sort of how evaluation directors see things. We are flabbergasted that others do not share our passion for accoutrements we believe would greatly improve our grantmaking. Even when we mean welland evaluation directors I know certainly doour frustration makes our actions inelegant.
In fact, we can find direction in our beloved social science. Communications theorist Everett Rogers, author of the classic text Diffusion of Innovations, has led the way in synthesizing more than seven decades of analysis that describes why some new practices spread and why some do not.
Pitching our services, we would do well to position our wares so that they are compatible with the customs of potential users. Without a doubt, it helps if our colleagues believe what we are selling has some relative advantage over the status quo. The idea should not be overly complicated.
It will help to let potential users observe others tinkering with evaluation before they have to decide whether to buy in. It also helps to allow time for trial runs, à la the money-back guarantees that give TV shoppers a no-lose chance at experimenting with their new purchase.
As one of several foundation evaluation directors working together to find ways to do well by doing good, let me share ten specific tactics I recommend as surefire ways to improve our fit in the business of philanthropy.
At various times, I have used all of the approaches I describe below. I don't always follow my own advice, but when I do, things tend to turn out favorably.
Of course, I am aiming for a partnership between evaluation directors and other foundation staff; thus, these tactics have implications for how we work together and what each of us brings to the joint effort.
1. Reduce the foundation's expectations for big results while improving basic documentation. Get used to saying: "No, I don't think this grant will change society, but yes, we should fund it." Our colleagues want our support in getting the board to approve their projects; anything likely to sink a proposal will be avoided. We'll make friends once we unhinge our support for projects from expectations of grand outcomes. This helps remind staff how rare it is to find a gem of a project that meets high standards of evidence. It's also a nonthreatening way to persuade them to lower the tone about expectations. In the end, this avoids setting up the grantee for failure.
2. Pitch our services as an addition tonot a replacement forinformation sources colleagues are already comfortable using. Indeed, ask colleagues to list the information sources they use to make decisions at all stages of the grant process. Armed with this, we can offer evaluation tools that fit alongside ofand do not replacethese resources. A reliable way to make allies on the program staff is to start with tools that improve the value of their site visits.
3. Work through the path of least resistance. Find a staff member inclined to use evaluation tools. Work closely with this colleague. We'll get more return on our investment this way than by arguing with staff members who don't want to buy what we're selling. When staff members see some of their peers getting this new type of supportand benefiting from itthey may develop an interest.
4. Don't attempt to evaluate everything. For that matter, don't focus on growing an evaluation staff. Make evaluation a scarce commodity and maintain high quality. We will only dilute our value by spreading ourselves thin. In fact, all we really need is oneone!gold-plated example that the president views as directly useful to the organization's mission. That success will increase demand for our services, so we should pick our spots for the rigorous evaluations.
5. Focus only on the future. We may believe that dozens of current grants stink, but what's done is done. Further, forget about grant summaries staff members are rushing to mail to the board. Set up quarterly conversations with individual colleagues to hear about ideas bubbling for next year. Then, when program officers come to us with three-year-old grant files and ask how the foundation should evaluate the finished project, we can tell them to trust their judgment. Then we can talk about the future. Once we convince our colleagues we are not interested in playing the "gotcha" game and criticizing past efforts that are dearly personal to them, we can make the conversation about program development and planning. This lowers the stakes and returns the focus to the work. Things won't always run smoothly, but the arguments will be more productive.
6. Continuously observe, listen and watch staff behavior. Look for colleagues who hit bumps in the road or experience problems planning for future grants. Then step up and offer ideas that can help them. Make the conversation about their work, not ours. Offer a menu of planning and evaluation solutions and let them pick the one they like best. And don't be fooled into thinking if we call our work something other than "evaluation" it'll automatically be embraced. This is about the work we do, not what we call it.
7. Get serious about packaging and translating. No doubt, there is a big pile of information on the desk of every evaluation director this very minute. But no one cares. And none of it matters a whit if it's not presented in a mannerand at a momentthat maximizes its relevance for decisionmakers. No one is going to read the 75-page literature review we loved. While we may wonder why trustees are not smart enough or interested enough to read our brilliant papers, remember, they are simultaneously wondering why we're not smart enough or interested enough to repair our turgid prose.
Write drafts. Get feedback. Repeat cycle. Make it mercifully concise. Folks will ask for more if they are interested. If evaluation directors have a point-and-click staff, create electronic deliverables that let them customize their content.
8. Volunteer for every facilitation gig, and get good at it. Expertise on the technical stuff is cheap and plentiful. Talent in facilitation is scarce and, thus, more highly valued. If we want to secure our place in the organization, we should develop and offer this service to anyone who needs it. It will put us front and center in the real conversations about program development.
9. We cannot promote our services or advocate for resources based on our ability to gather or crunch information. Instead, we must describe how knowledgehowever we come to possess itcan be used to produce the change our colleagues value. If we assume ready and equal access to all knowledge that exists in the world, then the winners are those who put the knowledge to some social good. In fact, as best we can, we should leverage utility from existing knowledge before making the case for new research. Existing knowledge may be skimpy, but there's more available than we regularly put to use.
10. As often as possible, we must get ourselves on the program at meetings or conferences. Most of all, make presentations to people other than our fellow evaluators. Perfecting our act in front of evaluation peers won't help us back in the office, in front of our board or with our foundation's grantees. Treat meetings and conferences as ready-made professional development opportunities with cheap focus groups eager to give us feedback. Sort of the evaluator's version of off-Broadway. We should not take our act to the president's office until we've auditioned it elsewhere.
Now, if evaluation directors are willing to do all this, what's fair to ask of the foundation's leadership? Of our colleagues on staff?
What's fair is this: Leadership should align foundation rewardsformal and informal oneswith behaviors it desires. Even when leaders explicitly support new and different values, staff will keep doing things the same old way until the informal reward structure is aligned with the emerging ideals.
Further, it's fair to expect to work with staff members who bring to their jobs high levels of curiosity and a willingness to challenge ideas and have theirs challenged.
With all of this, there will remain a friction between evaluation directors and program staff. But it's a win for everyone if we can turn it into creative friction.
John Bare is director of evaluation for the John S. and James L. Knight Foundation in Miami.