Raw Thought

by Aaron Swartz

Goods, Services, and Delegations

Things you can buy are typically divided into tangible things (goods) and intangible actions (services). But recently I’ve realized there’s a much more interesting type of thing to buy: delegations.

A delegation is like a service, except instead of asking someone to do a specific thing, you ask them to achieve some goal. Hiring someone to paint your wall white would be a service, hiring someone to make your house pretty is a delegation.

Delegations are a lot harder than services. In the same way you can be pretty sure that when you buy a pen it will write, you can be pretty sure that when you hire someone to paint your wall white, she’ll actually do it. And if she doesn’t, you can just not pay her.

But if you want to hire an interior designer, it’s a mess. Let’s say you pick one by looking through their portfolio and concluding that you like their work. But when they come to design your place, you hate the result. What can you do? You say that what you got looks nothing like the stuff in the portfolio and they’ll just say that every space is different and so has a different result. There’s no way to ever prove they did a bad job.

And that’s something fairly inconsequential. Imagine you’re wrongly charged with murder or stricken with a potentially-fatal illness. Picking the right lawyer or doctor could make the difference between life and death.

But now there’s not even a portfolio for you to look at. Sure, you can see if the lawyer’s won a lot of cases or if the doctor’s kept most of her patients alive, but that doesn’t tell you much — it probably just means they’re either very lucky or mostly choose easy cases.

Perhaps instead of looking at outcomes, you could look at the decisions they made along the way. But even if you could get your hands on those records, how could you possibly learn enough about law or medicine to evaluate them? And even if you somehow tried, there’s probably all sorts of relevant specific details about the circumstances that could never make it into even the most detailed histories.

But it’s not just hard for the delegator — for the same reason, it’s hard for the delegatee. If you want to be great at painting walls white, it’s easy to get pretty immediate feedback about whether you did the job correctly. But if you’re a elementary school teacher, you’ll really just never know. You hope you’re helping your kids succeed in life, but there’s no way for you to check that. And what are the chances that you started doing everything right just by intuition?

No, the expert performance movement has shown the only way to get really good at something is to practice, continually comparing what you did against the results it achieved. But in any sort of delegated job this is practically impossible: the uncertainties are too great, the feedback loops take too long, the opportunities to practice much too rare.

Traditionally, we solve these problems by having an academic discipline figure out the right thing to do using scale. If doctors were just on their own, they’d still be no better than witch doctors: people would come in with problems and they’d pick a random herb or spell to try and pray it made the patient feel better. They’d never really know whether they were helping or hurting. But they’re not on their own: because medical schools can conduct randomized trials with hundreds of people, they can just read the results and learn what actually works.

A lot is still left to individual judgment — there’s not a medical study for every scenario and even if there was, you’d still have to choose how to interpret the results — but there is definitely a trend toward knowing more. And some of the most exciting developments in medicine come from replacing human judgment with checklists and decision trees.

But medicine is probably the best-case scenario. I’ve never heard of lawyers reading up on the results of statistical trials and aesthetics is so subjective and fashion so temporary that I doubt anything like this could ever be possible for interior designers. (Education is probably somewhere in the middle.) How do people ever get good at these things?

Part of why running a nonprofit is so hard is that pretty much all nonprofits are delegation. Donors aren’t buying a particular thing they know they want, they’re buying a chance to help others, without knowing exactly what it is they want. And that’s why randomized controlled trials have been transformational for the nonprofit sector — they’ve converted a delegation into a service. Great nonprofits don’t have to guess at what will help people the most; they just need to look up the most helpful service and then purchase more of it.

Poor Economics is a remarkable book if only because it shows how crucial this is. It’s full of tales of small-scale experiments where well-intentioned do-gooders try hard to help some people and fail catastrophically. But they only notice because there are academics there collecting data; in the typical nonprofit, where the decisionmakers are far removed from the evidence on the ground, they’d probably never know that much was going wrong (assuming that they even cared).

But “political” nonprofits don’t get off so easily. It’s fairly impractical to do randomized controlled trials of things like lobbying, public campaigning, white papers, investigative journalism, public relations, strategic litigation, electoral campaigns, and the rest. There are brave and noble efforts to try to improve some of the details most amenable to testing, but while you can test which direct mail flyer makes people more likely to vote, it’s hard to test whether GOTV mail is a good use of campaign funds at all.

Now you can get a lot out of combining all these things so you can make big-picture decisions about how to allocate resources. And you can get a lot out of having the same team do them over and over so they can build up institutional expertise. But there’s also a lot of room for learning within each one as well. Just as randomized controlled trials have revolutionized development nonprofits, I think political nonprofits will be revolutionized by developing institutional structures to formalize the process of learning from campaigns.

What would this look like? A good first step would be developing a series of case studies of major campaigns, successful or unsuccessful, to get some sense of big picture stories. From this, you could distill a toolbook of various tactics, with some notes on which seem to be more and less effective and some open questions or avenues of exploration for each. Then within each tactic, you could bring together practitioners to swap best practices and try to improve the state of the art.

Interviewing people right after campaigns also seems like a fruitful avenue. As they look back on what happened, what do they see as the big mistakes? The big successes? What do they wish they’d had?

These might all seem like minor, parochial concerns, but when you stop to realize that the world is full of huge problems that can only be solved by collective action, figuring out how to inspire coordinated action most effectively doesn’t just seem interesting — it seems essential.

You should follow me on twitter here.

July 18, 2011


Hi, Aaron. The http://igs.berkeley.edu/ newsletter has been reporting out the after-action interviews for decades. I’d be interested in seeing IGS use their access to campaigners and practitioners to turn the reports from anecdotes and lore to quantitative and systematic analysis.


posted by Phil Wolff on July 21, 2011 #

You linked to GiveWell. Part of their story that might have parallels in learning from campaigns: most big charities weren’t ready to open up enough for a rigorous evaluation. But GiveWell “works” anyway because they managed to find a few groups that are both great and transparent, and they help the field by publishing everything from each evaluation.

So maybe an effort to learn from campaigns isn’t sunk if some political folks aren’t ready to be candid about big campaigns. As long as some people are doing big/interesting/novel projects and are ready to share nitty-gritty (and maybe numbers), we can still get a lot out of it. And, honestly, the candid and reflective pros might be the ones most likely to teach us something.

And, uh, I was sleepily writing a rambly comment on this post Monday night, filed it away, and then things happened — good luck to you and your delegates.

posted by Randall on July 22, 2011 #

“A good first step would be developing a series of case studies of major campaigns, successful or unsuccessful, to get some sense of big picture stories.”

I hadn’t thought of this, I’m gonna’ do this.

posted by jonathan on July 23, 2011 #

You can also send comments by email.

Email (only used for direct replies)
Comments may be edited for length and content.

Powered by theinfo.org.