The Wisdom of the Chorus
On creative influence and incubation
Here’s a fun trick you can try with your friends:
Find a quantity whose true value no one in the group knows—the population of Uzbekistan, say, or the circumference of the moon, or the number of jelly beans in that jar on the counter—the more random the better. Have everyone write down an estimate. Then, one at a time, report out your estimates and average them together.
Reliably, this method—known formally as the “wisdom of the crowd”—produces a number that is remarkably close to the true value, generally closer than any individual guess. If everyone performs the exercise faithfully, you’ll frequently find that some individual people are wildly off; but in many instances these patently absurd guesses are the same ones that save the group’s average. (Indeed, the wisdom of the crowd works because everyone is wrong for different, uncorrelated reasons, and those errors tend to cancel out.)
It gets cooler.
In a landmark 2008 paper, two researchers found that the wisdom of the crowd can be invoked—albeit less dramatically—by having a single individual make the same estimate on two separate occasions. That is, if you write down the number of jelly beans in the jar, then come back another time and repeat the process (having forgotten your original estimate), the average of the two numbers is likely to be closer than either individual guess. Notably, the accuracy of the “crowd within” increases with longer delays—as the authors describe:
Simply put, you can gain about 1/10th as much from asking yourself the same question twice as you can from getting a second opinion from someone else, but if you wait 3 weeks, the benefit of reasking yourself the same question rises to 1/3 the value of a second opinion.
Now let’s make a small, intuitive leap.
The wisdom of the crowd (both within and without) applies not only to quantitative judgments but also to qualitative ones. Even though there is no “true value” to estimate, in other words, we can use the same approach to produce better creative work and subjective decisions.
Harnessing the Wisdom of the Crowd
Taken seriously, the wisdom of the crowd offers four clear suggestions on how to best marshal new ideas and projects into the world:
Be wary of letting collaborators influence each other.
Be wary of inviting collaboration too early or often in a project.
Strive to revisit projects intermittently over time.
Consider reproducing ideas from scratch rather than simply revising existing versions.
In a moment, I’ll tackle these claims one at a time—but first, note the key principle that underlies all four: The wisdom of the crowd works by aggregating independent judgments.
We’ve already seen, for example, how a single individual’s average guess is less accurate when the two estimates are closer together (i.e., less independent of one another). Similarly, the wisdom of an actual crowd can be erased if its members are allowed to communicate in advance: When one person throws out a number, it serves as an anchor that biases those that follow—even if everyone sincerely tries to make an independent judgment.
With this core mechanism in mind, let’s now consider how to best harness the wisdom of the crowd toward creativity and problem-solving. (As usual, my primary lens here is on the writing process, but I believe these insights apply broadly to projects in many domains.)
1. Be wary of letting collaborators influence each other.
Collaboration is, of course, a crucial tool with which to test and sharpen our ideas—and, nearly as often, a means to shape and expand them into something better. But problems arise when would-be thought partners fail to structure their collaboration appropriately.
I’ve already alluded to the danger of “anchoring” others’ estimates, and it’s worth pausing to specify the actual mechanism that often leads to groupthink: information cascades. In their recent book Noise, Daniel Kahneman, Oliver Sibony, and Cass Sunstein offer an illustrative example of this—a hiring process gone awry:
To see how informational cascades work, imagine that ten people are in a large office, deciding whom to hire for an important position. There are three main candidates: Thomas, Sam, and Julie. Assume that the group members are announcing their views in sequence. Each person attends, reasonably enough, to the judgments of the others. Arthur is the first to speak. He suggests that the best choice is Thomas. Barbara now knows Arthur’s judgment; she should certainly go along with his view if she is also enthusiastic about Thomas. But suppose she isn’t sure about who is the best candidate. If she trusts Arthur, she might simply agree: Thomas is the best. Because she trusts Arthur well enough, she supports his judgment.
Now turn to a third person, Charles. Both Arthur and Barbara have said that they want to hire Thomas, but Charles’s own view, based on what he knows to be limited information, is that Thomas is not the right person for the job and that Julie is the best candidate. Even though Charles has that view, he might well ignore what he knows and simply follow Arthur and Barbara. If so, the reason is not that Charles is a coward. Instead it is because he is a respectful listener. He may simply think that both Arthur and Barbara have evidence for their enthusiasm.
Unless David thinks his own information is really better than that of those who preceded him, he should and will follow their lead. If he does that David is in a cascade. True, he will resist if he has very strong grounds to think that Arthur, Barbara, and Charles are wrong. But if he lacks those grounds, he will likely go along with them.
…
Now suppose that Erica, Frank, and George are expected to express their views. If Arthur, Barbara, Charles, and David have previously said that Thomas is best, each of them might well say the same thing even if they have good reason to think that another choice would be better. Sure, they might oppose the growing consensus if it is clearly wrong. But what if the decision isn’t clear? The trick in this example is that Arthur’s initial judgment has started a process by which several people are led to participate in a cascade, leading the group to opt unanimously for Thomas—even if some of those who support him actually have no view and even if others think he is not the best choice at all. (100-101; bold emphasis mine)
While the authors are careful to note that this is a contrived example, its dynamics will surely be familiar to anyone who has ever made a group decision: Italian food? is floated, and suddenly everyone is deciding whether they’re in the mood for pizza or pasta. The boss shares their opinion first, and suddenly all dissenting views seem to disappear. In short, if truth seekers are not vigilant about the sequence and method of information collection, the results are likely to be skewed arbitrarily toward certain (early) contributions.
(Of course, there is one distinct advantage that groupthink offers: consensus. Independently generated responses, after all, are more likely to be contradictory responses. If your goal is to truly optimize a decision or identify every possible hole in an idea, then it’s worth wading through this conflicting feedback to get the best possible outcome. But if your goal is simply to make a decision, or to feel good about a shared project, then consensus may be perfectly desirable: Let the crowd chatter!)
What does this principle mean in practice? In an everyday context, it might mean seeking input from different friends independently, rather than throwing the question out to the group chat. In a writing or research context, it might mean sharing separate copies of the paper to different editors rather than letting everyone comment on the same document. In a facilitator role (running a class or meeting, say), it might mean having everyone jot down ideas first before opening the floor up to conversation—perhaps collecting or distributing those ideas afterwards, as they may not all have gotten proportionate airtime.
2. Be wary of inviting collaboration too early or often in a project.
If suggestion #1 is about keeping collaborators’ judgments independent of each other, suggestion #2 is about keeping your collaborators’ judgments independent from your own.
Early on in their development, ideas are fragile. They’re a little crazy, perhaps, their outlines a little fuzzy, and therefore more susceptible to outside influence. Again, getting input on these ideas isn’t a bad thing—quite the opposite!—but there’s a risk in seeking out feedback too soon. Social desirability may lead you, consciously or unconsciously, to take your project in a less bold direction. Early feedback may lead you to abandon certain ideas, or frame your thinking in a way that prevents the most imaginative ones from arising in the first place.
Indeed, in summarizing the research on collaboration, science writer Annie Murphy Paul describes how groups tend to stifle creativity:
When we’re constantly in touch with others, we all end up gravitating toward the same pretty-good-but-not-great answers. Research finds that people who keep the lines of communication perpetually open consistently generate middling solutions—nothing terrible, but nothing exceptional either. Meanwhile, people who isolate themselves during the solution-generation phase tend to come up with a few truly extraordinary solutions—along with a lot of losers. (128; bold emphasis mine)
(I also found this summary chapter helpful in understanding the specific mechanisms that reduce generativity in group brainstorming—and what kinds of facilitation can counteract these losses.)
Now, the truth here is a little more nuanced than “Don’t tell anyone about anything you’re working on until it’s ready.” Clearly talking to others can often serve to spark new insights, and sometimes we need to do a little research before we’re capable of generating ideas on our own. So the wisdom of the crowd doesn’t forbid such early collaboration, so much as urge a bit of caution: After all, we can always seek external input after we’ve wrestled with an idea on our own, but once we get feedback on our work, we can’t easily unhear it.
By the same token, your editor/collaborator can’t unhear whatever you tell them about a given project in advance; for this reason, it’s wise to let someone read or respond with as few prior conceptions as possible. Only then, once you’ve secured an unfiltered reaction, should you start to draw their attention toward specific problem areas of the project, or explain what you were trying to do with it. While directed feedback has a lot of value, corralling them toward it too soon will rob you of the opportunity to see what emerges organically.
3. Strive to revisit projects intermittently over time.
A near-universal experience among anyone who has ever tried to write: The burst of self-expression that in the moment felt like a bolt of lightning reads, the next day, as a pile of dogshit. (Also, more rarely, the opposite: The tedious extrusion of prose reads the next day as utterly pristine and effortless.) Most of us, in other words, are quite poor judges of the quality of our ideas until we’ve gotten a little distance from them.
The crowd within helps explain this phenomenon: Each time we show up to work on a project, we are a subtly different person than the time before. We may be in an unusually good mood or an unusually sour one. We may have been introduced to a new theory or concept that sheds new light on the project. We may have been primed to think about race or gender or socioeconomic status. Far from being a strain on the coherence of the work, these different lenses provide multiple angles of scrutiny that help pressure test it—and ultimately make it stronger.
Of course, the same is true—in fact significantly more true—of showing the project to an external editor. (Recall that in the original “crowd within” paper, the benefit of a second opinion was up to ten times that of reasking yourself.) But practical constraints often make a second opinion impossible: You don’t always have time, or don’t know who to ask—and even if you do there’s a limit to how much help you can reasonably request. So again, it behooves creators to develop a work as much as they can on their own—that is, through intermittent revision. (Even if external feedback is readily available, I’d still contend that it will be most useful after rigorous self-editing: Why expend that person’s limited attention on mistakes you could have caught on your own?)
To be sure, there are many benefits of breaking up a project that are unrelated to the crowd within. First and foremost among these: increased focus and motivation for what might otherwise be an overwhelming task. Here’s the writer Anne Lamott, for example, describing how she emphasizes incremental progress in her work:
After I've completely exhausted myself thinking about the people I most resent in the world, and my more arresting financial problems, and, of course, the orthodontia, I remember to pick up the one-inch picture frame and to figure out a one-inch piece of my story to tell, one small scene, one memory, one exchange. (17)
Lamott seems to be focused mostly on idea generation here—in other words, easing the process of writing. But her approach is useful, too, in the practice of thorough editing—that is, a better outcome of writing. Once a basic draft or prototype has been generated, its quality will be similarly improved by the use of “one-inch picture frames”: an intermittent, iterative series of evaluations—perhaps with a narrow focus each time on a particular class of concerns.
[11/29/22 edit—a better example: In Vernacular Eloquence, Peter Elbow writes, “Throughout this process we can repeatedly step outside our language and our thinking—walk away, talk to others, or put it aside. I find it particularly powerful to forget about it for a day, a week, or a year so I can come back to it with fresh eyes and often with a different frame of mind” (44)]
Want some empirical evidence? In his provocatively titled paper, “Which is More Productive, Writing in Binge Patterns of Creative Illness or in Moderation?” researcher Bob Boice found that “binge writing” (characterized by fast-paced euphoric bursts, less planning, and higher rates of errors) was associated both with more depression-like symptoms after the fact, and lower publication rates (compared to shorter, more regular writing sessions):
How much of the success of the “regulars” is explained by their greater quantity of writing? Probably quite a bit! Honestly, I’m not entirely sure how you would disentangle the psychological/motivational benefits of intermittent work from the pure qualitative improvement a la the crowd within.
(If I do try to imagine a study design that teased these two factors apart, I think it might look something like this: Group A writes a couple pieces sequentially, spending three days on the first piece then three days on the second piece; group B spends the same total amount of time writing but alternates between the two—a day on the first piece, a day on the second piece, etc. up to three days for each piece. My hypothesis is that while both groups would derive the psychological benefits of breaking the work up, the longer incubation period and intervals of stepping away/returning would lead to work of somewhat higher quality in group B.)
Ultimately, regardless of the mechanism, the message from both art and science is clear: Revisiting the same project repeatedly over time produces better results.
4. Consider reproducing ideas from scratch rather than simply revising existing versions.
Sometimes, of course, we don’t have the luxury of setting something aside for three weeks and coming back to it. Or even if we do, each return to the project just lands us back in the groove of familiar ideas: We’re too much under the sway of decisions we’ve already made. In both these cases, there is a simple solution to simulate the experience of fresh eyes—an imperfect fix, to be sure, but better than nothing: Start over from scratch.
In support of this notion, novelist Matt Bell advises writers to “rewrite, don’t revise” first drafts:
Here’s the part of my personal process that no one ever wants to hear about but that is, in my mind, the most necessary and productive thing I do, the real way I transition from the mess of the first draft to the tighter, better-made second draft: I retype my second draft from scratch, rewriting as I go…More than anything else, committing to this process will divorce you from the sentences you wrote while you were figuring out what your novel was, making way for new prose written with a fuller understanding of who your characters are, of what their story is, and of how the story might best be shaped. (89-90; bold emphasis mine)
(Matt also puts out an excellent monthly Substack,
)And in his cult classic Writing Without Teachers, Peter Elbow (yes, that Peter Elbow), offers a similar example, holding up the traditional model of writing against an iterative, “developmental” one:
Imagine writing something three to five pages long and fairly difficult. It’s not something you have to research (or else you’ve already done the research), but you haven't’ really worked out what you want to say. Perhaps it is a school essay. Or perhaps it is a short story for which you have an idea but no sense yet of how to work it out. To make the clearest contrast between the two ways of writing, let’s say that you can only give one evening to the job.
If you wrote this normally, you would probably write it more or less once, but as carefully as possible. That is, you would probably spend anywhere from 15 minutes to an hour on planning: thinking, jotting, making an outline, or all three. And you would try hard to leave yourself at least half an hour at the end to go back over it and make clarifications and changes: usually while copying it over. Thus, though there may be a lot of “getting ready” beforehand, and “fixing” afterwards, you are essentially writing it once. And while you are doing the writing itself you probably do a lot of stopping, thinking, crossing out, going back, rewriting: everything that’s involved in trying to write as well as you can.
If on the other hand you adopt the developmental model of the writing process, you might well try to write it four times, not once, and try to help the piece evolve through these four versions. This sounds crazy and impossible because the writing process is usually so slow and tortured, but it needn’t be. You simply have to force yourself to write. Of course the first “version” won’t really be a version. It will simply be a writing down in the allotted time of everything on your mind concerning the subject. (19-20; bold emphasis mine)
For Elbow—as with Lamott—the primary benefit of this method is psychological: Though seemingly less efficient, it helps us get over the anxiety and avoidance of not knowing what to say:
This method of writing means more words written and thrown away. Perhaps even more work. But less banging your head against a stone wall—pushing with all your might against something that won’t budge. So though you are tired, you are less frustrated. The process tends to create a transaction that helps you expend more of your energy more productively. (22)
But it’s not just that the writing flows more easily; there’s also a kind of deliberate blinding to the history of the document: At the end of each writing block, the writer is advised to look over what they’ve just produced and “try to figure out what it would add up to if the missing parts were there. Sum up this main point, this incipient center of gravity, in a sentence.” Then, in the next version, they should simply “Start writing again. Start from your previous summing up assertion.” By iteratively stepping back and starting over, the creator avoids having to work within the confines of what is already on the page. This operates in stark contrast to the traditional model of writing, where a piece morphs slowly, almost imperceptibly, and with less intention.
Another underappreciated advantage of starting from scratch: the ability to more easily compare between, and possibly combine, different versions. Nine times out of ten, of course, the most recent version will be best. But every once in a while, an early draft will be superior—or at least have a superior element that can be repurposed and included.
Certainly, restarting from scratch isn’t always realistic. Some projects are too big or resource-intensive to do more than once. (You can’t exactly reshoot a movie scene for scene, or rerun a ten-year longitudinal study.) Still, it’s worth considering whether there are small ways to emulate the underlying logic. This paragraph, for example, is the fifth version I’ve written making a virtually identical point, with slight variations each time on the topic sentence, and two examples pulled from an earlier draft.
Coda: Let the Chorus Sing
I’ve just spent a long time detailing the benefits of pooling independent creative judgments—both between different people, and within one individual. Yet as I reach the end of the post, make my honest attempt to sum up and assess what I’ve written, it strikes me that there is one important way the original analogy breaks down: The evaluations I’m discussing really shouldn’t be wholly independent of each other.
Creative judgments, after all, are not estimates that just happen to have the same target—isolated attempts to count the jellybeans in the jar; they are efforts to move a project toward a shared target, whose final form is subjective and indeterminate. The purpose of aggregating multiple perspectives here is not to produce some statistically “average” prose or idea but, rather, to blend different voices in pursuit of a higher purpose. This is not a crowd, in other words, but a chorus.
So although I think that we often err towards carelessness and idiosyncrasy in creative processes, I wouldn’t want to live in a world wholly free of human discretion either. For some projects, it’s appropriate that certain voices carry more weight, or that certain choices be made in light of an earlier context. Some “songs,” so to speak, will require different blends of individual talent vs. group harmony.
Still, regardless of how many solos it doles out, a good chorus conducts itself wisely.
TL;DR
Aggregating independent ideas produces higher-quality creative work. This counsels:
Minding collaborators’ influence on each other…
…and on you.
Revisiting projects intermittently over time.
A willingness to reproduce work from scratch.