MadLib Feedback

How Medium is using narrative MadLibs to make feedback more relevant + actionable, and the process more human-centered.

TL;DR. Performance reviews + peer feedback have been a painful box-checking exercise for decades. It’s time we drastically improve the experience and outcomes. Here’s a look into what we’re trying at Medium.

The mission of People Ops at Medium is to build a world-class team and enable it to do its best work. We strive to anticipate — and at the very least be lightning-fast in response to — everyone’s needs.

As we’ve been growing this past year, it’s become clear that effective feedback was atop the ‘unmet need’ list for our people (i.e., they actually wanted it, believed it would improve their performance, and make our product + company better). We did a little research (primary and secondary) and quickly learned that

few tech companies our size — and even those 10x bigger — facilitate any meaningful type of feedback. Yikes.

We knew an off-the-shelf system wouldn’t work, so we designed one ourselves.

Five (Not-So-Surprising) Guiding Principles

Informed by our research, personal experience, and some inspiration, we came up with five guiding principles:

1. Better Experience → Quality Input → More Meaningful Output. If the experience of reviewing people is painful and takes forever, you’ll get lousy results. Make the reviewer experience better.
2. Explicit Expectations. If expectations for what people are accountable for aren’t explicit, how are they supposed to know what to do and be measured against? Clearly articulate expectations.
3. Build for the Long Haul. Continuity from one review to the next is necessary over time to make it relevant, credible, and truly growth-oriented. Stitch them together over time, like a narrative that unfolds.
4. Guided, Not Dictated: People respect feedback from peers, while wanting guidance on what they should/shouldn’t focus on from managers. Source from both peers + managers, but the manager guides.
5. Goals Matter. Goal-setting is actually important, not a throw-away or something to be rigged (unless managers fear what happens when their people succeed, or their people are afraid of failure — in which cases there may be bigger problems to contend with). Obvious, right?

One more principle rose to the top after the fact — a rather simple one — namely that

people don’t artificially separate personal and professional growth in themselves, so why shouldn’t we bring them together in their feedback process?

The Why (+ What) of MadLibs

For those who don’t know, MadLibs is a fill-in-the-blank game started in 1953. I used MadLibs in my time at IDEO as a tool in business development conversations and saw others at IDEO using them for ad hoc feedback.

We simply took it one step further, using MadLibs as the key mechanic to source feedback.

Why?

The fill-in-the-blank format works for two main reasons:

1. Better Experience for Reviewers: Open-ended questions (e.g., “What should this person start, stop, and continue doing?”) can be debilitating and massively time consuming for the reviewer, and difficult to synthesize/make sense of. On the other end of the spectrum, numeric grading systems (e.g., “Gabe scores a 7 out of 10 on Leadership!”) can be reductive, arbitrary and inactionable — and often leave reviewers shrugging their shoulders.
2. Easier to Synthesize Patterns + Identify Outliers: Standardized fill-in-the-blank answers enable us to line up reviewers’ responses next to one another, recognize and distill patterns, and easily identify the outliers (which we are able to either dismiss or probe more deeply on).

Here’s what the first draft of our MadLib form looked like:

I can count on Gabe for ____________.

He is viewed within Medium as ____________. He directs his energy toward ____________, and does so with ____________ effectiveness.

At his best, Gabe is ____________, ____________, and ____________. His strengths include ____________ and ____________.

When things aren’t going well, he does/is ____________ and ____________.

Some areas for growth include ____________ and ____________. He could operate on a whole new level if he were to ____________.

In the next six months, I’d like to see him explore possibilities in ____________ and ____________, and stretch his skills in ____________. Specifically, I’d recommend he connect with ____________ or try ____________.

What’s an example of when he totally crushed it?

What’s an example of a situation he didn’t handle well?

If he were an animal, he would be a ________ because ________. [metaphors can be informative + lighten things up]

Five Steps, Three Parts: The Story of Gabe

OK, so how do we do this? We begin by setting up a Google Doc (or a Medium post) for everyone being reviewed, and dub it their “story.”

Let’s use The Story of Gabe as the example.

Here’s the process:

1. Step One / Assignment: Gabe’s manager, Naureen, assigns between 4-7 reviewers (including Naureen herself) to provide MadLib feedback on him.
2. Step Two / Fill-in-the-Blank + Synthesis: Frank, who fills the role of Performance Guidance (a role we have in our People Ops group), sends out the MadLibs to the reviewers, and subsequently synthesizes the responses into a coherent, compelling compilation.
3. Step Three / Let’s Talk About It: Naureen (manager) and Frank (People Ops) together sit down with Gabe for ~30 minutes to talk him through the feedback, get reactions, and begin to discuss what’s ahead. Naureen gives her take on it (“I like the suggestion to grow empathy skills, and wouldn’t pay as much attention to developing your JS chops”), and at the end of the conversation Frank shares the Google Doc (viewing rights only) with Gabe so he can always have access.
4. Step Four / Goal-Setting Time: Gabe has the next two weeks to develop stretch goals for himself, which come in the form of personal OKR’s. He needs help and looks to others (including Naureen, but also his friend Dan in engineering) to help him out.
5. Step Five / Closing Out Chapter One: Naureen, Gabe and Frank sit down one more time for 15-30 minutes to tweak OKRs and finalize Chapter One of The Story of Gabe.

Six months later we’ll do it again, and add it to The Story of Gabe — next time as Chapter Two. [Worth noting: in Google Docs as well as Medium posts, managers have the ability to add comments in the margins over that six-month period.]

In the end, what does Chapter One of the story actually look like? About three pages long, with three key elements:

1. Expectation-Setting. Atop The Story of Gabe is a list of his roles + accountabilities, so he is clear on what is expected of him. [Process Note: At Medium we use a software tool called GlassFrog to help us clearly + transparently articulate roles + accountabilities for everyone.]
2. Synthesized MadLibs. This is the meat of the matter, typically netting out at ~500-700 words. To make it constructive takes time, and this is where People Ops (typically HR, Performance Management, or Talent in other orgs) delivers real value — by holding an objective perspective while saving everyone else time to synthesize a meaningful narrative.
3. Goal-Setting (OKRs). Gabe sets 2-4 Objectives and 3-5 Key Results per Objective, intended to help him look ahead. They can range from personally-oriented (e.g., develop presentation skills) to more organizationally focused (e.g., develop Medium’s human-centered research capability) — it’s up to him, with input from Naureen and anyone else whose opinion he values. Nothing groundbreaking here, but to do it well takes the right touch.

Let Them Opt-In

At Medium, everyone’s manager has the autonomy to choose how they give their people feedback, if at all. And most of our managers believe in regular 1:1's in order to do it early, often and honestly. We also run debriefs after each project, which always include reflection on what worked well, where there were tensions, and what to focus on in the future. So MadLibs supplement + systematize much of what’s already happening on an ongoing basis.

Since we launched this with engineering back in April, we’ve subsequently gotten full adoption at the company. All opt-in. That’s exciting.

The responses from those on the receiving end have been overwhelmingly positive as well (from “shockingly actionable” to “the best feedback I’ve received in the last 20 years”). That’s even more exciting.

We know we have much left to learn, and there’s certainly room to tighten it up. We’re going to continue working on it.

Looking ahead to our Chapter Twos this fall, we’re hoping the sequel lives up to the hype of the original.

PS for HR + Friends

If you’ve read this far, and you’re in HR or talent development, there’s one key take-away I’ll leave you with.

Stop being a process-driven administrator, and start designing action-oriented systems rooted in the needs of your people.

Whatever you end up creating, chances are, it’s better than what they’ve got now.