apply-merit-selection

Apply Merit Selection

When evaluating candidates for selection, promotion, or recommendation, test your judgment against the reversed-relationship check: would you make the same decision if this person were your personal enemy? If not, bias — not merit — is driving the evaluation.

Why This Is Best Practice

左传 襄公三年 (~570 BC) — 祁奚举贤:

外举不弃仇，内举不失亲。

"Externally recommend even your enemy; internally do not withhold recommendation from your own kin."

Why best: When the senior minister Qi Xi (祁奚) was asked to recommend a replacement for his own position, he immediately recommended Jie Hu — his personal enemy. When asked again for a recommendation later, he recommended his own son, Qi Wu. In both cases, the recommendation was on merit alone. The historian's commentary: "Qi Xi was impartial — he recommended his enemy without favoritism against him, and recommended his family without favoritism toward them." The principle is not that personal relationships are irrelevant, but that they must be explicitly counteracted: you test whether your evaluation would survive if the relationship were reversed. If it would not, the evaluation is not meritocratic.

Cecilia Rouse & Claudia Goldin — "Orchestrating Impartiality" (2000, American Economic Review): The most empirically rigorous study of bias in talent evaluation. When symphony orchestras switched from sighted to blind auditions (screen blocking the evaluators' view of the performer), the probability of a woman advancing past preliminary rounds increased by approximately 50%. The auditioners believed they were evaluating on merit before the blind audition was introduced; they were wrong. The performance itself did not change — only the evaluator's knowledge of the performer's identity changed. This study is cited in virtually every discussion of selection bias and is the empirical foundation for structured, blind evaluation processes at Google, Deloitte, KPMG, and others.

NFL Rooney Rule (2003): The Diversity Advisory Committee of the NFL established a requirement that teams must interview at least one minority candidate for any head coach opening. The finding that motivated the rule: teams were systematically not including minority candidates in consideration pools, not because of deliberate exclusion but because hiring decisions were made through informal networks where minority coaches were not present. The Rooney Rule forces the evaluation pool to include candidates outside the default relationship network. Since its introduction, the percentage of minority head coaches in the NFL increased significantly. Adopted in modified forms by Amazon, Cigna, Microsoft, and others as "diverse slate" requirements in hiring.

Iris Bohnet — "What Works: Gender Equality by Design" (2016, Harvard Kennedy School): Bohnet's research synthesis covers 50+ years of bias research in talent evaluation and reaches a consistent finding: awareness of bias, training about bias, and intentions to be fair do not reliably reduce bias in individual evaluation decisions. What works: structural interventions that change the evaluation environment — blind evaluation, standardized criteria, structured scoring. Bohnet's key finding relevant to apply-merit-selection: the single most effective behavioral intervention is evaluating candidates jointly (comparatively) rather than individually. When you evaluate candidates side-by-side against identical criteria, individual relationship bias diminishes. Adopted as policy at Harvard Kennedy School, used in executive education globally.

Why distinct from design-hiring-process: design-hiring-process designs structural mechanisms — rubrics, scorecards, interview panels, candidate tracking. apply-merit-selection is behavioral discipline at the evaluation moment — a specific mental test the evaluator applies regardless of whether a formal process exists. The two are complementary: structured processes reduce the opportunity for bias; this skill provides the mental discipline to apply even when no structure exists or when structure is being developed. design-hiring-process explains what infrastructure to build; apply-merit-selection explains what cognitive test to run when you're actually making the call.

Adopted by: Google, Deloitte, and KPMG adopted blind evaluation processes following the Rouse & Goldin research; the NFL Rooney Rule's diverse-slate requirement has been adopted in modified forms by Amazon, Cigna, and Microsoft; Iris Bohnet's structured comparative evaluation is taught and applied at Harvard Kennedy School and used in executive selection globally.

Impact: Blind symphony auditions increased the probability of women advancing past preliminary rounds by approximately 50%; the NFL saw significant increases in minority head coach representation after the Rooney Rule was introduced — without changing the talent pool, only the evaluation process.

Steps

1. Before making any selection decision, name your relationship to each candidate. Explicitly articulate how you know each person under evaluation. Categories: enemy or adversary, neutral/unknown, acquaintance, friend, family, former colleague, mentee, sponsor. This step is required because the reversed-relationship test cannot be applied if the relationship is not explicitly named.

2. Apply the reversed-relationship test. For each candidate, ask: if this person were at the opposite end of my relationship spectrum, would I still make the same recommendation? Specifically:

   • For someone you know and like: "If this person were my enemy — someone who had harmed my interests — would I still consider them the most qualified candidate?"
   • For someone you don't know or dislike: "If this person were my close ally — someone whose success I care about — would I rate them differently than I do now?"

   If the answer to either question is yes — if the recommendation would change if the relationship changed — do not change the decision yet. Instead, proceed to Step 3.

3. Re-evaluate on explicit, pre-specified criteria. Where the reversed-relationship test reveals potential bias, evaluate the candidate exclusively on explicit criteria that were specified before reviewing applications:

   • What specific demonstrated capabilities or results does this role require?
   • What evidence does each candidate have of those specific capabilities?
   • Rank candidates by evidence, not by impression.

   Criteria specified after reviewing applications are susceptible to post-hoc rationalization — you identify criteria the preferred candidate meets, then claim to be evaluating on criteria. Specify criteria first, apply them second.

4. Actively seek information on candidates outside your relationship network. One of the most common failure modes of the reversed-relationship problem is the evaluation pool itself: candidates are drawn from familiar networks, so candidates who would be meritocratically superior simply are not in the pool. Before evaluating, ask: who has the required capabilities that I have not yet identified? Extend the consideration pool to candidates from outside your immediate network before evaluating within it.

5. Evaluate comparatively, not individually. Iris Bohnet's research establishes that evaluating candidates against each other — ranking on explicit criteria simultaneously — reduces individual bias compared to evaluating each candidate alone and reaching an independent judgment. When possible, evaluate all candidates for a role against the same criteria in the same sitting rather than sequentially over time.

6. Document the evaluation and its reasoning. The discipline of writing down the specific evidence that supports a recommendation exposes post-hoc rationalization that would otherwise remain invisible. If you cannot write a specific evidence-based justification — "this candidate was selected because of X demonstrated capability evidenced by Y result" — the selection was not meritocratic. The documentation also provides accountability: future outcomes can be compared against predictions.

Rules

• The reversed-relationship test is applied to expose bias, not to reverse bias. The goal is not to disadvantage people you like or advantage people you don't — it is to identify when relationship is doing work that merit should be doing, then remove that influence.
• Specify evaluation criteria before reviewing candidates. Criteria derived after reviewing the application pool are structurally biased toward candidates already favored by other means. Criteria specified in advance provide a meritocratic standard.
• Awareness of bias does not reduce it. Bohnet's research is explicit: knowing you might be biased and intending to be fair does not produce measurably fairer outcomes in individual evaluation decisions. Structural interventions (blind review, standardized criteria, comparative evaluation) produce measurable improvement; good intentions do not.
• The evaluation pool is part of the evaluation. A meritocratic evaluation process applied to a non-representative consideration pool is not meritocratic. The reversed-relationship test must be applied not just to who gets selected from the pool, but to who gets into the pool.

Examples

Promotion decision: A manager must recommend one of three direct reports for a senior role. One is a close friend; one is a colleague with whom she has no strong personal relationship; one is someone she finds difficult to work with. She applies the reversed-relationship test: "If my friend were someone I found difficult and my difficult colleague were my friend, would my ranking change?" She realizes her ranking would reverse on two of the three dimensions she cares about. She returns to specified criteria — specific results delivered, demonstrated leadership behaviors, stakeholder feedback — and builds the ranking from evidence. The outcome changes.

Search committee: A university search committee is filling an endowed chair. The committee has three finalists. The committee chair knows one finalist personally; two are unknown. The committee chair applies the reversed-relationship test: "If the finalist I know were someone I'd never met, would I still be the most enthusiastic advocate for them in this room?" The answer is uncertain. The committee chair proposes that all three finalists be scored on a predetermined rubric before discussion, to surface any evidence-based differential before relationships influence the discussion. This surfaces a qualification gap in the personally-known candidate that was not visible before the structured scoring.

Reference check: An executive is asked for a reference for a former colleague who is now applying for a competitor's CEO role. The executive does not like the colleague personally. Reversed-relationship test: "If this person were my closest ally, what would I say about their capabilities?" The executive realizes their instinct was to damn with faint praise on dimensions where the colleague is genuinely strong, because of personal history. The reference is recalibrated to evidence.

Common Mistakes

• Mistaking the appearance of process for the substance of merit. A structured interview process administered by evaluators who have already decided who they want produces post-hoc rationalized justifications, not meritocratic decisions. Process is necessary but not sufficient; the reversed-relationship test is applied inside the process.
• Applying the test only to external hires. Promotion and recommendation decisions within an organization are more susceptible to relationship bias than external hires, because relationships are longer-established and the evaluation is typically less structured. The reversed-relationship test is most critical in internal talent decisions.
• Confusing familiarity advantage with meritocratic advantage. Candidates who are well-known to evaluators often appear more qualified because evaluators have more evidence about them, not because they are objectively more qualified. This familiarity advantage is a form of relationship bias: candidates who are unknown to evaluators are systematically underrated because evaluators have less evidence, not because they have less capability. Recognize familiarity as a potential source of inflation rather than a signal of superior qualification.