ai-applicant-reactions-and-communications

AI applicant reactions & communications (Concerns 9–10)

Two linked concerns about information flow around an AI selection tool: how applicants experience and react to it, and what the organization tells candidates and other stakeholders.

Applicant experience and reactions (Concern 9)

Employers want selection that is simple, quick, and engaging to attract qualified candidates, and technologically enhanced assessments are often highly engaging with little applicant effort. But innovative methods raise reaction concerns.

Pitfalls in reaction metrics

Many vendors collect applicant-reaction data, but the metrics are weak:

1. They rarely incorporate the full range of organizational considerations — vendors ask if the experience was engaging, but seldom whether applicants felt job-relevant KSAOs were measured.
2. Good comparative data are rarely available — reactions are used as a marketing tool, seldom compared to reactions to other tools.
3. It's hard to know the local range of reactions (how positive/negative) even with meta-analytic work (Hausknecht et al., 2004).
4. Reactions are affected by how well candidates think they performed — harder to gauge for novel games or tools with no obviously correct answers.
5. Whether a job offer followed is a huge driver of reactions — so consider measuring reactions before offers are extended.
6. Simply asking about the experience afterward may alter perceptions (e.g., prompting reflection on fairness/invasiveness of a video interview or scraped data).

Impact on well-qualified candidates and the faking question

The applicant reaction–behavior link is tenuous — the "Achilles heel" of applicant-reactions research (Sackett & Lievens, 2008). Still, organizations worry about effects on the quality and quantity of applicants they attract. It's unclear how candidates react on learning their selection hinged on an unknown weighted combination of facial expressions, voice quality, mouse clicks, and other data — versus, say, their MBA from a top school. Reactions may matter more now because applicants amplify them via social media (Twitter, Facebook, LinkedIn). A largely unresolved issue: whether training for a video interview is possible, and if so, whether it produces invalid variance (faking/lying) or valid variance (by ensuring candidates understand what's expected). Some organizations sidestep specifics by simply informing candidates whether the outcome indicates they met the employer's needs.

Questions to ask (Concern 9)

• How should applicants prepare for the assessment (are practice sessions allowable)?
• What is being measured, is it relevant to the job, and does the applicant know it's being measured?
• Why was an applicant not selected — can big data and the ML algorithm provide explanations?
• How can applicants improve to become more qualified upon retesting?
• How do applicants evaluate organizations that minimize personal interaction (chatbots, avatars)?
• What are applicant reactions to innovative approaches, and how do they affect the employer's ability to attract qualified candidates and its reputation?

Communications (Concern 10)

Managers (whose success depends on a competent workforce) care that job-critical skills are being measured; labor organizations and advocacy groups care about job relevance and fairness; and enforcement officials have a statutory/regulatory interest in what is measured and how.

Sharing selection-procedure information

A key question is what to tell people about how others were selected. There have always been limits:

• Organizations are unwilling to share anything that jeopardizes test use (item content, scoring keys) or increases legal/administrative challenge (e.g., adverse-impact data).
• The technical aspects of evaluating measurement or predictive bias are beyond the comprehension of most applicants and hiring managers (regression slopes, factor loadings), and bias in ML models is even harder to explain.
• Most applicants want to know at a basic level: what KSAOs are measured, how they're evaluated, and — if unsuccessful — what they can do to improve next time.
• Test-prep materials typically describe the process, give tips, and sometimes offer practice questions; but it's unclear what preparation can be offered when selection rests on face or voice characteristics.

Questions to ask (Concern 10)

• What information can and should be provided to unsuccessful applicants?
• What aspects of a selection procedure should an organization share with a range of stakeholders (manager, industry, clients, customers, shareholders)?

Pitfalls

• Treating "engaging" reaction data as evidence of fairness or job-relevance perceptions.
• Using vendor reaction stats (marketing) without comparative or local data.
• Ignoring that AI decisions are hard to explain — leaving rejected candidates with no actionable feedback.
• Encouraging "training" for video interviews without resolving whether it introduces faking.
• Over-sharing (compromising test security) or under-sharing (eroding trust/justice).

Checklist

• Reaction measurement captures job-relevance perceptions, not just engagement
• Reaction data benchmarked (comparative/local), not taken as marketing
• Offer-status and timing effects on reactions accounted for
• Faking-vs-training question for any video/behavioral interview addressed
• Basic, honest information (what's measured, how, how to improve) prepared for candidates
• Security-sensitive details (keys, items, adverse-impact data) protected
• Stakeholder communication plan defined (managers, regulators, advocacy groups)

See also

ai-candidate-data-control · ai-selection-ethics · ai-selection-legal-landscape · ai-claims-and-stakeholder-audit (second-party effects, justice) · administration-documentation (candidate communications, feedback)

Source: Tippins, Oswald & McPhail (2021), Concerns: "Applicant Experience and Reactions" and "Communications."