ai-extending-professional-standards

Extending professional standards (Call to Action)

The article's thesis and orchestration point: AI/technologically enhanced selection should be held to the same established professional standards as any other employment test, and I-O psychologists should lead in working out how. Use this skill to orient a project — or a professional-policy discussion — to the governing documents and the collaborative path forward. It ties the whole collection together.

The two guiding documents

Two documents guide research and practice in employee selection regardless of the form of assessment:

• Principles for the Validation and Use of Personnel Selection Procedures (SIOP, 2018) — see the personnel-selection collection.
• Standards for Educational and Psychological Testing (AERA, APA, NCME, 2014).

Both adopt the same definition of validity — the degree to which accumulated evidence and theory support specific interpretations of test scores for proposed uses — which is exactly why a technology is never "universally valid" (ai-selection-tech-data-algorithms) and why validity/reliability/fairness evidence is required for AI tools.

Why I-O psychologists are well-equipped — but not sufficient alone

I-O psychologists bring deep grounding in the factors critical for employment testing: psychological constructs (knowledge, personality, interests, engagement, teamwork, safety, performance, turnover), theories of testing and assessment (construct-oriented test development, psychometric modeling, appropriate scoring/interpretation), the types of evidence that support inferences (selection decisions, validity), psychometric properties (internal consistency, test–retest, alternate-forms reliability), and the evaluation of subgroup differences (differential prediction, measurement invariance, adverse impact). SIOP also has a long history of documenting consensus in the Principles.

But this knowledge must be supplemented by others in the field: data scientists and software developers (acquire/store/analyze data, build and evaluate algorithms), web designers and IT professionals (build engaging, effective interfaces), and the legal profession (compliance with federal/state/local law and regulatory requirements). I-O psychologists cannot regulate others' practice, but many serve as experts advising organizations and government and testifying about assessments — supporting and challenging them.

The Call to Action

• Develop interpretive guidance — don't rewrite the Principles. The recommendation is for SIOP to develop interpretive guidance that applies the Principles to technologically enhanced assessments, guiding developers and users in best practices and addressing the open questions the paper raises. The Principles already reflect the established science of selection; the goal is interpretation and consistency, not replacement.
• Collaborate across disciplines. Engage applied statistics, computer science, and other fields to learn about ML applications. Together, identify the strengths, critique the weaknesses, and understand appropriate vs. inappropriate applications. Interpretive guidance should help fill knowledge gaps among the participating parties.
• Engage proactively — not only selection specialists, but also those in recruiting, diversity and inclusion, and leadership — because doing so can improve assessment and promote the future relevance of the profession.

The guardrail: no "escape velocity"

The overarching responsibility: ensure that progress does not approach escape velocity from its moorings in scientific, psychometric, and practical knowledge; understanding of legal guidelines and professional/ethical obligations; and the many hard lessons learned in the employment-testing arena. New tools offer real advantages for employers and applicants — and we are responsible for keeping them anchored. Now is the time to consider how the Principles should be applied to new and evolving forms of assessment to reflect the research literature and best practices.

How to use this skill

• Orient any AI-selection project to the Principles and Standards as the benchmark, then route to the specific concern skills for the evaluation.
• Frame professional/policy discussions around applying (not rewriting) the Principles and building interdisciplinary collaboration.
• Audit your team composition: do you have psychometric, data-science, IT/UX, and legal expertise at the table?
• Apply the escape-velocity test: is any practice drifting away from scientific, legal, or ethical moorings?

Pitfalls

• Proposing to rewrite the Principles for AI rather than developing interpretive guidance that applies them — the established science of selection still holds.
• Treating a technology as "validated" rather than validating the inferences about constructs measured in a specific use.
• Assembling a team with psychometric expertise but no data-science, IT/UX, or legal voices (or vice versa) — the paper stresses no discipline suffices alone.
• Inventing ad hoc, tool-specific rules disconnected from the Principles and Standards.
• Letting innovation outrun (reach "escape velocity" from) scientific, legal, and ethical moorings in the name of efficiency.

Checklist

• Project benchmarked against the Principles and Standards
• Validity framed as evidence + theory for a specific inference (not a property of the technology)
• Interdisciplinary expertise assembled (psychometrics + data science + IT/UX + legal)
• Stance taken: interpret/apply the Principles to the tool, not invent ad hoc rules
• Open questions from the 11 concerns logged for the developer/vendor
• "Escape velocity" check applied to scientific, legal, and ethical moorings

See also

All skills in this collection (this is the orchestration point) · personnel-selection (the Principles operationalized) · ai-personnel-assessment (the audit framework) · validation-planning · technical-validation-report

Source: Tippins, Oswald & McPhail (2021), "Standards" and "A Call to Action," and the Conclusion.