AI interviewers: what the evidence shows, and what to check before using one

An AI interviewer is software that conducts a first-round interview, by voice or video, asking questions and recording the answers. The strongest evidence so far comes from a large field experiment with entry-level customer service hiring, where AI-led interviews, evaluated by human recruiters, led to more job offers, more people starting and more of them staying. The benefit came from consistency: the AI asked the questions the same way every time. That is encouraging, but it’s one setting, and it tells you nothing about tools that score or reject people on their own.

Three different things called “AI interviews”

Products sold under this label do very different jobs, and the risks differ accordingly.

What the AI does	Example	Main risk
Asks the questions	A voice agent runs a structured phone screen; a person reviews the recording or transcript	Applicant experience; technical failures
Scores the answers	Software rates each answer or the whole interview	Scores you can’t explain; bias in what it rewards
Decides who advances	Applicants below a score are automatically rejected	Every error or bias becomes a decision nobody reviewed

The evidence below is about the first kind. Be careful not to read it as support for the other two.

What the field experiment found

Jabarian and Henkel (2026) ran a natural field experiment with a recruitment firm hiring for entry-level customer service jobs in the Philippines, involving 70,884 applications. Applicants who passed an initial screening were randomly assigned to be interviewed by a human recruiter or by an AI voice agent working from the same interview guidelines. A third group could choose. In every case, human recruiters evaluated the interviews, together with a standardized test, and made the hiring decisions.

What they found:

• More offers: 9.73% of applicants interviewed by the AI received an offer, against 8.70% of those interviewed by people, a 12% increase.
• More starts and longer stays: applicants in the AI group were 18% more likely to start the job and 18% more likely to still be employed after a month, with similar gains at two, three and four months.
• No drop in productivity on the measures available for a subset of hires: time spent per customer, customer satisfaction and quality assurance scores.
• Why: analysing the transcripts, the authors found the AI followed the interview guidelines more closely and covered topics more consistently than human recruiters, while still asking follow-up questions, and so collected more information relevant to the hiring decision.
• Applicant reactions: applicants rated AI and human interviews similarly on stress, comfort and the quality of follow-up questions, but rated the AI interviews as significantly less natural. Given a choice, 78% picked the AI, though those who did had lower average test scores than those who chose a person.

What it doesn’t tell you

• It’s one setting: high-volume, entry-level customer service, with a standardized test alongside the interview, in one country. A small firm hiring one senior accountant is a very different situation.
• The AI collected information; people judged it. The study says nothing about software that scores or rejects applicants itself.
• It’s a working paper. It hasn’t yet been through peer review.

The most useful lesson may not be about AI at all. The gain came from consistency: the same topics, in the same order, asked the same way. That is the definition of a structured interview, and research on human interviewers has long found structure makes interviews far more predictive. If you hire a handful of people a year, you can get most of that benefit by structuring your own interviews.

The rules that apply

AI in interviews is one of the more regulated parts of hiring. As of 12 October 2026, for example:

• Illinois: an employer that uses AI to analyse recorded video interviews must, before the interview, tell the applicant AI may be used, explain how it works and what it evaluates, and get their consent. It may share the videos only with people needed to evaluate the applicant, must delete them within 30 days if the applicant asks, and, if it relies solely on the AI to decide who gets an in-person interview, must report demographic data to the state each year (Artificial Intelligence Video Interview Act).
• New York City: an automated employment decision tool needs a bias audit within the year before use, a public summary of the results, and notice to candidates (NYC DCWP).
• Ontario: employers with 25 or more employees must say in a public job posting if they use AI to screen, assess or select applicants (Ontario Ministry of Labour).

Our article on what job postings must disclose covers these and others. This is not legal advice; check what applies where you hire.

Questions to ask before using one

Run these against any AI interview product, and get the answers in writing:

1. Which of the three things does it do? Ask, collect, score or decide? Can you switch scoring and automatic rejection off?
2. Who writes the questions? You should be able to set them from your own requirements, the same for every applicant.
3. What does a reviewer see? A recording and a transcript you can check, or only a score?
4. If it scores, on what? The content of answers, or voice, facial expression or speaking style? Ask for evidence that whatever it measures relates to job performance, and for its bias testing.
5. What happens when it fails? Dropped calls, poor connections, accents it transcribes badly, applicants who need an accommodation. Is there a human alternative?
6. How long are recordings kept, who can see them, and can applicants ask for them to be deleted?
7. What does it tell applicants, and when? Will it support the notices and consent the law requires where you hire?

If your concern is applicants using AI rather than you using it, see AI-written applications.

Then pilot it the way our article on testing AI screening tools describes: on a role you know well, comparing its output with your own judgement before relying on it.

Where SortTheStack fits

SortTheStack doesn’t conduct or score interviews. It screens written applications against your requirements, with the evidence quoted, and when you invite someone to interview it sends them your own scheduling link. The interview itself, and the decision, stay with you. See how the screening works.

Sources

• Jabarian, B., & Henkel, L. (2026). Voice AI in firms: A natural field experiment on automated job interviews. arXiv:2607.28222 (working paper, v2, 11 September 2026). arxiv.org/abs/2607.28222
• Artificial Intelligence Video Interview Act, 820 ILCS 42, ilga.gov, accessed 12 October 2026.
• NYC Department of Consumer and Worker Protection, Automated employment decision tools, accessed 12 October 2026.
• Ontario Ministry of Labour, Immigration, Training and Skills Development, Requirements related to publicly advertised job postings, accessed 12 October 2026.