What to automate in hiring, and what to keep human

Automate the steps where a fixed rule gives the right answer: messages, scheduling, applying your own yes/no knockout questions, and combining interview ratings by a formula. Let software assist, with a person checking, where the work is reading applications. Keep people on the conversations and on every decision that turns someone away or chooses a hire. One step surprises most teams: combining ratings is better done by a rule than by discussion, and that is well supported by research.

This guide gives a test you can apply to any step, a stage-by-stage table, and what it takes for “a human reviews it” to mean something.

A three-question test

For any step in your hiring process, ask:

1. Can you write down the rule? If the right answer follows from a rule you could state in a sentence (“send a confirmation when an application arrives”, “exclude applicants who answer No to the licence question”), a machine applies it more reliably than a busy person. If the answer takes judgement about a particular person, it can’t be a rule.
2. What does a mistake cost, and who would notice? A late confirmation email costs a little goodwill, and the applicant notices. A wrongly rejected applicant costs you a candidate, and nobody notices, because rejected applications are rarely looked at again.
3. Can a person see the result and reverse it? Automation you can see and undo is a convenience. Automation that acts silently is a decision nobody made.

Those answers sort most steps into three groups: automate, assist (the software does the work, a person checks it) and keep human. This is our framework, built on the evidence below, not an industry standard.

Stage by stage

Stage	Recommendation	Why
Posting the job to boards	Automate	Pure administration
Confirming receipt; status updates	Automate	A rule, and silence loses candidates (see ghosting)
Scheduling interviews	Automate	Booking links remove the back-and-forth
Knockout questions on yes/no facts	Automate, rule reviewed	A rule you wrote; keep it to genuine requirements
Reading applications against requirements	Assist	Reading at volume is where software helps, but it misreads; check the evidence
Rejecting applicants	Keep human	Mistakes are invisible, and the rules increasingly expect a person
Phone screens and interviews	Keep human	Evidence on AI interviewers is early (see below)
Combining interview ratings	Automate (by formula)	A fixed rule predicts better than discussion
The hiring decision	Keep human, one named person	Someone has to own it and be able to explain it
References and the offer	Keep human	Conversations, negotiation and commitments

A few rows need more than a line.

Knockouts: automate the rule, review the rule

A knockout question (“Do you hold a valid Class 1 licence?”) is automation you designed. It is safe when the question is a genuine requirement with a yes/no answer, and it quietly costs you good people when it isn’t. Review each one before posting; our guide to screening questions covers which make fair knockouts.

Reading applications: assist, and check

Software is good at applying the same requirements to every application, the 140th as carefully as the first. It also misreads. Use a tool that shows the evidence for each verdict, read the applicants it marks unclear yourself, and spot-check the rest. Our article on why an AI tool ranked a candidate has a five-minute check, and can AI help you identify the best applicants? covers the wider risks.

Interviews: keep human for now

AI voice interviewers exist, and one large field experiment found they made interviews more consistent in entry-level customer-service hiring. That is one setting and a working paper. Our article on AI interviewers covers what it does and doesn’t show.

The step to automate that most teams keep human

Teams usually combine interview ratings by talking: everyone shares their impressions and the group settles on a view. The research says a fixed rule does better.

A meta-analysis by Kuncel and colleagues (2013) compared mechanical combination (a formula or fixed rule) with holistic judgement in selection and admissions decisions. It found “consistent and substantial loss of validity when data were combined holistically”, even when the people combining were experts who knew the jobs. In predicting job performance, the difference amounted to “an improvement in prediction of more than 50%”.

“Automate” here can mean a spreadsheet column that averages the ratings. The judgement stays where it belongs: in each interviewer’s ratings against the anchors, and in the discussion of what the formula can’t capture. Our guides to consistent interview scoring and making the final decision show how.

The step most often automated that should stay human

Rejection. Many tools can reject applicants below a threshold without anyone looking. That turns every error in the tool into a decision nobody reviewed, and the people affected are the ones you never see again. Our screening audit is the way to find out how often it happens.

The rules point the same way. They differ by place, change quickly and this is not legal advice, but as of 1 November 2026:

• European Union. The GDPR gives people the right not to be subject to a decision “based solely on automated processing” that significantly affects them, and gives “e-recruiting practices without any human intervention” as an example (Article 22 and Recital 71 of Regulation (EU) 2016/679).
• New York City. Local Law 144 covers tools that substantially assist or replace discretionary decisions, including screening. Using one requires a bias audit within the past year and notice to candidates (NYC DCWP).
• Ontario. Employers with 25 or more employees must say in a publicly advertised job posting if they use AI to screen, assess or select applicants (Ontario Ministry of Labour).
• United States generally. The EEOC’s position is that “the employer is still responsible for ensuring that its tests are valid”, whoever built them (EEOC).

Making human review real

“A person reviews every decision” means nothing if the person clicks Approve three hundred times in an hour. People tend to over-rely on automated advice, a pattern known as automation bias.

A systematic review by Goddard, Roudsari and Wyatt (2012) covered 74 studies, mostly in healthcare decision support. How much people over-relied depended on workload, task complexity and time pressure, and on how far they trusted the system. What helped included training, emphasising the user’s accountability, and providing information rather than a recommendation. Clinical decisions aren’t hiring decisions, but the conditions it describes, a busy person approving a machine’s verdicts at speed, are familiar.

In practice:

• Show the reviewer the evidence, not just the verdict. Checking a quote against an application is review. Agreeing with a label is not.
• Name who is accountable for each rejection and each hire. A decision with an owner gets looked at.
• Make overriding easy, and record why. If disagreeing with the tool takes five clicks and agreeing takes one, the tool decides.
• Check the agreements too. A reviewer who never overrides is either working with a perfect tool or not reviewing. Re-read a sample of what was approved.
• Budget the time. If review gets thirty seconds an application, decide in advance which applications get the full read (the unclear ones) and which get a spot-check.

Our guide to reviewing an AI shortlist turns these into a routine for one job, with a worked example.

Where to start this week

If you’re a small team doing most of this by hand, three changes cover most of the benefit:

1. Automate the confirmations. Every applicant hears that you received their application, the same day.
2. Use a booking link for phone screens and interviews.
3. Average your interview ratings in a spreadsheet before you discuss anyone.

Then decide what to do about reading. If you get more applications than you can read properly, that’s where software earns its keep, as long as it shows its evidence and doesn’t reject anyone on its own.

Where SortTheStack fits

Here is what SortTheStack does, sorted by the same three groups.

• Automates: a confirmation to applicants who apply through your SortTheStack careers page, and, when a job closes, a neutral notice to candidates still at the Interviewing stage that a decision has been made. Knockout rules you write are applied exactly as set, with no AI involved.
• Assists: it reads every application against your requirements and marks each requirement met, not met or unclear, with the evidence quoted. Every quote is checked against the resume before you see it. Setting aside clear mismatches automatically is off by default; when switched on for a job, it is logged and reversible.
• Leaves to you: no rejection is sent unless someone on your team chooses to send one, and the hiring decision is yours. When someone changes the product’s recommendation they add a reason, saved to the candidate’s decision record. You can send an interview invitation with your own scheduling link.

See how the screening works.

Sources

• Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clinical data combination in selection and admissions decisions: A meta-analysis. Journal of Applied Psychology, 98(6), 1060–1072. doi.org/10.1037/a0034156
• Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias: A systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 19(1), 121–127. doi.org/10.1136/amiajnl-2011-000089
• Regulation (EU) 2016/679 (General Data Protection Regulation), Article 22 and Recital 71, EUR-Lex, accessed 1 November 2026.
• NYC Department of Consumer and Worker Protection, Automated Employment Decision Tools: Frequently asked questions (29 June 2023), accessed 1 November 2026.
• Ontario Ministry of Labour, Requirements related to publicly advertised job postings, accessed 1 November 2026.
• U.S. Equal Employment Opportunity Commission, Employment tests and selection procedures, accessed 1 November 2026.