How to screen a large pile of resumes without missing good candidates

To screen a large pile of resumes well, decide what you are looking for before you open the first one, read every application against that same short list, sort into three groups rather than two, and then check a sample of the ones you turned down. That last step is missing from most screening advice, and it is the only way to find out whether your screen is working.

The rest of this guide shows how to do each step, using one fictional hire as a worked example. The whole method is also on a one-page screening checklist (PDF), free, no sign-up.

Why the usual method goes wrong

Most first-pass screening looks like this: open the folder, start at the top, make a quick yes or no on each resume, and stop when the yes pile looks big enough or the afternoon runs out.

Three things go wrong with it.

• The criteria shift as you read. Without a written list, the first strong resume quietly becomes the standard everyone after it is measured against. Applicant 40 is judged against a different bar than applicant 4.
• Order decides too much. If you stop when you are tired or when the yes pile is full, everyone below that point was screened by the calendar, not by you.
• “Not mentioned” gets treated as “no”. A resume that says nothing about a requirement is not evidence the person lacks it. A quick yes/no screen cannot tell the difference, so it rejects both.

None of these show up as errors. The people you screened out never reappear, so you never learn which of them you should have interviewed.

The worked example

Larkspur Partners, a fictional accounting firm, is hiring a Senior Accountant. The posting brought in 180 applications in two weeks: some through its careers page, some forwarded from a job board, a few by email. The managing partner has to get from 180 to about 8 people worth a phone call.

Step 1: Write three to five requirements as evidence

Before reading anything, write down what the job needs, and for each item, what you would expect to see in an application from someone who has it.

The discipline is to describe qualifications the person must already have, not duties they will perform. “Will review junior staff’s work” is a duty. It belongs in the job description, not the screen, or everyone who has not held that exact title fails it.

For Larkspur:

Requirement	Type	What counts as evidence
CPA designation	Must have	Designation listed, or “CPA candidate” with an expected completion date
Runs month-end close independently	Must have	Describes owning or leading a close, not only contributing to one
Prepares audit working papers	Nice to have	Mentions preparing files or schedules for an external audit
Experience with cloud accounting	Nice to have	Names a cloud ledger, or describes migrating to one

Notice what is not on the list: “five or more years of experience”. Across the research on hiring methods, the number of years someone has worked is one of the weakest predictors of how well they will do a job. A 2022 re-analysis of selection research put it at .07, against .42 for a structured interview. The years were only ever a proxy for “has done this work independently”, so the second requirement asks for that directly.

Keep the must-haves few. Every must-have is a gate that can remove someone, and five gates remove more people than you intend. Must-have vs nice-to-have has a four-question test for deciding which requirements earn a place.

Step 2: Decide your knockouts, and keep them honest

A knockout is a requirement that ends the application on its own: a licence the law requires, the right to work in the country, availability for a shift that cannot move. These are worth asking as a direct screening question on the application form (“Do you hold a CPA designation?”) rather than inferring from a resume.

Use them only for genuine yes-or-no requirements. “Answered the question in the subject line” or “used our company name in the cover letter” filters for attention to an arbitrary instruction, not for ability to do the job.

Step 3: Read every application against the same list

Now read. For each application, mark each requirement as one of three things:

• Shown: the application gives evidence for it.
• Unclear: the application does not say either way.
• Not met: the application gives evidence against it, such as a designation from a different profession where a CPA is required.

Write a few words of evidence as you go: the line or role that convinced you. It takes seconds per requirement and it is what lets you, or anyone else, check your call later.

Then sort by a rule you decided in advance, not by overall impression. For Larkspur:

• Recommended: both must-haves shown.
• Possible: no must-have marked not met, but at least one unclear.
• Requirements not met: a must-have not met, or a knockout failed.

The nice-to-haves order people within a group; they do not move anyone between groups.

Sorting by rule rather than by feel is not just tidier. A meta-analysis of selection and admissions decisions by Kuncel and colleagues (2013) found that combining information by a consistent rule predicted job performance substantially better than holistic judgement, even when the people judging were experts in the jobs concerned. You will still use judgement. Use it on each requirement, then let the rule combine them.

Step 4: Give “Possible” a real look

The Possible group is where good people are easiest to lose, because an unclear resume is easy to drop when the Recommended group already looks full.

Read the Possible group with one question: what would I need to know to move this person up or down? Often the answer is a single fact a phone call or a one-line email would settle: “Your resume mentions the year-end close at your current firm. Did you run it, or support it?” If a polished application seems to claim more than the person did, see how to screen AI-written applications for a proportionate way to check.

At Larkspur, 180 applications might sort into something like 20 Recommended, 30 Possible and 130 Requirements not met. Those numbers are illustrative, not data from a real search; the split depends entirely on the role and where it was advertised.

Step 5: Audit the pile you turned down

This is the step that turns a screen you hope works into one you know works.

Pick at random about one in ten of the applications you marked Requirements not met, and at least ten in total. Read each one again, slowly, against the same list. You are looking for one thing: someone you would now put in Recommended or Possible.

• If you find none, your screen is probably doing what you intended.
• If you find one or two, look at why. A common cause is a requirement worded so that an ordinary way of describing the experience did not count as evidence. Fix the wording and re-check the rest of that pile for the same pattern.
• If you find several, the screen is not working. Stop and rewrite the requirements before going further.

Reading a sample of 13 takes perhaps 20 minutes. It is cheap insurance against the one error a screen never reports on its own.

Step 6: Budget the time before you start

Screening expands to fill whatever time is left, which is how resumes end up read at the end of a long day. Set the budget first.

Using an assumed careful first read of two to four minutes an application, which is enough to check four requirements and note evidence:

Applications	At 2 minutes each	At 4 minutes each
50	1 hour 40 minutes	3 hours 20 minutes
100	3 hours 20 minutes	6 hours 40 minutes
180	6 hours	12 hours

These are assumptions for planning, not measurements. Your own pace will depend on the role and the resumes. What the arithmetic shows is that a pile of 180 is one to two working days of careful reading, which is why it so rarely gets read carefully. Split it across sessions of an hour or so, and work through the applications in a different order from the one they arrived in, so that the same people are not always read when you are freshest or most tired.

What to do next

By the end you should have a Recommended group, a resolved Possible group, a checked rejection pile, and a few words of evidence against every name. That is a shortlist you can defend, and the evidence notes are the starting point for the next stage: they become the first rows of a candidate scorecard. A structured interview, with the same questions for every finalist, is the strongest single step you can take after this one; see what the research says about which methods predict performance, and how to build and run a structured interview for the same Senior Accountant hire.

Doing the first read with software

The method above works with a spreadsheet and an afternoon you have protected. It is also exactly the method SortTheStack follows. It drafts the requirements from your job description for you to edit, reads every application against them, quotes the evidence it relied on, marks what it could not find as unclear rather than missing, and sorts applicants into the same groups used here: Recommended, Possible and Requirements not met, plus Needs review for an application with an inconsistency worth checking, such as overlapping full-time jobs. You still make every decision, and the rejected pile is still worth auditing. See how the screening works, or read more about what AI resume screening should and should not do.

If you advertise jobs publicly in Ontario and employ 25 or more people, and you use AI to screen, assess or select applicants, the province requires you to say so in the posting. See the Ontario government’s guide to job posting requirements. Rules elsewhere differ.