What actually predicts job performance? What the 2022 re-analysis changed

The short answer: a well-run structured interview, a test of the specific knowledge the job needs, and a realistic sample of the work are the strongest predictors of job performance we have. Years of experience is one of the weakest. And every method predicts less than the figures you have probably seen quoted.

That ranking comes from a 2022 re-analysis by Paul Sackett, Charlene Zhang, Christopher Berry and Filip Lievens in the Journal of Applied Psychology. It revisited the meta-analyses behind one of the most widely cited summaries in hiring research and concluded that the old numbers had been systematically inflated.

This article explains what changed, what the numbers do and do not mean, and how a team without an HR department can use them.

The numbers most people quote are from 1998

If you have read about evidence-based hiring, you have probably met Schmidt and Hunter (1998). Their summary of 85 years of research put work sample tests, cognitive ability tests and structured interviews at the top, each with an estimated validity above .50. It has shaped hiring advice for a quarter of a century.

Sackett and colleagues (2022) looked at how those estimates were produced. Raw correlations between a selection method and later job performance are usually corrected upward, partly because studies only measure the people who were hired, and that narrower group understates how well the method separates the whole applicant pool. The correction is legitimate. The authors’ argument is that the way it was usually applied overcorrected, often by a lot, because it assumed far more restriction than most studies actually had.

Re-doing the corrections, they found that most methods kept their rank but lost .10 to .20 of their estimated validity. In their words, selection procedures “remain useful, but selection predictor–criterion relationships are considerably lower than previously thought.”

The revised ranking

These are the paper’s estimates for the methods a small business is most likely to use, from its Table 3. “Validity” here is the estimated correlation between the method and later job performance.

Method	1998 estimate	2022 estimate	Practical for a small team?
Structured interview	.51	.42	Yes. Costs preparation time, not money.
Job knowledge test	.48	.40	Yes, where the job has a body of knowledge.
Empirically keyed biodata	.35	.38	Rarely. Needs data from many past hires to build.
Work sample test	.54	.33	Yes, if it is short and resembles the real work.
Cognitive ability test	.51	.31	Possible, with care. See subgroup differences.
Integrity test	.41	.31	Usually bought from a test publisher.
Unstructured interview	.38	.19	The default, and only half as predictive.
Conscientiousness (general)	.31	.19	Weak on its own.
Job experience (years)	.18	.07	Barely predictive.

Two methods often used in small-business hiring do not appear in the revised list at all. Reference checks and years of education were excluded because there was not enough information to re-estimate them, so the paper offers no updated figure for either.

Three things the table changes

1. The interview you run matters more than the fact that you interview

The gap between structured and unstructured interviews is the most useful number in the paper for a small business: .42 against .19. A structured interview asks every candidate the same job-related questions and rates the answers against anchors decided in advance. An unstructured interview is a conversation that goes wherever it goes.

Both can fit in the same hour. On these estimates, one is more than twice as predictive as the other.

The structured interview’s estimate also varies a lot from study to study (a standard deviation of .19, with a lower 80% credibility value of .18). That spread is the warning: “structured” covers a range of practice, and a loosely structured interview will not get you the average result.

2. Years of experience is a weak screen

A validity of .07 means that, across the studies reviewed, knowing how many years someone had worked told you very little about how well they would do the job. An earlier meta-analysis dedicated to the question, Van Iddekinge and colleagues (2019), reached a similar conclusion: pre-hire work experience correlated about .06 with job performance across 44 samples, and not at all with turnover.

That does not make experience irrelevant. It means the count of years is a poor stand-in for what the years were supposed to produce. “Five or more years of accounting experience” is easy to check and weakly predictive. “Has run a month-end close independently” is harder to check and much closer to the thing you need.

This is the pattern behind most weak hiring criteria. They are chosen because they are easy to evaluate, not because they predict much. A year count, a degree title and a recognisable employer name can all be read in seconds. The evidence that someone can do the work takes longer to find.

3. Everything predicts less than you think

A correlation of .42 is strong by the standards of this research and modest by the standards of everyday intuition. Candidates who do better in a structured interview tend to perform better, with many exceptions in both directions.

The practical consequence is that no single method should decide a hire. Combining two or three good ones beats leaning on any one, and the strongest single predictor still leaves plenty of room for surprises.

What the numbers do not tell you

The paper is careful about its limits, and they matter for how you use it.

• These are averages across many jobs and studies. A method’s validity in your firm, for your role, can be higher or lower.
• “Job performance” is largely supervisor ratings, mixed with some objective measures. Ratings are themselves an imperfect measure of how good someone is at a job.
• Many of the underlying studies tested current employees, not applicants. The work sample estimate rests on 54 studies, 53 of them with incumbents, and the job knowledge estimate comes entirely from such samples because no applicant meta-analysis exists. Applicants are a more varied group, and the correction for that difference is exactly what the paper argues was overdone.
• It ranks methods, not resumes. Nothing in the table measures how well someone reads a resume. Resume screening sits upstream of all of it: it decides who gets the chance to be evaluated by the stronger methods at all.

Validity is not the only consideration

The paper pairs each method with its average Black–White subgroup difference, because a method can predict well and still screen out one group of applicants far more than another. Cognitive ability tests show a large difference (d = .79 in the paper’s table); structured interviews a much smaller one (d = .23). The paper notes that some of these subgroup figures come from samples other than job applicants, so treat them as approximate.

The authors call this the validity–diversity tradeoff. For a small employer the practical reading is simple: the structured interview combines the highest validity in the table with a comparatively small subgroup difference, which is one more reason to start there. Whether a particular test creates legal risk depends on where you operate and how you use it, which is a question for an employment lawyer, not a research table.

A hiring process built on the evidence

Here is one way to turn the ranking into a process a small team can run. It is our interpretation of the research, not a procedure the paper prescribes.

1. Write the requirements as evidence, not proxies. For each thing the job needs, write what you would expect to see from someone who has it. “Has prepared year-end working papers for an external audit” rather than “3+ years audit experience”.
2. Screen resumes for that evidence. Read for whether the application shows the thing, says nothing about it, or shows the opposite. Treat “says nothing” as unknown rather than as a no. Our guide to screening a large pile of resumes walks through this step in detail.
3. Add one short work sample or knowledge check for the shortlist, built from the real job. Keep it short enough that a strong candidate with other options will still do it.
4. Run a structured interview. Same questions, in the same order, for every finalist. Write down what a strong, acceptable and weak answer looks like before the first interview. Our guide to running a structured interview has a worked example.
5. Rate each piece separately, then combine by a rule you set in advance. A meta-analysis by Kuncel and colleagues (2013) found that combining information mechanically, by a formula or fixed rule, predicted job performance substantially better than holistic judgement, by more than 50%, even when the judges were experts in the jobs concerned. Discuss the result afterwards; just do not let the discussion replace the ratings. A free candidate scorecard does this on one page.

None of this needs software. It needs the requirements written down before anyone reads an application, and the discipline to evaluate everyone against them.

Where SortTheStack fits

SortTheStack covers step 2. It drafts requirements from your job description for you to review, then reads every application against them and quotes the evidence it found, marks what it could not find as unclear rather than missing, and leaves the decision to you. It does not run interviews or tests, and it does not predict job performance. If step 2 is where your time goes, you can see how the screening works.

Sources

• Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. doi.org/10.1037/apl0000994. Figures in this article are from its Table 3.
• Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology. Psychological Bulletin, 124(2), 262–274. doi.org/10.1037/0033-2909.124.2.262
• Van Iddekinge, C. H., Arnold, J. D., Frieder, R. E., & Roth, P. L. (2019). A meta-analysis of the criterion-related validity of prehire work experience. Personnel Psychology, 72(4), 571–598. doi.org/10.1111/peps.12335
• Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clinical data combination in selection and admissions decisions: A meta-analysis. Journal of Applied Psychology, 98(6), 1060–1072. doi.org/10.1037/a0034156