When Recruiters Meet Algorithms: Advice Taking and Bias in AI‐Assisted Recruitment

Classified as probably not a substantial claim. May combine two claims.

Some studies suggest that human expert advice has a stronger influence than algorithmic advice, but this assumption has not yet been thoroughly tested using designs that directly measure behavioral change after an initial decision.

“While some studies suggest that human expert advice has a stronger influence than algorithmic advice (Lacroux and Martin-Lacroux 2022;Lee 2018;Newman et al. 2020), this assumption has not yet been thoroughly tested using designs that directly measure behavioral change after an initial decision.”

• cites Lacroux and Martin-Lacroux 2022 sourceDisputed

  The citation is attached to the claim that human expert advice has a stronger influence than algorithmic advice. In Analytical Strategy and Results, the paper reports that recruiters were more influenced by algorithmic recommendations; the Discussion reiterates that they trusted human experts more but did not behave accordingly. The later statement about designs measuring change after an initial decision is not attributed to this citation. (LLM, conf=0.90)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites Lee 2018 sourceNot checked — no full text of the source
• cites Newman et al. 2020 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

The relative influence of, and trade-offs between, human and AI sources of advice in hybrid recruitment remain poorly understood.

“As Baines et al. (2024) note, the relative influence of, and trade-offs between, human and AI sources of advice in hybrid recruitment remain poorly understood.”

• cites Baines et al. 2024 sourcePartly supported

  In Discussion > Future research directions (page unavailable), the review says research has not sufficiently examined how agreement between human and AI advisors affects advice use. Practical implications (page unavailable) discusses AI in recruitment, but does not establish that the relative influence of, and trade-offs between, human and AI advice specifically in hybrid recruitment remain poorly understood. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

May combine two claims.

Existing studies often rely on vignette-based designs, samples of students or laypeople rather than professional recruiters, or indirect indicators of influence such as confidence ratings and ranking scores.

“Existing studies often rely on vignette-based designs (Oberst et al. 2020), samples of students (Cecil et al. 2024) or laypeople (Neumann and Kuncel 2025) rather than professional recruiters, or indirect indicators of influence such as confidence ratings and ranking scores.”

• cites Oberst et al. 2020 sourceNot checked — no full text of the source
• cites Cecil et al. 2024 sourcePartly supported

  The Experiment 1a Method identifies a sample of 370 university students, supporting the part attributed to Cecil et al. But the Experiment 1b Method reports a replication with HRM employees, 54.5% of whom worked in selection or recruiting. The paper therefore does not support characterizing its samples as students rather than professional recruiters without qualification. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites Neumann and Kuncel 2025 sourceSupported

May combine two claims.

This pattern is consistent with recent findings in personnel selection and with the broader literature on overreliance on automated systems.

“This pattern is consistent with recent findings in personnel selection (Cecil et al. 2024) and with the broader literature on overreliance on automated systems (Parasuraman and Manzey 2010;Parasuraman and Riley 1997;Skitka et al. 1999), although it awaits confirmation through formal inferential comparison.”

• cites Cecil et al. 2024 sourceSupported
• cites Parasuraman and Manzey 2010 sourceNot checked — no full text of the source
• cites Parasuraman and Riley 1997 sourceNot checked — no full text of the source

May combine two claims.

Recruiters confronted with incongruent advice maintained their initial judgment more frequently than they revised it, with only 44.2% revising, significantly below the 50% chance rate.

“Results revealed widespread egocentric advice discounting.”

• Also stated in: § 7.2 | Hypothesis Testing · plain

  “Only 44.2% of recruiters revised their initial judgment (335 out of 758), which was significantly lower than the chance rate of 50%, p < 0.001, one-sided 95% CI [0, 0.47].”

May combine two claims.

The interaction between advice source and trust was significant (Waldχ2(1) = 12.8, p < 0.001), and trust in the source significantly affected the likelihood of changing one's decision only when the advice came from a human expert.

“This interaction was significant (Waldχ 2 (1) = 12.8, p < 0.001), and graphical analysis (see Figure 1) showed that trust in the source significantly affected the likelihood of changing one's decision only when the advice came from a human expert.”

May combine two claims.

In dual-advice conditions (N = 460), convergent advice had a significantly stronger influence on decision revision than divergent advice (AME = -0.16, p < .001); participants exposed to divergent advice were 16% less likely to change their decision than those exposed to convergent advice.

“A logistic model estimated exclusively under dualrecommendation conditions (N = 460; Table 4) showed that convergent advice exerted a significantly stronger influence than divergent advice (AME = -0.16,p < 0.001).Participants exposed to divergent advice were 16% less likely to change their decision than those exposed to convergent advice, supporting Hypothesis 4.”

May combine two claims.

Inaccurate advice from the ARS produced the highest rate of change (40%), whereas accurate advice from the human expert produced the lowest rate (22%).

“Inaccurate advice from the ARS produced the highest rate of change (40%), whereas accurate advice from the human expert produced the lowest rate (22%).”

May combine two claims.

In divergent dual-advice conditions, a slight majority of participants aligned with the ARS (53.2%) rather than the human expert (46.8%), although this did not differ significantly from chance (p = 0.299, 95% CI [47.3%, 58.9%]).

“A slight majority of participants aligned with the ARS (53.2%) rather than the human expert (46.8%), although this did not differ significantly from chance, exact binomial test, p = 0.299, 95% CI [47.3%, 58.9%].”

May combine two claims.

Advice accuracy significantly predicted both the deterioration and the improvement of recruitment decisions.

“Advice accuracy significantly predicted both the deterioration and the improvement of recruitment decisions.”

May combine two claims.

Accurate advice improved decision optimality, with the largest gain observed for accurate AI advice, whereas inaccurate advice reduced it, with the largest decline also occurring for inaccurate AI advice.

“As expected, accurate advice improved decision optimality, with the largest gain observed for accurate AI advice, whereas inaccurate advice reduced it, with the largest decline also occurring for inaccurate AI advice.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In congruent cases, 91.6% of participants maintained their initial decision.

“in these cases, 91.6% of participants maintained their initial decision.”

May combine two claims.

Neither objective expertise nor perceived self-efficacy in recruitment significantly affected decision change (AME = -0.02, p = 0.73 for objective expertise; AME = -0.0007, p = 0.96 for perceived self-efficacy).

“The results showed that neither objective expertise nor perceived selfefficacy in recruitment significantly affected decision change: average marginal effect (AME) = -0.02,p = 0.73 for objective expertise; and AME = -0.0007,p = 0.96 for perceived self-efficacy in recruitment (Table 2).”

May combine two claims.

Recruiters' expertise was not significantly related to the likelihood of changing their decision after receiving advice: neither objective recruitment expertise nor perceived self-efficacy significantly predicted decision change.

“Contrary to earlier findings (Bruun et al. 2022), recruiters' expertise was not significantly related to the likelihood of changing their decision after receiving advice: neither objective recruitment expertise nor perceived selfefficacy significantly predicted decision change.”

May combine two claims.

The accuracy of the initial decision had no significant effect on decision change; the rate of change was slightly higher among respondents whose first decision was correct than among those whose first decision was incorrect (45.8% vs 41.4%, χ2(1, N = 758) = 1.37, p = 0.24).

“The accuracy of the initial decision had no significant effect.The rate of change was slightly higher among respondents whose first decision was correct than among those whose first decision was incorrect: 45.8% versus 41.4%, χ 2 (1, N = 758) = 1.37, p = 0.24.”

May combine two claims.

A trust-reliance paradox emerged: recruiters reported lower trust in the ARS yet were behaviorally more influenced by it.

“a trust-reliance paradox emerged: recruiters reported lower trust in the ARS yet were behaviorally more influenced by it.”

• AssumptionRests on an assumption: Participants perceived the advice as originating from the intended source (human expert vs. ARS).

May combine two claims.

Human advisors are often perceived as more socially legitimate, trustworthy, and context-sensitive, whereas algorithmic systems may be perceived as more objective and consistent.

“Human advisors are often perceived as more socially legitimate, trustworthy, and contextsensitive, whereas algorithmic systems may be perceived as more objective and consistent.”

• cites Lee 2018 sourceNot checked — no full text of the source
• cites Newman et al. 2020 sourceNot checked — no full text of the source

May combine two claims.

Convergent recommendations may reinforce perceived validity and reduce decisional ambiguity, thereby increasing the likelihood of judgment revision.

“Convergent recommendations may reinforce perceived validity and reduce decisional ambiguity, thereby increasing the likelihood of judgment revision.”

May combine two claims.

Individuals often favor congruent information over objectively accurate but contradictory recommendations, a tendency driven by confirmation and self-validation processes.

“Existing research shows that individuals often favor congruent information over objectively accurate but contradictory recommendations, a tendency driven by confirmation and self-validation processes (Yaniv 2004).”

• cites Yaniv 2004 sourceNot checked — no full text of the source

May combine two claims.

The human and algorithmic advice differed in more respects than the source alone: the ARS provided numerical scores together with a ranking, whereas the human expert delivered a narrative verbal assessment that included qualifying remarks and no numerical scores.

“The human and algorithmic advice differed in more respects than the source alone: the ARS provided numerical scores (soft skills, hard skills, managerial profile, and overall match) together with a ranking, whereas the human expert delivered a narrative verbal assessment that included qualifying remarks and no numerical scores.”

The quote only partly states this claim.

Recruiters overrely on algorithmic advice.

“At the organizational level, the observed overreliance on algorithmic advice suggests that organizations should not treat user trust as the sole metric of successful AI implementation.”

May combine two claims.

Disagreement among advisors reduces trust and decreases the weight assigned to external inputs.

“Disagreement among advisors reduces trust and decreases the weight assigned to external inputs (Bonaccio and Dalal 2006;Wanzel et al. 2017;Yaniv et al. 2009).”

• cites Bonaccio and Dalal 2006 sourceNot checked — no full text of the source
• cites Wanzel et al. 2017 sourceNot checked — no full text of the source
• cites Yaniv et al. 2009 sourceNot checked — no full text of the source

May combine two claims.

Four human resource management experts reviewed the CVs and confirmed that the profiles were sufficiently distinguishable, with one application considered objectively stronger because it met a greater number of the job requirements.

“To ensure realism and relevance, four human resource management experts reviewed the CVs and confirmed that the profiles were sufficiently distinguishable: one of the two applications was considered objectively stronger because it met a greater number of the requirements listed in the job description.”

May combine two claims.

Human oversight alone may be insufficient to mitigate algorithmic harm: rather than supervising the system objectively, recruiters may be influenced by its recommendations in ways that affect decision quality.

“These results suggest that human oversight alone may be insufficient to mitigate algorithmic harm: rather than supervising the system objectively, recruiters may be influenced by its recommendations in ways that affect decision quality.”

Classified as probably not a substantial claim. May combine two claims.

Advice influence was measured exclusively in cases in which the advice conflicted with participants' initial decisions; congruent cases were excluded because they involved no cognitive conflict and offered little opportunity to observe judgment revision.

“Advice influence was measured exclusively in cases in which the advice conflicted with participants' initial decisions.Congruent cases were excluded because they involved no cognitive conflict and offered little opportunity to observe judgment revision; in these cases, 91.6% of participants maintained their initial decision.”

May combine two claims.

The two conditions differed not only in source, but also in the format, richness, and consistency of the informational cues provided.

“The two conditions therefore differed not only in source, but also in the format, richness, and consistency of the informational cues provided.”

May combine two claims.

Overreliance can negatively affect decision quality and progressively erode human expertise.

“Such overreliance can negatively affect decision quality and progressively erode human expertise (Diefenhardt 2023).”

• cites Diefenhardt 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Trust formation, trust calibration, and reliance on automated advice are inherently dynamic processes that develop over repeated interactions, as users observe the accuracy of advice and adjust their reliance accordingly.

“Trust formation, trust calibration, and reliance on automated advice are inherently dynamic processes that develop over repeated interactions, as users observe the accuracy of advice and adjust their reliance accordingly (Lee and See 2004).”

• cites Lee and See 2004 sourceNot checked — no full text of the source

May combine two claims.

The experimental manipulations were effective: the different conditions produced significantly different effects (χ2(7, N = 1,208) = 16.68, p = 0.02).

“These results indicate that the experimental manipulations were effective: the different conditions produced significantly different effects (χ 2 (7, N = 1,208) = 16.68,p = 0.02).”

Classified as probably not a substantial claim.

Trust is a key determinant of reliance on automation.

“This finding aligns with broader literature positioning trust as a key determinant of reliance on automation (Glikson and Woolley 2020; Lee and See 2004;Solberg et al. 2022).”

• cites Glikson and Woolley 2020 sourceNot checked — no full text of the source
• cites Lee and See 2004 sourceNot checked — no full text of the source
• cites Solberg et al. 2022 sourceNot checked — no full text of the source

May combine two claims.

Empirical findings on behavioral reliance on algorithmic advice remain inconsistent, with some studies reporting reluctance to use algorithmic decision aids in recruitment and others showing that recruiters continue to follow algorithmic recommendations even when they are inaccurate.

“Empirical findings on behavioral reliance on algorithmic advice remain inconsistent.Some studies report reluctance to use algorithmic decision aids in recruitment contexts (Acikgoz et al. 2020;Langer et al. 2019;Oberst et al. 2020), whereas others show that recruiters continue to follow algorithmic recommendations even when they are inaccurate (Cecil et al. 2024;Lacroux and Martin-Lacroux 2022).”

• cites Acikgoz et al. 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Langer et al. 2019 sourceNot checked — no full text of the source
• cites Oberst et al. 2020 sourceNot checked — no full text of the source

May combine two claims.

Convergent advice from multiple sources increases perceived validity and strengthens confidence in external recommendations.

“Research indicates that convergent advice from multiple sources increases perceived validity and strengthens confidence in external recommendations (Bonaccio and Dalal 2006;Soll et al. 2022).”

• cites Bonaccio and Dalal 2006 sourceNot checked — no full text of the source
• cites Soll et al. 2022Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Both streams of research remain incomplete if they rely exclusively on self-reported preferences, as explicit distrust does not prevent individuals from relying on algorithmic advice.

“Both streams of research remain incomplete if they rely exclusively on self-reported preferences, as explicit distrust does not prevent individuals from relying on algorithmic advice.”

Classified as probably not a substantial claim.

Participants who failed to answer all four attention-check questions correctly were excluded from the analysis.

“Participants who failed to answer all four questions correctly were excluded from the analysis.”

Probably not a substantial claim: may not pass S2 (checkable).

The central challenge lies in behavioral integration under cognitive conflict.

“The central challenge lies in behavioral integration under cognitive conflict.”

Classified as probably not a substantial claim. May combine two claims.

This design choice was motivated by ecological validity, as human advice in real recruitment settings is rarely delivered as standardized scores; it nevertheless limits the internal validity of source comparisons.

“This design choice was motivated by ecological validity, as human advice in real recruitment settings is rarely delivered as standardized scores; it nevertheless limits the internal validity of source comparisons.”

• AssumptionRests on an assumption: In real recruitment settings, human advice is rarely delivered as standardized scores.

May combine two claims.

Recruiters are subject to egocentric bias but remain capable of revising their initial decisions, sometimes at the expense of decision quality.

“Although recruiters are subject to egocentric bias, they remain capable of revising their initial decisions, sometimes at the expense of decision quality.”

Classified as probably not a substantial claim.

The experiment involved a single decision round.

“Third, the experiment involved a single decision round.”

Classified as probably not a substantial claim. May combine two claims.

Because judges make predictions with greater confidence when cues are coherent and unambiguous, the highly consistent quantitative output of the ARS may have appeared more diagnostic than the more nuanced human assessment, and the expert's explicit reservations may have signaled lower advisor confidence, which is known to reduce advice taking.

“Because judges make predictions with greater confidence when cues are coherent and unambiguous (Kahneman and Tversky 1973;Slovic 1966), the highly consistent quantitative output of the ARS may have appeared more diagnostic than the more nuanced human assessment, and the expert's explicit reservations may have signaled lower advisor confidence, which is known to reduce advice taking.”

• cites Kahneman and Tversky 1973 sourceNot checked — no full text of the source
• cites Slovic 1966 sourceNot checked — no full text of the source

May combine two claims.

These findings highlight two distinct vulnerabilities in AI-assisted recruitment: insufficient reliance on valid but incongruent advice, driven by egocentric discounting, and susceptibility to overreliance on automated recommendations.

“These findings highlight two distinct vulnerabilities in AI-assisted recruitment: insufficient reliance on valid but incongruent advice, driven by egocentric discounting, and susceptibility to overreliance on automated recommendations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Personnel selection remains a domain in which professional intuition is frequently valued despite evidence supporting more structured approaches.

“In recruitment, this tendency is particularly relevant because personnel selection remains a domain in which professional intuition is frequently valued despite evidence supporting more structured approaches (Highhouse 2008).”

• cites Highhouse 2008 sourceNot checked — no full text of the source

May combine two claims.

Although egocentric discounting persists, algorithmic advice may exert stronger behavioral influence than human expertise despite lower explicit trust.

“By drawing on the JAS framework, we show that although egocentric discounting persists, algorithmic advice may exert stronger behavioral influence than human expertise despite lower explicit trust.”

May combine two claims.

For hybrid recruitment, multi-source advice is not inherently beneficial: consensus can reinforce accurate decisions, but it can also amplify errors when both human and algorithmic advisors are incorrect.

“For hybrid recruitment, this suggests that multi-source advice is not inherently beneficial: consensus can reinforce accurate decisions, but it can also amplify errors when both human and algorithmic advisors are incorrect.”

May combine two claims.

When trust exceeds capabilities, individuals may rely too heavily on automated recommendations and reduce vigilant information processing.

“Conversely, when trust exceeds those capabilities, individuals may rely too heavily on automated recommendations and reduce vigilant information processing (Logg et al. 2019;Mosier et al. 1998;Parasuraman and Manzey 2010).”

• cites Logg et al. 2019 sourceNot checked — no full text of the source
• cites Mosier et al. 1998 sourceNot checked — no full text of the source
• cites Parasuraman and Manzey 2010 sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

A one-shot decision scenario cannot capture these dynamics, and inferences about trust processes and human oversight drawn from a single decision should therefore be generalized with caution.

“A one-shot decision scenario cannot capture these dynamics, and inferences about trust processes and human oversight drawn from a single decision should therefore be generalized with caution.”

May combine two claims.

Divergent advice created an arbitration problem and made recruiters less likely to revise their initial judgments, whereas convergent advice facilitated decision revision.

“Whereas divergent advice created an arbitration problem and made recruiters less likely to revise their initial judgments, convergent advice facilitated decision revision.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The categorical approach (binary decision change) lacks the granularity needed to detect more subtle judgment adjustments.

“Although this categorical approach clarifies final outcomes, it lacks the granularity needed to detect more subtle judgment adjustments.”

May combine two claims.

Judges tend to give greater weight to their own initial assessments because they have direct access to the reasoning underlying their own judgments, but not to the reasoning underlying external advice.

“This finding is consistent with Yaniv (2004) argument that judges tend to give greater weight to their own initial assessments because they have direct access to the reasoning underlying their own judgments, but not to the reasoning underlying external advice.”

• cites Yaniv (2004) sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The baseline subjectivity of the recruitment task, with 37% of respondents initially selecting the weaker candidate, highlights the need for stricter experimental control to isolate the unique variance of algorithmic influence from idiosyncratic biases.

“In addition, the baseline subjectivity of the recruitment task, with 37% of respondents initially selecting the weaker candidate, highlights the need for stricter experimental control to isolate the unique variance of algorithmic influence from idiosyncratic biases.”