Why Your Job Description Is Filtering Out the People You Actually Need

The standard AI job description, years of experience with named tools, certifications, GitHub repos, screens for the wrong things and actively filters out the candidates you want.

Most AI job descriptions are written by people who are trying to solve a legitimate problem, finding someone who can actually do the work, using the wrong tools for the job. They copy the credential conventions of established technical roles, add AI tool names to the requirements list, and post. The result is a filter that selects for a particular kind of candidate: one who has been optimizing for resume signals rather than building real capability.

The Stanford HAI AI Index Report 2024 documented that AI-related job postings in the US grew faster than overall tech job postings in 2023. At the same time, industry accounts consistently describe a gap between the volume of AI applications received and the number of hires who perform as expected. The pipeline is large. The qualified yield is not. This gap is partly a hiring problem, and the job description is the front end of the hiring problem.

The Credential Trap

AI certifications have proliferated. Virtually every major platform, AWS, Google Cloud, Microsoft Azure, Coursera, and dozens of others, now offers AI certification programs. This creates a convenient credential proxy: require certification X and screen for people who have it.

The problem is that these certifications largely measure tool familiarity, not thinking quality. They test whether someone can answer multiple-choice questions about APIs, model categories, and platform-specific features. They do not test whether someone can decompose a vague business problem into a well-structured AI approach, evaluate whether a model's output is wrong in a way that matters, or design a workflow that compounds over time rather than requiring constant manual intervention.

The candidates who pursue certifications most aggressively are often optimizing for credential acquisition rather than capability development. The genuinely effective practitioners are often busy doing the work, not collecting badges that testify to it. A hiring filter built around certifications skews toward the credential optimizers and away from the capability builders.

A certification tells you someone studied the documentation. It does not tell you what they built, what failed, and what they learned from it.

The Experience Paradox

The standard requirement of "3 years of experience with AI tools" carries an assumption that does not hold in this domain: that more years means more current and more valuable knowledge. In most fields, three years of experience means three years of compounding practice. In AI, it may mean three years of exposure to approaches that have been superseded.

The GitHub Octoverse 2024 report documented how rapidly AI developer tool adoption patterns shifted in a single year. Models, interfaces, and best practices that were considered standard in 2022 were materially different from those considered standard in 2024. A candidate with three years of experience who stopped actively learning eighteen months ago may have deep familiarity with outdated approaches, a kind of fluency that can be harder to unlearn than no fluency at all.

The experience requirement also disadvantages the candidates who are most current. Someone who has spent the past nine months doing intensive, project-based work with current AI tools may be more practically capable than someone with a longer resume that reflects earlier, different conditions. The experience proxy, useful in stable domains, misfires in one where the relevant knowledge base turns over as quickly as this one does.

What Resumes Cannot Show

The three things that most distinguish top AI practitioners from adequate ones are exactly the three things a resume cannot show: judgment under ambiguity, the ability to redesign workflows rather than just add a tool to an existing one, and the capacity for compounding leverage over time.

Judgment under ambiguity. When the problem is not cleanly specified, when the AI output is plausibly right but hard to verify, when the tool returns something unexpected: how someone navigates that situation is the core of what matters. Resumes show previous roles. They do not show the quality of reasoning in unclear situations.

Workflow redesign. Adding AI to an existing workflow as a search accelerator or draft generator is one thing. Recognizing that a workflow is structured in a way that AI makes obsolete, and restructuring it accordingly, is another. The second capability is what creates compounding value over time. It is also invisible on a resume.

Compounding leverage. The most effective AI practitioners build tools that keep working, scaffolding and templates that improve with use, evaluation frameworks that make future work faster. The resume shows point-in-time contributions. It does not show whether those contributions were one-off outputs or foundations that keep generating returns.

What the Signal Actually Is

The right signals for identifying genuine AI mastery are qualitative and require engagement to surface. They include:

How to Rewrite the JD

The rewrite requires changing what you screen for at each stage. In the job description itself: replace tool lists with descriptions of the problems the role will actually need to solve. Instead of "3+ years experience with LLMs," write "experience redesigning high-volume knowledge workflows using AI, with demonstrated ability to evaluate output quality and iterate on approach." This screens for the activity, not the tool.

Replace certification requirements with portfolio requirements: "please submit one example of a workflow you redesigned using AI, describing the before state, the approach you took, and the measurable outcome." This forces the candidate to produce evidence of the thing you actually want to evaluate, and it significantly reduces the volume of credential-optimizing applicants who will not have a portfolio to submit.

At the screening stage: route applicants to a short structured prompt that asks them to describe a specific problem they solved with AI and walk through their reasoning. The quality of that description, not the length or polish of it, is the signal you are evaluating.

The Organizational Cost of the Wrong Hire

The downstream cost of a misaligned AI hire is worth making explicit, because it is often understated when organizations are evaluating the investment required to improve the hiring process. A senior AI hire who looks right on a credential-heavy JD and underdelivers on the actual work produces several costs beyond the compensation.

There is the opportunity cost of the twelve to eighteen months that organization spent waiting for results that did not materialize. There is the organizational skepticism that follows a high-visibility AI hire who did not deliver: future AI investments face higher scrutiny and more resistance because the initial hire set negative expectations. There is the churn cost of restarting the search, which includes recruiter fees, leadership time, and the period of reduced capacity while the role is open. And there is a subtler cost: the AI initiatives that did not get started or were paused while the organization waited for the hire to ramp up.

Against this cost, the investment required to redesign the job description and the evaluation process is modest. The most expensive element is structured design time, not additional headcount or technology. Organizations that treat hiring process improvement as a discretionary nicety rather than a strategic investment are consistently underestimating the cost of the status quo.

Why the Standard JD Attracts the Wrong Pool

The credential-heavy job description does not just fail to screen for the right people. It actively makes the role less attractive to them. The top-1% practitioner, the person who has developed genuine mastery through sustained practice on real problems, reads a job description that asks for "AWS AI Practitioner certification, 3+ years with GPT API, GitHub portfolio of AI projects" and draws one of two conclusions: either the organization does not understand what good AI work actually looks like, or it is optimizing for surface signals because it lacks the internal expertise to evaluate substance. Neither conclusion makes the role look attractive.

The candidates who are most enthusiastic about a credential-heavy JD are the ones who have spent their time accumulating credentials. That is a useful filter, but it is filtering in the wrong direction for most of what organizations actually need from an AI hire.

The GitHub Octoverse 2024 report found a significant divergence between AI tool installation rates and active daily usage rates among developers. This divergence is a proxy for the broader gap between adoption and mastery. A job description that screens for GitHub repositories of AI projects is selecting for people who have installed tools and published code. It is not selecting for people who have integrated those tools into how they actually work on consequential problems.

A Rewrite Comparison

The practical difference between a standard JD and a rewritten one is substantial. Consider a standard requirement: "5+ years of AI/ML experience, proficiency in Python and major cloud AI platforms, completion of at least one AI certification program, strong GitHub portfolio demonstrating AI projects." This requirement selects for tenure, language familiarity, credential completion, and published code. It does not select for any of the things that make an AI practitioner actually effective in an enterprise context.

A rewritten requirement for the same role might read: "Experience redesigning knowledge workflows using AI tools, with the ability to describe a specific before/after transformation in your own work and the measurable outcome it produced. Demonstrated ability to evaluate AI output quality and identify failure modes in your domain. Track record of building workflow components that continue to generate value after initial implementation, not just one-off outputs." This requirement selects for exactly the activities that distinguish effective practitioners from occasional users.

The rewrite requires more judgment to evaluate at the screening stage. That is a feature, not a bug. The evaluation that is harder to automate is also more likely to surface the signal that matters. Organizations that invest in a structured screening process calibrated to these signals will consistently find better candidates than organizations that run resumes through a keyword filter and call the survivors for a generic technical interview.

The Structural Change Required in Recruiting

Rewriting the JD is necessary but not sufficient. The structural change required in recruiting is a shift in what the evaluation process is actually evaluating. Most AI hiring processes involve a technical screen, a system design interview, and behavioral questions. The technical screen tests knowledge of algorithms, frameworks, and APIs. The system design interview tests architecture knowledge. The behavioral questions test communication ability. None of these exercises, in their standard form, evaluates the cognitive habits that distinguish top-1% AI practitioners.

The evaluation process needs to include at least one component that directly tests the things the role requires: a portfolio review of real work the candidate has actually done, a live decomposition exercise with an ambiguous real problem, and a failure case discussion where the candidate describes a time AI produced a wrong answer and what they did. Part 5 of this series covers interview design in detail. The key structural point here is that the JD and the evaluation process need to be aligned: if the JD is asking for portfolio evidence and the evaluation process is running a LeetCode-style technical screen, the signals are contradictory and the process will produce confused results.