In 2004, Google's headcount was around 3,000. The company processed roughly 100,000 applications a year. Engineers worked on a monolithic C++ codebase. The technical interview—a 45-minute whiteboard exercise featuring graph traversals, dynamic programming, and the occasional brain teaser—made a kind of intuitive sense. The job was algorithms. You wrote search infrastructure, indexing pipelines, and distributed storage systems from near-scratch, and raw CS fluency was a genuine signal of competence.
Twenty-two years later, the job has changed beyond recognition. Google alone has over 180,000 employees across hundreds of services. A typical engineer spends their day navigating codebases they didn't write, debugging distributed failures across services they only partly understand, reviewing pull requests, triaging alerts, and collaborating with AI copilots on every step. Writing a novel algorithm from scratch is a rare event—most engineers go months without doing it.
And yet, the interview hasn't changed. Not meaningfully.
We optimised hiring for the ability to solve puzzles on a whiteboard—and somehow convinced ourselves that this predicts who can ship reliable software in a 3-million-line codebase.
The data is damning#
In 2013, Google's Senior Vice President of People Operations, Laszlo Bock, told the New York Times that the company had done a rigorous internal study correlating interview scores with on-the-job performance. The finding: there was essentially zero relationship. Brain teasers were dropped immediately. But algorithm questions survived—not because the data supported them, but because they were familiar and standardised.
This isn't unique to Google. Decades of industrial-organisational psychology research, most notably the meta-analysis by Frank Schmidt and John Hunter covering 85 years of data across hundreds of thousands of workers, consistently finds the same hierarchy of predictive validity:
Work-sample tests—assessments where candidates perform tasks resembling actual job duties—outperform every other method. Unstructured interviews, which most algorithm interviews effectively are (different interviewers ask different questions, apply different rubrics, and reach wildly different conclusions), trail well behind both.
A 2022 study by CodeSignal analysing outcomes across 350,000 technical assessments found that candidates who scored highly on real-world coding tasks were 3.2× more likely to pass probation than those who scored highly on algorithm-only assessments but poorly on applied tasks. The correlation between algorithm interview performance and first-year retention was 0.12—statistically significant only because the sample was enormous. For practical hiring decisions, it's noise.
What algorithm interviews actually measure#
When you give a candidate 45 minutes to implement Dijkstra's algorithm on a whiteboard, you're testing a very specific set of abilities:
- Recall of specific algorithms and data structures — not the ability to engineer solutions, but the ability to reproduce textbook patterns from memory under pressure.
- Performance under artificial constraint — no documentation, no IDE, no tests, no git history, no co-workers, no AI tools. An environment the candidate will never work in.
- Practice volume — LeetCode has over 3,000 problems. High performers on algorithm interviews typically report 200–400 hours of dedicated practice. That's not engineering skill. That's test prep.
- Socioeconomic privilege — who has 400 hours to burn on practice problems? Not the working parent, not the self-taught engineer working full-time, not the candidate from a non-traditional background.
Meanwhile, here's what algorithm interviews don't test: code comprehension, debugging skills, system design instinct, the ability to work within an existing codebase, communication through code, collaboration, tooling fluency, testing discipline, or the judgment to use AI tools effectively.
In other words, algorithm interviews filter for the traits that matter least and ignore the traits that matter most. They're a sorting mechanism optimised for a reality that no longer exists.
The hidden costs companies refuse to count#
The direct cost of a bad hire is well-studied—typically 1.5× to 3× the role's annual salary once you factor in recruiting, onboarding, lost productivity, and severance. But algorithm-based hiring creates a second cost that's harder to measure and far more insidious: the cost of the candidates you never hire.
A 2024 survey by Hired.com found that 62% of senior engineers (8+ years of experience) reported declining to proceed with at least one interview process in the preceding 12 months specifically because it required algorithm-style assessments. These aren't weak candidates. They're experienced professionals who've decided their time is better spent elsewhere.
The companies that rely on algorithm interviews are, by design, building their pipeline from the subset of engineers willing to play the LeetCode game. That's a selection bias with compounding consequences: your team becomes more homogeneous, your culture reinforces the belief that "real engineers" can invert a binary tree on demand, and your hiring bar drifts further from what actually predicts success.
What the job actually demands#
Ask any engineering manager what separates a strong engineer from an average one on their team. You'll hear the same themes:
- They can pick up an unfamiliar codebase and contribute meaningfully within days, not weeks.
- They read code carefully and debug systematically rather than guessing.
- They write code that other people can understand and maintain.
- They make sensible design trade-offs without over-engineering.
- They use tools effectively—including AI assistants—without becoming dependent on them.
- They ask good questions and communicate technical decisions clearly.
None of these show up in an algorithm interview. All of them show up when you put someone in a real codebase and watch how they work.
The alternative: work-sample assessments at scale#
The concept isn't new. Work-sample tests have been the gold standard in hiring research since the 1980s. What's new is that technology has finally made them feasible at scale. Cloud-based development environments can spin up a full IDE with a pre-configured codebase in seconds. AI-powered evaluation can supplement human review with objective metrics. And modern infrastructure makes it possible to run thousands of assessments concurrently without ops overhead.
The candidate opens a browser and lands in a full VS Code environment with a real project—a multi-file codebase with existing architecture, tests, and documentation. The task: understand the existing code, find and fix a bug that spans multiple components, then extend a feature in a way that's consistent with the project's patterns. They have access to a terminal, a test suite, git history, and AI tools—the same setup they'd have on day one of the job.
The evaluation captures not just whether the code works, but how they got there: comprehension depth, debugging strategy, code quality, and how they integrated AI assistance.
This approach gives hiring managers the one thing algorithm interviews can't: a preview of how the candidate actually works. Not how they perform under artificial constraints, but how they think, debug, and build in an environment that mirrors the job.
The shift is already happening#
The companies with the most sophisticated hiring operations have been moving in this direction for years. Stripe's famous "work on a real Stripe problem" interview. Shopify's craft-focused assessments. Basecamp's paid trial projects. These companies realised that the closest you can get to seeing someone do the job is to let them do the job—or something very close to it.
What's changing now is that this approach is becoming accessible to companies that don't have the resources to build custom assessment infrastructure from scratch. Platforms that offer real-codebase assessments in cloud environments are making work-sample testing practical for companies of any size.
The algorithm interview isn't going to disappear overnight. It persists because it's familiar, because interviewers are trained on it, and because change is hard. But the data is clear, the candidate market is pushing back, and the tools to do better now exist.
The question isn't whether the industry will shift away from algorithm interviews. It's whether your company will be ahead of the curve or behind it.