Are CS degrees or bootcamps better for hiring engineers?

Experience matters more than the source of education. A fresh graduate from a CS program without internships is still a junior engineer, similar to someone from a bootcamp, as both lack real-world experience. Focus on assessing their ability to handle projects and real-world responsibilities, rather than just their degree.

What are the most important skills to evaluate in engineer interviews?

While interviews often measure the ability to solve small problems quickly, the crucial skill for employees is solving large, sprawling problems over time. It's hard to measure this directly, but focusing on practical application and problem-solving within a relevant environment can be more predictive than abstract or overly difficult theoretical questions.

Why are easy interview questions sometimes better predictors than hard ones?

Easy questions reveal how well a candidate can solve a problem efficiently and without struggle. While difficult questions might identify deep knowledge, they also have a high false negative rate. Success on simpler, more practical problems often correlates better with on-the-job performance and indicates a more stable engineering skill.

How should a new grad prepare for engineering interviews?

New graduates should solidify their understanding of core computer science concepts like hash tables and breadth-first search, as these are frequently tested. Additionally, practicing coding under time pressure is crucial, as it simulates stressful interview conditions and improves performance.

Do side projects actually matter for getting hired as an engineer?

Side projects primarily help at the initial resume screening stage to get noticed. However, they are rarely given significant weight during the actual interview process itself. Companies often favor a consistent, standardized process over evaluating projects, as it's difficult to gauge the true effort or contribution involved.

What interview methods work best for hiring engineers?

No single method is perfect, but consistency is key. Trial employment can be effective but deters top candidates. Despite their inaccuracies, interviews remain the most common method. The goal is to make them as accurate as possible by standardizing questions, evaluation criteria, and focusing on what truly matters for the role.

How can companies reduce bias and noise in their engineering interviews?

Companies should strive for consistency by asking all candidates the same questions and evaluating them uniformly. Introspection is also vital to consciously decide what qualities are desired in hires, rather than making decisions by accident. Whiteboard coding can skew towards academic skills, so consider practical coding on a computer instead if that's not the target.

What qualities indicate an engineer will stay with a company long-term?

Engineers who are genuinely excited about the company's mission and the specific job have a higher probability of staying than those solely chasing the highest paycheck. While compensation is important, enthusiasm and a good fit with the company culture are strong indicators of retention.

Should engineers constantly keep up with every new technology for interviews?

While companies (especially smaller ones) rarely evaluate flashy new tech directly in interviews, not keeping up poses a long-term risk. Making decisions to ignore new technologies today could leave an engineer behind in ten years, impacting their future career prospects and interview opportunities.

Key Moments

Hiring Engineers with Ammon Bartram

Y Combinator

Science & Technology3 min read42 min video

May 17, 2017|28,661 views|376|22

YC Y Combinator Podcast

Save to Pod

Key Moments

TL;DR

Ammon Bartram discusses effective engineering hiring, emphasizing clear definitions of desired skills and consistent, practical evaluation methods.

Key Insights

Define your company's specific engineering needs rather than a generic 'good engineer'.

Interviews are often noisy, with low correlation between interviewer opinions across different companies.

Experience matters more than credentials (bootcamp vs. CS degree) for real-world problem-solving.

Easy, practical coding problems are often better predictors of performance than complex algorithmic challenges.

Consistency in the interview process is paramount to reducing noise and enabling optimization.

Avoid 'hazing' candidates; focus on objective evaluation and, if necessary, teaching during a failing interview.

DEFINING THE IDEAL ENGINEER FOR YOUR COMPANY

The most crucial first step in hiring engineers is to precisely define what 'good' means for your specific company. Companies often have different definitions of a good engineer, leading to a mismatch where productive engineers might fail in highly academic interviews, and vice versa. Early-stage startups, in particular, should identify the single biggest bottleneck holding them back—be it productivity or the need for deep technical problem-solving—and tailor their hiring process accordingly.

THE NOISE AND INCONSISTENCY OF INTERVIEWS

Ammon Bartram highlights that traditional interview processes are riddled with noise and inconsistency. A candidate performing exceptionally well at one company might fail spectacularly at another, with inter-interviewer agreement comparable to movie reviews. This suggests that the signals gleaned from interviews are often unreliable. Companies often ask questions they themselves are best at, inadvertently biasing their hiring towards similar skill sets and potentially missing out on diverse talent.

CREDENTIALS VS. REAL-WORLD EXPERIENCE

When evaluating candidates, experience in the industry—especially the ability to own and solve large, sprawling problems over time—is generally more valuable than academic credentials like CS degrees or bootcamp completions. While interviews can measure the ability to solve small problems quickly, this doesn't perfectly correlate with the skills needed for long-term project success. Senior engineers, while possessing this real-world experience, may even perform worse in typical interview problem-solving scenarios than more junior candidates.

EFFECTIVE EVALUATION METHODS: BEYOND ALGORITHMS

Triplebyte employs a multi-faceted approach to evaluation, separating the assessment of programming productivity, academic CS knowledge, debugging skills in a large codebase, and system design. Notably, they've found that asking relatively easy coding questions, and observing how easily a candidate solves them and their ability to come up with the correct answer, is often more predictive of job performance than complex algorithmic riddles. This approach aims to reduce false negatives associated with overly difficult problems.

THE ROLE OF SIDE PROJECTS AND RESUME SCREENING

While side projects can be helpful for getting past the initial resume screening by catching a recruiter's eye, they are rarely heavily weighted during the actual interview process. This is partly because assessing the quality and effort behind a project is difficult and inconsistent. Companies often default to credential-based screening (top companies, top schools) due to the sheer volume of applicants and the need for a consistent, albeit crude, filtering mechanism.

CONSISTENCY AND ADAPTABILITY IN HIRING

Bartram emphasizes that consistency in the interview process is the most critical factor for reducing noise. This means asking all candidates the same questions and evaluating them using the same criteria. While interviews are inherently imperfect, this standardization allows companies to then iterate and optimize their process based on observed results. He also advises against 'hazing'—making interviews unnecessarily difficult or stressful—and suggests shifting from evaluation to teaching mode if a candidate is clearly struggling.

PREPARING FOR INTERVIEWS AND LONG-TERM CAREER GROWTH

For new graduates, solidifying foundational computer science concepts like hash tables and breadth-first search is key, alongside practicing coding under timed conditions. For all engineers, staying current with technology is important for long-term career growth, though most companies prioritize fundamental skills and problem-solving over flashy, new technologies. Excitement about the company's mission, rather than just salary, often correlates with higher employee retention.

INTERVIEW DYNAMIC AND COMPANY-SPECIFIC NEEDS

The interview format should align with the company's needs. Whiteboard coding may favor academic skills, while assigning practical coding tasks on a computer might be more appropriate for evaluating productivity. For larger companies, assessing innate intelligence is often the goal, while smaller companies might focus more on immediate productivity within their specific tech stack. Interview panels should be kept small (maximum two-on-one) to reduce candidate stress and maintain consistency.

Mentioned in This Episode

●Software & Apps

●Companies

●Books

Engineer Hiring Checklist

Practical takeaways from this episode

Do This

Clearly define what type of engineer your company needs (productivity vs. intellectual/academic).

Design a consistent interview process that evaluates specific skills in isolation.

Focus on evaluating skills that are predictive of long-term job performance (e.g., problem-solving in a familiar environment).

For new grads, ensure a solid understanding of core CS concepts like hash tables and breadth-first search, and practice coding under pressure.

For experienced candidates, look for ability to own large, sprawling projects and make architectural decisions.

When interviewing, identify what's holding your company back and hire to address that specific need.

If a candidate is struggling significantly, shift from evaluation to teaching mode to be helpful.

Prioritize excitement about the company and its mission for long-term employee retention.

Be consistent in your interview questions and evaluation criteria.

Consider the candidate's comfort with data analysis and broad knowledge for specific roles.

For large companies, expect more focus on innate intelligence; for smaller companies, focus on specific skills and immediate productivity.

Avoid This

Assume all companies define a 'good' engineer the same way.

Ask interview questions that only test what you are best at, biasing your hiring towards your own skillset.

Over-rely on brain teasers or single, nugget-style programming problems, as they introduce significant noise.

Dismiss candidates from bootcamps; focus on real-world experience over educational pedigree.

Give too much weight to side projects during the interview process itself (they are more important for resume screening).

View interview failures as a definitive referendum on a candidate's skill (most engineers don't pass 100% of interviews).

Demand trial employment periods, as this can deter top candidates with multiple options.

Have inconsistent interview processes where candidates are asked different questions.

Treat interviewing as hazing; avoid being mean, aggressive, or cutting interviews short unless the candidate is in pain.

Undervalue the importance of keeping technical skills up-to-date over the long term.

Conduct interviews with large panels, as this increases candidate stress without proportional benefit.

Common Questions

Early-stage startups should first define what's holding them back. If productivity is the bottleneck, prioritize engineers who can bang out code. If security or complex problem-solving is key, focus on those skills. Avoid simply hiring engineers who fit the existing team's strengths, as this can lead to a narrow skillset.