Can AI pass a first-semester CS class like Cornell CS2112?

A Cornell TA tested three top models (ChatGPT, Claude, Gemini) on every graded element of the class. The results were mixed; some early tasks scored high but later tasks and some exams revealed significant limitations, showing that AI isn't reliably passing yet.

What grades did the AI models receive in the CS2112 exercise?

Early tasks saw high scores (e.g., ChatGPT 102/104), but later tasks varied (ChatGPT 32/100 on one, Claude 6/100, Gemini 13/100 on another). Overall, Claude and Gemini ended with C+ in the class, while ChatGPT fared better (B+ in this informal assessment).

Why is Dario Amodei's PhD-level claim considered misleading?

The host argues that describing LLMs with human education levels anthropomorphizes the models and ignores their specialization. The claim originated from solving certain math problems—not from a general intelligence level—and isn't a good framework for understanding how these tools are used.

What does 'AI washing' mean in this context?

AI washing refers to journalists attributing layoffs or strategic moves to AI to paint the company in a future-facing light, rather than acknowledging other causes like pandemic overhiring.

What is the current outlook on the future of AI in programming and best practices?

The host says progress is real but not revolutionary in every setting. There will likely be a middle ground with evolving best practices, less hyper-multi-agent setups, and more standardized planning and review processes as tooling matures.

What final takeaway does Cal Newport emphasize about AI coverage?

Take AI seriously, but avoid hype and unfounded conclusions. He plans to continue gathering data from real programmers and reporting on what's actually happening rather than sensational claims.

Did the AI Job Apocalypse Just Begin? (Hint: No.) | AI Reality Check | Cal Newport

Deep Questions with Cal Newport

People & Blogs5 min read30 min video

Mar 5, 2026

Save to Pod

Key Moments

TL;DR

Block layoffs misattributed to AI; real impact is nuanced and not apocalyptic.

Key Insights

Block's 40% layoff likely reflects overhiring during the pandemic and strategic right-sizing, not a direct AI mandate.

Media framing tends to attribute layoffs to AI ('AI washing'), which can mislead about what's actually changing.

Anthropic's 'PhD-level' claim is an anthropomorphism; a Cornell CS2112 experiment shows AI as a specialized tool with limits.

Professional programmers use AI in varied ways; a spectrum from heavy reliance to cautious adoption; multi-agent setups are not universal.

Best practices and standards for agentic coding are still emerging; productivity gains come with overhead and review requirements.

The takeaway is to track real impacts with careful analysis rather than hype, and to prepare for gradual, not earth-shattering, shifts.

BLOCK LAYOFFS: HYPE VS. REALITY

The Block layoff episode serves as a case study in hype versus evidence. Jack Dorsey's X-post suggested the move was not driven by trouble but by the emergence of 'intelligence tools' that enable leaner teams and a new way of working. The reporting around it quickly framed the decision as AI-driven; New York Times-style headlines used language like 'due to AI'. The lack of specificity is telling: no explicit tool or function is named, and the rationale rests on broad assertions about productivity. Meanwhile Block's own numbers indicate a pandemic-era overhiring, acquisitions, and two consecutive earnings misses, suggesting structural adjustments rather than a clean 'AI layoff'. Analysts and observers push back against the easy AI narrative: overhiring, sector headwinds, and strategic bets explain much of the headcount changes. The concept of 'AI washing'—journalists using AI as a scapegoat to justify layoffs or to juice stock—adds another layer of skepticism. Newport urges caution: AI will affect jobs, but the evidence for a rapid, broad-based apocalypse is missing; we should demand specificity, guard against sensational headlines, and track actual workforce changes to see where automation or workflow changes are really taking hold.

ANTHROPIC'S PHDS IN A DATA CENTER: A TA'S EYE-OPENING TEST

Anthropic's high-level framing of LLMs as a 'data center full of PhDs' or a 'country of geniuses' in your infrastructure has always sounded impressive, but real-world tests tell a different story. A Cornell CS2112 TA ran every graded assignment from a freshman CS class through three top models (ChatGPT, Claude, Gemini) and graded them with the same rubrics as human students. The results were mixed: early tasks could be handled well by the models, with scores around high 90s; a final exam scored in the 80s for some models, but other assignments showed stark failures (scores as low as 13-32). The models failed to consistently follow constraints, hallucinated rules, or produced outputs that didn’t match the assignment’s requirements. The bottom line: the models are not general-purpose substitutes for a college-educated mind; the 'PhD' analogy obscures their strengths and their limits. The takeaway is that these are specialized tools whose value comes from the human plus tool workflow, where prompts, verification, and integration with human processes determine real outcomes.

REAL PROGRAMMERS, REAL USES: WHAT OUR SURVEY SHOWS ABOUT AGENTIC AI

Newport's conversations with hundreds of professional programmers reveal a spectrum of adoption and practice. One excerpt describes a workflow where planning, iterating, and executing with AI, aided by a single-agent tool, vastly accelerates progress, though it requires careful verification. The developer emphasizes ongoing oversight, version control with Git, and the limits of multi-agent setups in practice due to context switching and degraded quality. A second excerpt captures a more cautious reality: automation makes repetitive tasks easier but also introduces overhead—prompt crafting, output checking, and heavier code-review burdens when output is AI-generated. Taken together, about 45% of respondents reported producing the majority of their code via an agentic tool, but there is wide variation in enthusiasm and approach. The upshot is that best practices are still forming; hyper-multi-agent configurations are not yet mainstream for serious work; and the future of programming will likely feature standardized planning and review processes that balance productivity gains with quality control.

AI WASHING, HYPE, AND THE NEED FOR NUANCE

A recurring theme across Newport's discussion is the tension between hype and evidence. The narrative that AI will instantly automate vast swaths of employment is seductive but premature; media narratives tend to box complex trends into a single headline. 'AI washing'—attributing changes to AI to please audiences or boost stock—erodes trust and distorts policy and investment decisions. The Cornell experiment and Block's case highlight that change is uneven across industries and tasks; the most robust gains come from targeted workflow redesign, human oversight, and careful verification rather than hype-driven mandates. To responsibly cover AI's impact, journalists, researchers, and leaders must separate correlation from causation, demand specificity about what tools are used, and focus on measurable changes in productivity, job roles, and training needs rather than sweeping predictions.

LOOKING AHEAD: PRACTICAL PATHS FOR AI IN TEAMS

What emerges from this reality check is a practical path forward: automation will reshape how teams work, not simply erase jobs. Best practices will crystallize, with clearer roles for humans and agents, more explicit planning documents, and standardized review protocols that preserve quality. Expect a shift toward greater reskilling and ongoing education so workers can effectively supervise AI outputs and intervene when outputs go astray. The trend will be gradual: productivity gains from targeted automation, new tools, and smarter workflows, rather than a universal, instantaneous replacement of human labor. Newport urges us to move beyond hype, to document real workflows, and to learn from real programmers’ experiences. The goal is to build resilient teams that leverage AI to augment capability while maintaining accountability and clarity about what changes and why.

Mentioned in This Episode

●Tools & Products

●People Referenced

Common Questions

No. The host notes that the attribution to AI is not well-supported by specifics, pointing to overhiring during the pandemic and other non-AI factors. He stresses the need for careful attribution rather than accepting hype about an AI-driven ‘job apocalypse’.