Jim Keller: Moore's Law, Microprocessors, and First Principles | Lex Fridman Podcast #70

Initially, branch prediction involved recording the last outcome or using simple counters. Modern CPUs use sophisticated deep pattern recognition, similar to neural networks, to predict branches with over 99% accuracy by analyzing execution flows and historical data, making large instruction windows effective.

For correct C programs, the formal definition states they must produce the same answer every time. Even though the internal execution flow of a modern computer (with out-of-order execution and branch prediction) is highly non-deterministic, the final result is consistently deterministic.

Jim Keller suggests completely rewriting computer architectures every three to five years to achieve substantial progress, opposing the tendency for incremental improvements that eventually hit fundamental limits.

Beyond just doubling transistor count, Moore's Law operationally means increasing computer performance by 2x every 2-3 years. This is driven by thousands of innovations, not just shrinking transistors, creating a cascade of 's-curves' that result in exponential growth.

Moore's Law continues due to thousands of ongoing innovations, each with its own S-curve, including advances in materials science, chemistry, and manufacturing methods. He believes shrinking transistors alone has potential for another 10-20 years of significant improvement, perhaps a factor of 100.

Every order of magnitude increase in computation fundamentally changes the kind of computation possible. This has evolved from simple rule sets and basic searches to complex deep searches and topological computations, enabling new mathematical abstractions and AI capabilities that are not merely optimizations of old methods.

The main challenge is balancing the need for specialization, to achieve significant performance gains over generic GPUs (2-5x), with the rapidly evolving AI algorithms. Over-specialization can render an accelerator ineffective if algorithms change too much. Additionally, cost constraints are critical for widespread adoption.

Yes, Jim Keller is confident that autonomous driving can be solved within 1-10 years. He bases this on the fact that driving is not an inherently 'hard' problem for computers, especially due to their superior attention, continuous data gathering, improving algorithms, and ability to process ballistic and topographic data.

No, he does not. He believes that superintelligence is likely to create new, interesting possibilities rather than conflicts. He suggests that common fears stem from human self-esteem issues about being the 'smartest local person' and that the interests of a super-intelligent entity might be astronomically different from human desires for things like 'a square foot of dirt'.