AI Forecasting
- Bump them across each time
- "Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters."
- Will there be a $10 billion cluster by the end of 2025, 2026, 2027, 2028, 2029
- Will there be a $100 billion cluster by the end of 2026, 2027, 2028, 2029
- How are we gonna do adjusted by inflation?
- Will there be major AI legislation before 2028
- Will the largest model in [date] be larger than [size] according to epochs retroactive estimate
- Forecast different benchmarks
- How to split of the growth from different aspects
- Open Philanthropy Read https://www.openphilanthropy.org/research/new-report-on-how-much-computational-power-it-takes-to-match-the-human-brain/
- Conceptual Overview:
- There are various methods to estimate the Floating Point Operations per Second (FLOP/s) required for a machine to match the human brain's task-performance. While these estimates offer valuable insights, the underlying science remains tentative, with no consensus among experts.
- 1) Limit Method:
- · This method determines an upper bound for required FLOP/s based on physical limits, emphasizing Landauer's principle, which deals with the minimum energy costs of erasing bits.
- · Computers perform FLOP operations that erase bits. An ideal computer operating on the brain's 20W energy can perform up to ~7×10^21 standard FLOP/s.
- · However, the brain's operations might not mirror standard FLOPs, so this doesn't serve as a definitive upper bound.
- · Questions arise about the FLOP/s required to match brain function and the max bits per second the brain can erase. While many experts believe the latter exceeds the former, these beliefs are based on algorithmic and hardware arguments. Both these arguments present general considerations, but the report leans more towards the hardware arguments.
- · The report suggests that it's unlikely the required FLOP/s exceeds the bounds proposed by the limit method, but this isn't considered a strict physical limit.
- 2) Communication Method:
- · This method assesses the brain's computational capacity using its communication bandwidth. The value of computation and communication are interconnected, influencing each other.
- · Two example estimates include:
- o Dr. Paul Christiano approximated the brain's communication capacity to be similar to or less than a V100 GPU, hinting at a computational capacity of around 10^14 FLOP/s or less.
- o AI Impacts proposes using Traversed Edges Per Second (TEPS) to quantify the brain's communication ability, suggesting a range of ~2×10^13 to 6×10^14 TEPS for the brain. This translates to around 10^16-3×10^17 FLOP/s. However, these estimates haven't been thoroughly validated.
- 3) Mechanistic and Functional Estimates:
- · Mechanistic estimates, which suggest that 10^13–10^17 FLOP/s might be enough for machines to match the human brain's performance, are viewed favorably by the author. While there are arguments for both higher and lower numbers, the lower range appears more convincing.
- · Functional method estimates, especially those based on the visual cortex, serve as weak evidence that the 10^13–10^17 FLOP/s range isn't a drastic underestimation. Some estimates based on deep neural network models provide higher numbers but are seen as even weaker evidence.
- Conclusions:
- · Despite the wide range of estimates, the author believes that it's more probable than not that 10^15 FLOP/s would be enough for a machine to perform tasks on par with the human brain, assuming the right software is available. There's a perceived less than 10% chance that more than 10^21 FLOP/s would be needed. However, the FLOP/s needed to run a model can be available even while the resources to train it are unattainable.
- · Even with appropriate computational capabilities, achieving AI performance on par with the human brain involves challenges beyond FLOP/s. Factors like memory, computational efficiency, AI development complexity, and more play significant roles. This means that even if computational benchmarks align with these estimates, it doesn't guarantee that we'll see human-equivalent AI systems in the near future.
- Predictions for 2025
- Energy
- Largest model
- Capabilties
- Cluster
- Largest chinese cluster
- Largest chinese model
- Will a chinese model be in top 5
- AI regulation
- Any new major lab
- How to judge algorithmic gains
- How to judge unhobbling
Forecasts I want to see