/explore
|
You are here 
|
www.lesswrong.com | ||
| | | | |
dynomight.net
How likely are we to hit a barrier? |
|
| | | | | ||
| | | | |
gwern.net
On GPT-3: meta-learning, scaling, implications, and deep theory. The scaling hypothesis: neural nets absorb data & compute, generalizing and becoming more Bayesian as problems get harder, manifesting new abilities even at trivial-by-global-standards-scale. The deep learning revolution has begun as foretold. |
|
| | | | | ||
| | | | |
www.greaterwrong.com
It'll take until ~2050 to repeat the level of scaling that pretraining compute is experiencing this decade, as increasing funding can't sustain the current pace beyond ~2029 if AI doesn't deliver a transformative commercial success by then. Natural text data will also run out around that time, and there are signs that current methods of reasoning training might be mostly eliciting capabilities from the base model. If scaling of reasoning training doesn't bear out actual creation of new capabilities that are sufficiently general, and pretraining at ~2030 levels of compute together with the low hanging fruit of scaffolding doesn't bring AI to crucial capability thresholds, then it might take a while. Possibly decades, since training compute will be growing 3x-... |
|
| | | | | ||
| | | | |
www.lesswrong.com
Preface The following text is my submission for theAI Safety Public Materials contest. In it, I try to lay out the importance of AI Safety Research... |
|
| | | |||
