/explore

Click through on any links that interest you or select the planets on the right to continue exploring the Outer Web.
You are here

 lambdalabs.com
GPT-3 A Hitchhiker's Guide
| | blog.moonglow.ai
Three Kuhnian Revolutions in ML Training
2.2 parsecs away

Travel
| | Parameters and data. These are the two ingredients of training ML models. The total amount of computation ("compute") you need to do to train a model is proportional to the number of parameters multiplied by the amount of data (measured in "tokens"). Four years ago, it was well-known that if
| | lacker.io
Giving GPT-3 a Turing Test
1.7 parsecs away

Travel
| | I've been playing around with OpenAI's new GPT-3 language model. When I got beta access, the first thing I wondered was, how human is GPT-3? How close is it ...
| | gwern.net
The Scaling Hypothesis ยท Gwern.net
2.2 parsecs away

Travel
| | On GPT-3: meta-learning, scaling, implications, and deep theory. The scaling hypothesis: neural nets absorb data & compute, generalizing and becoming more Bayesian as problems get harder, manifesting new abilities even at trivial-by-global-standards-scale. The deep learning revolution has begun as foretold.
| | sander.ai
Perspectives on diffusion - Sander Dieleman
11.8 parsecs away

Travel
| Perspectives on diffusion, or how diffusion models are autoencoders, deep latent variable models, score function predictors, reverse SDE solvers, flow-based models, RNNs, and autoregressive models, all at once!