Explore >> Select a destination

You are here

 distill.pub
Self-Organising Textures
| | sander.ai
The paradox of diffusion distillation - Sander Dieleman
10.2 parsecs away

Travel
| | Thoughts on the tension between iterative refinement as the thing that makes diffusion models work, and our continual attempts to make it _less_ iterative.
| | qualiaresearchinstitute.org
Digital Sentience Requires Solving the Boundary Problem
10.8 parsecs away

Travel
| | Digital computers will remain unconscious until they recruit physical fields for holistic computing using well-defined topological boundaries.
| | neptune.ai
Introduction to State Space Models as Natural Language Models
13.7 parsecs away

Travel
| | State Space Models (SSMs) have the potential to address one of the key challenges of transformers: scaling efficiently with sequence length.
| | lilianweng.github.io
Generalized Language Models | Lil'Log
65.6 parsecs away

Travel
| [Updated on 2019-02-14: add ULMFiT and GPT-2.] [Updated on 2020-02-29: add ALBERT.] [Updated on 2020-10-25: add RoBERTa.] [Updated on 2020-12-13: add T5.] [Updated on 2020-12-30: add GPT-3.] [Updated on 2021-11-13: add XLNet, BART and ELECTRA; Also updated the Summary section.] I guess they are Elmo & Bert? (Image source: here) We have seen amazing progress in NLP in 2018. Large-scale pre-trained language modes like OpenAI GPT and BERT have achieved great performance on a variety of language tasks using generic model architectures. The idea is similar to how ImageNet classification pre-training helps many vision tasks (*). Even better than vision classification pre-training, this simple and powerful approach in NLP does not require labeled data for pre-training, allowing us to experiment with increased training scale, up to our very limit.