/explore
|
You are here 
|
jaykmody.com | ||
| | | | |
zserge.com
|
|
| | | | | Finally, building a simple GPT model that would finish our sentences. | |
| | | | |
amatria.in
|
|
| | | | | [AI summary] The provided text is an extensive overview of various large language models (LLMs) and their architectures, training tasks, and applications. It includes detailed descriptions of models like GPT, T5, BERT, and others, along with their pre-training objectives, parameter counts, and specific use cases. The text also references key research papers, surveys, and resources for further reading on LLMs and related topics. | |
| | | | |
peterbloem.nl
|
|
| | | | | [AI summary] The text provides an in-depth overview of the Transformer architecture, its evolution, and its applications. It begins by introducing the Transformer as a foundational model for sequence modeling, highlighting its ability to handle long-range dependencies through self-attention mechanisms. The text then explores various extensions and improvements, such as the introduction of positional encodings, the development of models like Transformer-XL and Sparse Transformers to address the quadratic complexity of attention, and the use of techniques like gradient checkpointing and half-precision training to scale up model size. It also discusses the generality of the Transformer, its potential in multi-modal learning, and its future implications across d... | |
| | | | |
jalammar.github.io
|
|
| | | Discussions: Hacker News (397 points, 97 comments), Reddit r/MachineLearning (247 points, 27 comments) Translations: German, Korean, Chinese (Simplified), Russian, Turkish The tech world is abuzz with GPT3 hype. Massive language models (like GPT3) are starting to surprise us with their abilities. While not yet completely reliable for most businesses to put in front of their customers, these models are showing sparks of cleverness that are sure to accelerate the march of automation and the possibilities of intelligent computer systems. Let's remove the aura of mystery around GPT3 and learn how it's trained and how it works. A trained language model generates text. We can optionally pass it some text as input, which influences its output. The output is generat... | ||
