/explore
|
You are here 
|
terrytao.wordpress.com | ||
| | | | |
gilkalai.wordpress.com
What will be our next polymath project? A polymath project (Wikipedia) is a collaboration among mathematicians to solve important and difficult mathematical problems by coordinating many mathematicians to communicate with each other on finding the best route to the solution. The project began in January 2009 on Timothy Gowers's blog when he posted a problem... |
|
| | | | | ||
| | | | |
www.depthfirstlearning.com
[AI summary] The provided text is a detailed exploration of the mathematical and statistical foundations of neural networks, focusing on the Jacobian matrix, its spectral properties, and the implications for dynamical isometry. The key steps and results are as follows: 1. **Jacobian and Spectral Analysis**: The Jacobian matrix $ extbf{J} $ of a neural network is decomposed into $ extbf{J} = extbf{W} extbf{D} $, where $ extbf{W} $ is the weight matrix and $ extbf{D} $ is a diagonal matrix of derivatives. The spectral properties of $ extbf{J} extbf{J}^T $ are analyzed using the $ S $-transform, which captures the behavior of the eigenvalues of the Jacobian matrix. 2. **$ S $-Transform Derivation**: The $ S $-transform of $ extbf{J} extbf{J}^T $ is... |
|
| | | | | ||
| | | | |
francisbach.com
[AI summary] This text discusses the scaling laws of optimization in machine learning, focusing on asymptotic expansions for both strongly convex and non-strongly convex cases. It covers the derivation of performance bounds using techniques like Laplace's method and the behavior of random minimizers. The text also explains the 'weird' behavior observed in certain plots, where non-strongly convex bounds become tight under specific conditions. The analysis connects theoretical results to practical considerations in optimization algorithms. |
|
| | | | | ||
| | | | |
bdtechtalks.com
Gradient descent is the main technique for training machine learning and deep learning models. Read all about it. |
|
| | | |||
