/explore
|
You are here 
|
www.depthfirstlearning.com | ||
| | | | |
peterbloem.nl
[AI summary] The pseudo-inverse is a powerful tool for solving matrix equations, especially when the inverse does not exist. It provides exact solutions when they exist and least squares solutions otherwise. If multiple solutions exist, it selects the one with the smallest norm. The pseudo-inverse can be computed using the singular value decomposition (SVD), which is numerically stable and handles cases where the matrix does not have full column rank. The SVD approach involves computing the SVD of the matrix, inverting the non-zero singular values, and then reconstructing the pseudo-inverse using the modified SVD components. This method is preferred due to its stability and ability to handle noisy data effectively. |
|
| | | | | ||
| | | | |
francisbach.com
[AI summary] This text discusses the scaling laws of optimization in machine learning, focusing on asymptotic expansions for both strongly convex and non-strongly convex cases. It covers the derivation of performance bounds using techniques like Laplace's method and the behavior of random minimizers. The text also explains the 'weird' behavior observed in certain plots, where non-strongly convex bounds become tight under specific conditions. The analysis connects theoretical results to practical considerations in optimization algorithms. |
|
| | | | | ||
| | | | |
neuralnetworksanddeeplearning.com
[AI summary] The text provides an in-depth explanation of the backpropagation algorithm in neural networks. It starts by discussing the concept of how small changes in weights propagate through the network to affect the final cost, leading to the derivation of the partial derivatives required for gradient descent. The explanation includes a heuristic argument based on tracking the perturbation of weights through the network, resulting in a chain of partial derivatives. The text also touches on the historical context of how backpropagation was discovered, emphasizing the process of simplifying complex proofs and the role of using weighted inputs (z-values) as intermediate variables to streamline the derivation. Finally, it concludes with a citation and licens... |
|
| | | | | ||
| | | | |
www.nicktasios.nl
In the Latent Diffusion Series of blog posts, I'm going through all components needed to train a latent diffusion model to generate random digits from the MNIST dataset. In the second post, we will bu |
|
| | | |||
