/explore
|
You are here 
|
dustintran.com | ||
| | | | |
xcorr.net
|
|
| | | | | Earlier, I discussed how I had no luck using second-order optimization methods on a convolutional neural net fitting problem, and some of the reasons why stochastic gradient descent works well on this class of problems. Stochastic gradient descent is not a plug-and-play optimization algorithm; it requires messing around with the step size hyperparameter, forcing you... | |
| | | | |
fa.bianp.net
|
|
| | | | | MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['$','$'], ["\\(","\\)"] ], displayMath: [ ['$$','$$'], ["\\[","\\]"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, Macros: { RR: "{\\mathbb{R}}", argmin: "{\\mathop{\\mathrm{arg\\,min}}}", bold: ["{\\bf #1}",1] } }, "HTML-CSS": { availableFonts: ["TeX"] } }); TL;DR: I describe a method for hyperparameter optimization by gradient descent. Most machine ... | |
| | | | |
francisbach.com
|
|
| | | | | [AI summary] This text discusses the scaling laws of optimization in machine learning, focusing on asymptotic expansions for both strongly convex and non-strongly convex cases. It covers the derivation of performance bounds using techniques like Laplace's method and the behavior of random minimizers. The text also explains the 'weird' behavior observed in certain plots, where non-strongly convex bounds become tight under specific conditions. The analysis connects theoretical results to practical considerations in optimization algorithms. | |
| | | | |
angusturner.github.io
|
|
| | | Machine Learning and Data Science. | ||
