/explore
|
You are here 
|
www.ethanepperly.com | ||
| | | | |
blog.georgeshakan.com
|
|
| | | | | In this post, I talk about the mathematical foundations of PCA | |
| | | | |
jaketae.github.io
|
|
| | | | | In this post, we will take a look at Nyström approximation, a technique that I came across in Nyströmformer: A Nyström-based Algorithm for Approximating Self-Attention by Xiong et al. This is yet another interesting paper that seeks to make the self-attention algorithm more efficient down to linear runtime. While there are many intricacies to the Nyström method, the goal of this post is to provide a high level intuition of how the method can be used to approximate large matrices, and how this method was used in the aforementioned paper. | |
| | | | |
nickhar.wordpress.com
|
|
| | | | | 1. Low-rank approximation of matrices Let $latex {A}&fg=000000$ be an arbitrary $latex {n \times m}&fg=000000$ matrix. We assume $latex {n \leq m}&fg=000000$. We consider the problem of approximating $latex {A}&fg=000000$ by a low-rank matrix. For example, we could seek to find a rank $latex {s}&fg=000000$ matrix $latex {B}&fg=000000$ minimizing $latex { \lVert A - B... | |
| | | | |
statisticaloddsandends.wordpress.com
|
|
| | | If $latex Z_1, \dots, Z_n$ are independent $latex \text{Cauchy}(0, 1)$ variables and $latex w= (w_1, \dots, w_n)$ is a random vector independent of the $latex Z_i$'s with $latex w_i \geq 0$ for all $latex i$ and $latex w_1 + \dots w_n = 0$, it is well-known that $latex \displaystyle\sum_{i=1}^n w_i Z_i$ also has a $latex... | ||
