As the backward algorithm of SVD is prone to have numerical instability, we implement a variety of end-to-end SVD methods by manipulating the backward algortihms in this repository. They include:
- SVD-Pad'e: use Pad'e approximants to closely approximate the gradient. 
- SVD-Taylor: use Taylor polynomial to approximate the smooth gradient. 
- SVD-PI: use Power Iteration (PI) to approximate the gradients. 
- SVD-Newton: use the gradient of the Newton-Schulz iteration.
- SVD-Trunc: set a upper limit of the gradient and apply truncation.
- SVD-TopN: select the Top-N eigenvalues and abandon the rest.
- SVD-Original: ordinary SVD with gradient overflow check.