Why Neural Networks Discover Fourier: The Jacobian Makes It Almost Inevitable

The Puzzle