Make a Neural Network Infinitely Wide and It Stops Being a Neural Network.

Everything mysterious about deep learning — the non-convex landscape, the strange training dynamics, the sense that nobody quite knows why…