A Lite BERT, Built from Scratch: The Two Ideas That Shrink the Model Without Shrinking Performance

What if you could get BERT-level performance without carrying around hundreds of millions of parameters?