How to Choose Batch Size When Fine-Tuning Large Models

Different studies have shown that there is a “critical threshold” for the amount of data processed per training step. By finding this…