Optical Character Recognition WITHOUT Machine Learning

Example with MNIST

By Suvanjanprasai — Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=156115980

I decided to issue myself the challenge of doing the classic MNIST digit recognition without using any machine learning. This does skip the need to dedicate compute to training steps, but still requires training data. At a high level it works by finding the most similar looking image in the training data and then assumes that image’s class. This method has got me an 87% accuracy without using any machine learning packages, just pandas for loading the data and numpy for some basic manipulation of data.

This works relatively well for character MNIST recognition because there are few classes, lots of data, and the data is relatively homogenous (the MNIST data set was normalized before it was published). There’s only so many ways of writing a number and with a large enough data set you can find a very similar image to whatever you drew. This brings me to the drawback of it not being a time/compute efficient method. 7 years ago I was able to train a rudimentary neural network and run it over all the test data to get a 97% percent accuracy in just under *2 minutes, this method takes slightly over [2 hours](https://www.kaggle.com/code/brwalk/image-similarity?scriptVersionId=296411340)* to run.

In practice even a 97% accuracy isn’t really good enough to trust in a real world setting since when reading even a 5 digit number you’d only have an accuracy of .97⁵ (86%) so the need to double check is quite high. The true point of MNIST though, is to benchmark different ML methods, I think this basic image similarity method does offer yet another example of running a quick baseline that gives a hint about the quality of ML. In other tasks you may try to train a very explainable model (wether computer vision or not), but it may be just as time efficient for you to just do a similarity check to existing data. Though it this “model” doesn’t generalize well, this method is a much better baseline than random guesses and you should consider using this method instead of a random guess.