Machine Learning in Computer Algebra

A look at how a once thought dead-end in machine learning has transformed into promising solutions.

Cover of Artificial Intelligence, A Modern Approach (3rd Edition)

Although Artificial Intelligence has been around for decades, it wasn’t until the advent of the stochastic process of Machine Learning that allowed the field to blossom. The birth of machine learning has it found itself in almost every facet of science and engineering, delivering promising methods to problems and challenges that were once deemed difficult or near impossible to solve. One of those problems happens to be the efficiency of using machine learning in mathematical problems.

Machine learning has always opposed rule-based approaches in favor of statistical learning. Combining symbolic reasoning with statistical learning is one of the challenges symbolic computation faces in the rapidly increasing field of machine learning. The only investigations on mathematics with machine learning deal with simple arithmetic tasks, of which machine learning approaches tend to perform poorly. For some time this has led researchers to believe that mathematics is a dead-end when it comes to machine learning.

In a recent research article, however, Guillaume Lample and François Charton from Facebook AI Research attempt to bridge this gap by showing that machine learning can be used to solve some tricky mathematical problems. To display this, they have a machine learning model attempt to tackle two complex computations in mathematics: indefinite integration, and differential equations. These tasks are difficult for both humans and computers to solve; in some cases, computer algorithms can be up to 100 pages long.

As a result of trying to tackle this monstrous task, Lample and Charton were able to display the vast potential machine learning has on complex computations that even our leading commercial computational systems like Matlab and Mathematica have extreme difficulty with. Furthermore, this research can give us an insight into what applications can be achieved because of this, and what this means for future scientific research and software.

Generating Training Data

In order to take on this task of solving complex computations, Lample and Charton first needed to create training data for the machine learning model to learn from. To do this, a method to take mathematical expressions, and shape them in a way so that computers can understand them must be created.

What mathematics boils down to when dealing with computers is the representation of expressions as trees. We can think of a problem like integration or differential equations as the mapping of one tree (the problem itself) to another (the solution).

For a model to operate, however, a generated tree needs to originate from a sequence of symbols in order to preserve its mathematical meaning. Therefore, the first step in representing mathematics as a computer-readable language is to first take the expression and then convert it to a notation that the computer can read quickly and efficiently. After that, the sequence is then parsed into a tree representation, where the meaning is preserved for the computer to interpret accurately.

Once an expression can be turned into a tree for computers to read, the final step is to create training data for the model to get an idea of how solutions to our problems will look in comparison to the problem itself. It goes without saying that given the amount of mathematical operators and functions, there is a large training set. Even with using only 15 tree nodes, there are over 10⁶² different possible generations for expressions.

To make the model more efficient, dataset cleaning is required. Generated solutions are simplified to reduce unique generated expressions. Similarly, invalid expressions that are syntactically correct but make no mathematical sense, like taking the square of a negative, are discarded from the dataset. Functions that don’t have an integral or can solve a differential equation are discarded as well.

Experiments

Now that a proper training set is created, it’s time to put the machine learning model to the test and then see how it compares to the latest computer software designed to tackle such difficult problems.

In all the experiments, 256 randomly generated expressions are given for the model to hypothesize every run. When a semantically incorrect mathematical expression is given, it is simply discarded so that the model doesn’t waste any time.

At the end of every run, every hypothesis in the set is verified for correctness, since verification is rather quick. The percentage of correct hypothesis versus incorrect ones are then recorded.

The machine learning model was tested on over 5000 different and unique mathematical expressions. The results show that the model did significantly better than the three most prominent computational systems: Mathematica, Matlab, and Maple. For integration, Mathematica achieved 84% solvability, Matlab achieved 65.2% and Maple achieved 67.4%. This points to a near 10% gap with the model, which reached an upwards of 93% solvability.

In the case of differential equations, Matlab and Maple achieved less than 50%solvability for both first and second-order differential equations, whereas Mathematicaachieved 77.2% for first-order differential equations, and 61.6% for second-order differential equations. Mathematica’s performance with the latter is the only category in which a computational system was able to outperform the machine learning model, which performed at 81.2% solvability for the first category, but only 40.8% for the second category.

Conclusion

What Lample and Charton concluded in their research was an attempt in bridging the gap between machine learning and symbolic computation, by demonstrating the ability of a machine learning model to perform complex computations such as integration and differential equations with much better relative speed and accuracy compared to the most prominent commercial and enterprise computational systems out there.

What does this mean for future scientific endeavors? It should go without saying that finding a way for a computer to use machine learning to solve intense computational problems with refined accuracy and speed is the catalyst to accelerate future scientific discoveries. Imagine that a complex problem like simulating the atmospheric conditions of the earth and comparing the results to current satellite data can be computed in a manner of seconds. Or constructing a 3D model of a ship’s hull and measuring its resistance in a simulated ocean can be accomplished within your own personal computer.

A hurdle in the science of today is trying to find answers to these questions within a reasonable amount of time and a reasonable amount of computing power. Research grants and electricity bills aside, computer software can only do so much, but machine learning is able to bypass most of these hurdles with ease. Self-driving cars can be achieved by having a machine learning system be able to calculate billions of operations, such as distance and speed, in fractions of a second, faster than any software system out there today.

Even daily usages can be seen from the discovery of machine learning mathematics. Schools can implement personal computer tutors, for example, that can be modeled to the speed and intelligence of every individual student, ensuring that everyone learns at their own pace. Traffic lights can simulate and model current traffic to determine the optimal decision making at intersections, drastically reducing congestion on the roads. The possibilities that machine learning has on mathematics has no end in sight.

Overall, the bridge between machine learning and symbolic computation seems to be on the horizon, as we find great success in seeing the raw power in machine learning models solving complex computations. Hopefully, the work done by Lample and Charton may pave the way for new discoveries in symbolic computation as the research done behind machine learning only gets more fruitful with time.