Generating Question Solving Programs for Mathematics

A Neural Network Solves and Generates Mathematics Problems by Program Synthesis: Calculus, Differential Equations, Linear Algebra, and More

We demonstrate that a neural network pre-trained on text and fine-tuned on code solves mathematics problems by program synthesis.We turn questions into programming tasks, automatically generate programs, and then execute them, perfectly solving university-level problems from mit's large mathematics courses (single variable calculus 18.01, multivariable calculus 18.02, differential equations 18.03, introduction to probability and statistics 18.05, and linear algebra 18.06), columbiauniversity s coms3251 computational linear algebra course, as well as questions from a math dataset (on prealgebra, algebra, counting and probability, number theory, and precalculus), the latest benchmark of advanced mathematics problems specifically designed to assess mathematical reasoning.We explore prompt generation methods that enable transformers to generate question solving programs for these subjects, including solutions with plots.We generate correct answers for a random sample of questions in each topic and quantify the gap between the original and transformed questions and perform a survey to evaluate the quality and difficulty of generated questions.