MaximaST: A Computer Algebra System Kernel Rewritten in Rust

What happens when you ask AI to vibe-code a 24,000-line computer algebra system kernel in Rust — with full Maxima syntax compatibility?

I recently developed a fascinating open-source project that feels like a glimpse into the future of software engineering: **MaximaST**, a computer algebra system (CAS) kernel rewritten in Rust, with syntax fully compatible with the venerable Maxima CAS. What makes it extraordinary isn’t just the technical ambition — it’s the methodology. The entire ~24,000-line codebase was built through an AI-driven, human-in-the-loop development process across roughly 90 pull requests, organized into 5 structured sprints. Since it is a totally re-written, it is Apache 2.0/MIT lic comparing to the original GPL license.

In this post, I’ll walk through what MaximaST is, how it’s architected, what it can do, and why its development approach tells us something important about where software engineering is headed.

What Is MaximaST?

MaximaST is a from-scratch reimplementation of a Maxima-compatible CAS kernel in Rust. If you’ve ever used Maxima, Mathematica, Maple, or SymPy, you’re familiar with the genre: symbolic math software that manipulates equations, computes integrals, solves differential equations, and handles matrix operations — all while keeping expressions in exact symbolic form (so 1/3 + 1/6 evaluates to 1/2, not 0.5).

Maxima itself has roots going back to the 1960s (it originated as Macsyma at MIT). It’s powerful, battle-tested, and free — but it’s built on a legacy Lisp codebase that can be challenging to extend or embed in modern applications. MaximaST asks: what if we rebuilt this from the ground up in Rust?

The result is a fast, memory-safe CAS kernel with:

• Full Maxima syntax compatibility — existing .mac scripts work out of the box
• ~24,000 lines of Rust across a clean, modular crate architecture
• 920+ integration tests and 29 interactive walkthrough tutorials
• A REPL with tab completion, syntax highlighting, and multi-line input
• Dual licensing (MIT/Apache-2.0)

And it’s all the product of an experimental “auto-research” development process.

Project Architecture: Five Crates, Clean Separation

The project uses a Cargo workspace with five purpose-built crates:

maximast/
├── crates/
│   ├── core/       # Expr enum, operators, symbol interning — the DNA
│   ├── parser/     # Lexer + Pratt parser — turns text into AST
│   ├── poly/       # Polynomial ring operations, GCD, factoring
│   ├── eval/       # The brain: evaluator, ~90% of the codebase
│   └── repl/       # Interactive shell with tab completion
├── walkthrough/    # 29 .mac tutorial scripts
├── research/       # Auto-generated technical reports and surveys
└── sprint/         # Design documents from 5 development sprints

crates/core: The Foundation

This is the immutable(ish) bedrock. It defines:

• **Expr** — the core expression enum representing every mathematical object (integers, rationals, symbols, function calls, lists, matrices, sets)
• **Operator** — all binary and unary operators
• Symbol interning — efficient string storage so symbol comparison is cheap

The core crate rarely changes. It's the contract everything else builds on.

crates/parser: From String to Tree

The parser uses a Pratt parser (top-down operator precedence), which is both compact and powerful enough to handle Maxima’s expression syntax naturally. The pipeline is:

Text → Lexer (tokens) → Parser (AST as Expr) → Evaluator (result)

crates/poly: Polynomial Arithmetic

A self-contained polynomial library handling:

• Polynomial GCD (Euclidean algorithm)
• Square-free factorization
• Polynomial division and remainder

This crate is “math library” territory — it doesn’t know about the rest of the system and could theoretically be published as a standalone crate.

crates/eval: Where the Magic Happens

This is the heart of the system, containing 25+ specialized modules:

ModulePurposeeval.rsCore dispatch engine (~7K lines, ~200 function match arms)integrate.rsMain integration engine (pattern matching + heuristics)risch_integrate.rsRisch algorithm for elementary function integrationsimp.rsAlgebraic simplifier (canonical forms)gruntz.rsLimits via the Gruntz algorithmlaplace.rsLaplace transforms and inverse transformsode.rsOrdinary differential equation solverseries.rsTaylor series expansionzeilberger.rsGosper-Zeilberger for closed-form summationpattern.rsPattern matching and rule-based transformationsets.rsSet theory operationsnumtheory.rsNumber theory (primes, totient, CRT, Jacobi)complex.rsComplex number arithmeticbigfloat.rsArbitrary-precision floating pointtex.rsLaTeX output generationplot.rs2D plotting via SVG and gnuplotstrings.rsString manipulation functionsmatrix opsDeterminant, inverse, eigenvalues, arithmetic

The architecture follows a simple but effective pattern: eval.rs contains a giant match on function names, and dispatches to specialized modules. Adding a new function means implementing it in the right module and adding one arm to the match.

crates/repl: The User Interface

Built on the excellent [rustyline](https://github.com/kkawakam/rustyline) library, the REPL provides:

• Tab completion for all built-in functions (150+ entries)
• Command history with arrow-key navigation
• Syntax-highlighted output
• Multi-line input (expressions accumulate until terminated with ;)
• Batch mode for running .mac script files
• Quiet modes for piping and scripting

A Guided Tour: What MaximaST Can Do

The project’s walkthrough/ directory contains 29 .mac scripts that serve as both tutorials and integration tests. Here's a curated highlight reel.

1. Exact Arithmetic That Stays Exact

(%i1) 1/3 + 1/6;
(%o1) 1/2

(%i2) 100!;
(%o2) 9332621544394415268169923885626670049071596826438...
    ...161459512975165328980000000000

(%i3) 2^100;
(%o3) 1267650600228229401496703205376

MaximaST uses arbitrary-precision integers via Rust’s num-bigint, so 100! and 2^100 compute instantly without overflow. Rational arithmetic stays in rational form — no premature floating-point conversion.

2. Symbolic Algebra

(%i4) factor(x^6 - 1);
(%o4) (x - 1)*(x + 1)*(x^2 + x + 1)*(x^2 - x + 1)

(%i5) expand((a + b + c)^3);
(%o5) c^3 + 3*b*c^2 + 3*a*c^2 + 3*b^2*c + 6*a*b*c
    + 3*a^2*c + b^3 + 3*a*b^2 + 3*a^2*b + a^3

(%i6) partfrac(1/(x^3 - x), x);
(%o6) 1/(2*(x - 1)) - 1/x + 1/(2*(x + 1))

The factor, expand, ratsimp, and partfrac functions give you full control over expression structure. Partial fraction decomposition handles irreducible quadratics as well.

3. Calculus: Differentiation and Integration

Differentiation is straightforward:

(%i7) diff(x^x, x);
(%o7) x^x*(log(x) + 1)

(%i8) diff(sin(x^2), x);
(%o8) 2*x*cos(x^2)

Integration is where things get interesting. MaximaST handles multiple integration strategies:

(%i9) integrate(x * exp(x), x);
(%o9) (x - 1)*exp(x)

(%i10) integrate(exp(x) * sin(x), x);
(%o10) exp(x)*(sin(x) - cos(x))/2

(%i11) integrate(1/(x^4 + 1), x);
/* Full partial fraction decomposition with radicals */

The integration engine combines:

• Direct pattern matching for standard forms (polynomials, exponentials, trig)
• Integration by parts (automatic when beneficial)
• Substitution detection (e.g., u = x^2 in x*exp(x^2))
• Partial fractions for rational functions
• The Risch algorithm for elementary function integration

Definite integrals work too, including improper integrals:

(%i12) integrate(exp(-x), x, 0, inf);
(%o12) 1

(%i13) integrate(x^3 * exp(-x), x, 0, inf);
(%o13) 6

(%i14) integrate(1/(x^2 + 1), x, minf, inf);
(%o14) %pi

4. Limits via the Gruntz Algorithm

(%i15) limit((x^2 - 1)/(x - 1), x, 1);
(%o15) 2

(%i16) limit(sin(x)/x, x, 0);
(%o16) 1

(%i17) limit(exp(x)/x^5, x, inf);
(%o17) inf

The gruntz.rs module implements a variant of the Gruntz algorithm for computing limits, handling indeterminate forms and asymptotic comparison of growth rates.

5. Taylor Series

(%i18) taylor(sin(x), x, 0, 7);
(%o18) x - x^3/6 + x^5/120 - x^7/5040 + ...

(%i19) taylor(exp(x), x, 0, 5);
(%o19) 1 + x + x^2/2 + x^3/6 + x^4/24 + x^5/120 + ...

6. Linear Algebra

(%i20) A: matrix([1, 2], [3, 4]);
(%o20) matrix([1, 2], [3, 4])

(%i21) determinant(A);
(%o21) -2

(%i22) invert(A);
(%o22) matrix([-2, 1], [3/2, -1/2])

(%i23) eigenvalues(A);
(%o23) [[(sqrt(33) - 5)/2, -(sqrt(33) + 5)/2], [1, 1]]

Matrices support arithmetic, determinants, inverses, characteristic polynomials, eigenvalues, and eigenvectors.

7. Laplace Transforms

(%i24) laplace(1, t, s);
(%o24) 1/s

(%i25) laplace(t^3, t, s);
(%o25) 6/s^4

(%i26) laplace(sin(w*t), t, s);
(%o26) w/(s^2 + w^2)

(%i27) ilt(s/(s^2 + 4), s, t);
(%o27) cos(2*t)

The Laplace transform module handles linearity, the shift theorem, and inverse transforms — useful for solving differential equations in engineering contexts.

8. Ordinary Differential Equations

(%i28) ode2('diff(y,x) = x*y, y, x);
(%o28) y = %c*exp(x^2/2)

(%i29) ode2('diff(y,x,2) + y = 0, y, x);
(%o29) y = %k1*sin(x) + %k2*cos(x)

(%i30) ode2('diff(y,x,2) + 4*'diff(y,x) + 4*y = 0, y, x);
(%o30) y = (%k2*x + %k1)*exp(-2*x)

The ode2 function handles first-order separable and linear equations, plus second-order constant-coefficient equations with distinct real, repeated real, and complex conjugate roots.

9. Programming in MaximaST

MaximaST includes a full programming language with functions, blocks, loops, conditionals, and lambda expressions:

/* Define a function */
f(x) := x^2 + 3*x + 1;

/* Block with local variables */
block([sum: 0],
  for i:1 thru 10 do sum: sum + i,
  sum
);

/* Recursive factorial */
fact(n) := if n <= 1 then 1 else n * fact(n - 1);

/* Lambda (anonymous function) */
lambda([x], x^2 + 1)(5);

There’s even a 24-game solver in the walkthroughs demonstrating recursive search with backtracking

10. LaTeX Output

(%i31) tex(integrate(x^2*sin(x), x));
$$-{{2\,\sin x-x^2\,\sin x-2\,x\,\cos x}\over{1}}$$

The tex() function renders any expression as LaTeX, making it easy to embed results in academic papers.

Running MaximaST

Getting started is trivial if you have Rust installed:

# Clone and build
git clone https://github.com/yfyang86/maximast.git
cd maximast
cargo build --release

# Start the REPL
cargo run

# Evaluate a single expression
cargo run -- -e "integrate(1/(x^4+1), x);"

# Run a walkthrough script
cargo run -- -b walkthrough/03_calculus.mac

The REPL greets you with a clean banner and the familiar (%i1) / (%o1) prompt style that Maxima users know:

╔══════════════════════════════════════════════════╗
║  MaximaST   v5.0.0                               ║
║  A Computer Algebra System                       ║
╚══════════════════════════════════════════════════╝

(%i1) factor(x^6 - 1);
(%o1) (x - 1)*(x + 1)*(x^2 + x + 1)*(x^2 - x + 1)

(%i2) integrate(exp(x)*sin(x), x);
(%o2) exp(x)*(sin(x) - cos(x))/2

The Auto-Research Methodology

What truly distinguishes MaximaST is its development process. The project is explicitly described as an “AI-driven, human-in-the-loop auto-research and vibe-coding project.”

The repository contains several unusual artifacts:

CLAUDE.md: The Development Constitution

Inspired by Karpathy-style rules of thumb, this document establishes development principles:

1. Think Before Coding — state assumptions explicitly, surface tradeoffs, push back when warranted
2. Simplicity First — minimum code that solves the problem, no speculative abstractions
3. Surgical Changes — touch only what you must, match existing style
4. Goal-Driven Execution — define success criteria, loop until verified

rules.md: 379 Lines of Hard-Won Lessons

This is essentially a field manual for LLM agents working on the codebase. It documents:

• The exact pattern for adding a new function (choose module → implement → wire into eval.rs → add tests → register in REPL)
• How the evaluator’s giant match statement works
• Debugging strategies for “returns noun form” bugs
• Simplification layer interactions (evaluator → simplifier → integrator normalization)
• Performance tuning guidance (don’t parallelize — the Environment is single-threaded by design)

skills.md: Reusable Recipes

Step-by-step procedures for common tasks:

• Adding a built-in function
• Adding new syntax/operators
• Fixing simplification bugs
• Fixing wrong formulas (with numerical verification via Python/scipy)
• Refactoring large files
• Adding walkthrough tutorials

research/: Auto-Generated Technical Documentation

The research/ directory contains:

• Technical manuscript — auto-generated technical report on the system
• Algorithm survey — literature review of CAS algorithms
• **integralformulalist.toml** — a structured database of integral formulas used for testing

sprint/: 5 Sprints of Development

The sprint directory contains design documents from each development phase:

• Sprint v1.0: Initial kernel — core arithmetic, algebra, basic calculus
• Sprint v2.0: Integration engine — pattern matching, partial fractions, Risch
• Sprint v3.0: Analysis tools — limits, series, summation, assumptions
• Sprint v4.0: Linear algebra, complex numbers, special functions
• Sprint v5.0: Sets, strings, number theory, Laplace transforms, ODEs, pattern matching, bfloat, plotting

Each sprint document was AI-generated and used as-is without manual editing — a fascinating artifact of what “vibe coding” at scale looks like.

What This Tells Us About the Future

MaximaST is explicitly experimental and the README warns that “many edge cases remain untested.” But even as a proof of concept, it’s remarkable.

Consider what was accomplished:

• ~24,000 lines of working Rust code implementing a nontrivial domain (computer algebra)
• 920+ passing tests covering core functionality
• 29 tutorial walkthroughs demonstrating real capabilities
• 5 major feature sprints completed through human-AI collaboration
• Clean architecture with proper separation of concerns

This isn’t a toy. It’s a functioning CAS kernel that can differentiate, integrate, solve ODEs, compute limits, manipulate matrices, and transform Laplace — all while maintaining exact symbolic arithmetic.

The project’s development artifacts (CLAUDE.md, rules.md, skills.md) represent something new: explicit, codified development practices for human-AI collaboration. Rather than treating AI as a fancy autocomplete, MaximaST treats it as a development partner with defined roles, constraints, and quality gates. The rules about "surgical changes," "verify numerically first," and "don't improve adjacent code" are exactly the kind of discipline that makes large-scale AI-assisted development viable.

For the Rust ecosystem specifically, MaximaST demonstrates that the language’s algebraic data types, pattern matching, and ownership model are excellent fits for symbolic computation. The Expr enum naturally represents the recursive structure of mathematical expressions, and Rust's performance characteristics mean this kernel could eventually outpace its Lisp ancestor.

Should You Use It?

As the README candidly states: “This is highly unstable and intended for experimentation only. Use at your own risk.”

If you need a production CAS today, use Maxima, SymPy, or Mathematica. They’re mature, well-tested, and documented.

But if you’re interested in:

• Rust-based scientific computing — the crate architecture is instructive
• AI-assisted software development at scale — the methodology documents are a goldmine
• CAS implementation — the integration of algorithms (Risch, Gruntz, Gosper-Zeilberger) is well-organized
• Programming language implementation in Rust — the parser, evaluator, and REPL are clean examples

Then MaximaST is absolutely worth studying. Clone it, run the walkthroughs, read the sprint documents, and trace how a symbolic math system actually works.

Getting Involved

The project welcomes contributions. The maintainers have planned a larger test suite of 10,000+ cases based on known integral tables and textbook problems. If symbolic computation, Rust, or AI-assisted development interests you, there’s real work to be done.

The contribution path is well-defined thanks to skills.md: pick a function, implement it following the established pattern, add tests, and submit. The project's own documentation makes it unusually approachable for newcomers.

Final Thoughts

MaximaST sits at an interesting intersection: it’s a legitimate technical achievement (a working CAS kernel in Rust) built through an experimental process (AI-driven development) that itself produces interesting artifacts (codified AI development rules, auto-research documents).

Whether or not it becomes a production-worthy alternative to Maxima, it demonstrates three things concretely:

1. Rust is a viable language for symbolic computation, with expressive type system features that map naturally to mathematical structures.
2. AI-assisted development can produce substantial, well-architected codebases when guided by clear principles and quality gates.
3. The methodology matters as much as the code — CLAUDE.md and rules.md are arguably as important as any source file, because they make the development process reproducible and extensible.

If you want to see what the future of software engineering might look like, clone the repo and run a few walkthroughs. The math is fun, but the meta-story is fascinating.

Quick Reference

ResourceLocationSource codecrates/Interactive tutorialswalkthrough/ (29 .mac files)Development principlesCLAUDE.mdCoding rules for AI agentsrules.mdHow-to recipesskills.mdUser manualuser-manual.mdTechnical reportsresearch/manuscript/, research/survey/Sprint design docssprint/Test suitecrates/eval/tests/ (14 files, 920+ tests)

License: MIT OR Apache-2.0

Have thoughts on AI-driven development or CAS implementation? Drop a comment below — I’d love to discuss where this approach could go next.