DSL in Rust — Part 6

Grammar-Driven Parsing: pest and ANTLR

Parser combinators let you build parsers compositionally in code. You define small parsing functions and combine them into larger ones. The result lives in your Rust files, compiled alongside your business logic, debugged with your standard tools. This approach works well, but it has a limitation: the grammar exists implicitly in code structure rather than explicitly in a readable specification.

Grammar-driven parsing flips this relationship. Instead of encoding syntax rules in function composition, you write a declarative grammar specification in its own file. A tool reads this specification and generates the parser for you. The grammar becomes documentation. A colleague can open your .pest or .g4 file and understand the language’s syntax without tracing through combinator chains.

This separation of concerns brings practical benefits. Grammars are easier to review because they express intent directly. Tools can analyze grammars for ambiguity, unreachable rules, or conflicts. IDEs can provide syntax highlighting and validation for grammar files. Error messages improve because the parser generator understands the grammar’s structure and can report what it expected at any given position.

The tradeoff is control. With combinators, you decide exactly how parsing proceeds, how errors propagate, and how results accumulate. With grammar-driven tools, you delegate these decisions to the generator. The abstraction handles common cases well but may fight you on unusual requirements.

This article covers two grammar-driven approaches: pest, which brings Parsing Expression Grammars to Rust, and ANTLR, an industrial-strength parser generator used across the software industry for decades. Understanding both helps you choose the right tool for your DSL’s complexity and your project’s constraints.

What we’ll build

By the end of this article, we’ll have working parsers for a small expression language using both pest and ANTLR. You’ll understand the key differences between PEG (Parsing Expression Grammar) semantics and traditional LL parsing, and you’ll know when each tool fits your needs. We’ll also implement a configuration file parser that demonstrates real-world patterns in both tools.

pest: PEG for Rust

Understanding PEG semantics

Parsing Expression Grammars differ from the context-free grammars you may have learned in compiler courses. The fundamental difference lies in how alternatives work. In a CFG, the grammar A → B | C means “A can be either B or C,” and the parser figures out which one applies based on the input. If the grammar is ambiguous, this choice might not be unique.

PEGs eliminate ambiguity through ordered choice. The rule A = { B | C } means “try B first; if B succeeds, use that result; only try C if B fails.” First match wins, always. This makes PEG parsers deterministic by construction. You cannot write an ambiguous PEG grammar because the semantics define exactly which alternative applies.

This determinism comes with tradeoffs. Repetition operators like * and + are greedy: they consume as much input as possible. If you write identifier = { ASCII_ALPHA+ } followed by keyword = { "if" }, the identifier rule will consume “if” as part of any longer identifier, potentially causing the keyword rule to never match. You must order your rules carefully and use negative lookahead to prevent unintended consumption.

PEGs also support lookahead without consumption. The & operator succeeds if its expression matches but doesn’t consume any input. The ! operator succeeds if its expression fails to match, also without consuming input. These primitives let you peek ahead to decide which parsing path to take.

Setup

Add pest to your project’s dependencies:

[dependencies]
pest = "2"
pest_derive = "2"

The pest crate provides the runtime, while pest_derive provides the procedural macro that compiles your grammar into Rust code at build time.

Your first grammar

Create a file src/grammar.pest with the following content:

// Implicit whitespace handling
WHITESPACE = _{ " " | "\t" | "\n" | "\r" }
COMMENT    = _{ "//" ~ (!"\n" ~ ANY)* }

// Entry point
program = { SOI ~ statement* ~ EOI }

// Statements
statement = { assignment | expression }

assignment = { identifier ~ "=" ~ expression ~ ";" }

// Expressions with precedence (lowest to highest)
expression = { term ~ ((add | subtract) ~ term)* }
term       = { factor ~ ((multiply | divide) ~ factor)* }
factor     = { number | identifier | "(" ~ expression ~ ")" }

// Operators
add      = { "+" }
subtract = { "-" }
multiply = { "*" }
divide   = { "/" }

// Primitives
identifier = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "_")* }
number     = @{ ASCII_DIGIT+ }

This grammar defines a simple expression language with assignments. Let’s walk through what each part means.

The WHITESPACE rule uses the underscore prefix = _, making it a silent rule. Silent rules match input but don’t create nodes in the parse tree. pest automatically inserts WHITESPACE matching between elements of non-atomic rules. This means you don’t need to explicitly handle spaces in rules like assignment: pest handles them for you.

The program rule starts with SOI (Start Of Input) and ends with EOI (End Of Input). These anchors ensure the parser consumes all input and fails if anything remains unparsed.

Expression precedence requires factoring the grammar. The expression rule handles addition and subtraction, the lowest precedence operations. It delegates to term for multiplication and division, which are higher precedence. Finally, factor handles the highest precedence: literals, identifiers, and parenthesized expressions. This structure ensures that 1 + 2 * 3 parses as 1 + (2 * 3) rather than (1 + 2) * 3.

The @ prefix on identifier and number marks them as atomic rules. Atomic rules don’t allow implicit whitespace between their components, and they report their contents as a single token rather than exposing internal structure. This prevents my identifier from matching as an identifier with embedded whitespace.

Grammar syntax reference

pest’s grammar syntax is concise but expressive:

+--------+-------------------------------------------------------------------------+
| Syntax | Meaning                                                                 |
+========+=========================================================================+
| =      | Regular rule (creates pair in AST)                                      |
| = _    | Silent rule (matches but doesn’t create AST node)                       |
| = @    | Atomic rule (no whitespace handling, children reported as single token) |
| = $    | Compound atomic (atomic but children still parsed)                      |
| ~      | Sequence                                                                |
| |      | Ordered choice                                                          |
| *      | Zero or more                                                            |
| +      | One or more                                                             |
| ?      | Optional                                                                |
| !      | Negative lookahead                                                      |
| &      | Positive lookahead                                                      |
+--------+-------------------------------------------------------------------------+

Connecting the grammar to Rust

pest uses a derive macro to compile the grammar into Rust code:

use pest::Parser;
use pest_derive::Parser;

#[derive(Parser)]
#[grammar = "grammar.pest"]
pub struct DslParser;

fn main() {
    let input = "x = 1 + 2 * 3;";

    let pairs = DslParser::parse(Rule::program, input)
        .expect("parse failed");

    for pair in pairs {
        print_pair(pair, 0);
    }
}

fn print_pair(pair: pest::iterators::Pair<Rule>, indent: usize) {
    let span = pair.as_span();
    println!(
        "{:indent$}{:?}: {:?}",
        "",
        pair.as_rule(),
        span.as_str(),
        indent = indent
    );

    for inner in pair.into_inner() {
        print_pair(inner, indent + 2);
    }
}

The #[grammar = "grammar.pest"] attribute tells pest where to find your grammar file relative to src/. The derive macro generates a Rule enum with a variant for each rule in your grammar and implements the Parser trait for DslParser.

Calling DslParser::parse(Rule::program, input) returns an iterator over Pair objects. Each pair contains the matched rule, the span of input it consumed, and access to its child pairs. The print_pair function recursively displays this structure, revealing exactly how pest parsed your input.

Building an AST from the parse tree

pest gives you a parse tree, not an AST. The parse tree mirrors your grammar’s structure exactly, with nodes for every rule that matched. For most DSLs, you’ll want to transform this into a cleaner AST that represents your domain concepts.

Define your AST types:

#[derive(Debug)]
enum Expr {
    Number(i64),
    Ident(String),
    BinOp(Box<Expr>, Op, Box<Expr>),
}

#[derive(Debug)]
enum Op { Add, Sub, Mul, Div }

#[derive(Debug)]
struct Assignment {
    name: String,
    value: Expr,
}

Then write a recursive function that walks the parse tree and constructs AST nodes:

fn build_expr(pair: Pair<Rule>) -> Expr {
    match pair.as_rule() {
        Rule::expression => {
            let mut inner = pair.into_inner();
            let mut left = build_expr(inner.next().unwrap());

            while let Some(op_pair) = inner.next() {
                let op = match op_pair.as_rule() {
                    Rule::add => Op::Add,
                    Rule::subtract => Op::Sub,
                    _ => unreachable!(),
                };
                let right = build_expr(inner.next().unwrap());
                left = Expr::BinOp(Box::new(left), op, Box::new(right));
            }
            left
        }
        Rule::term => {
            let mut inner = pair.into_inner();
            let mut left = build_expr(inner.next().unwrap());

            while let Some(op_pair) = inner.next() {
                let op = match op_pair.as_rule() {
                    Rule::multiply => Op::Mul,
                    Rule::divide => Op::Div,
                    _ => unreachable!(),
                };
                let right = build_expr(inner.next().unwrap());
                left = Expr::BinOp(Box::new(left), op, Box::new(right));
            }
            left
        }
        Rule::factor => {
            let inner = pair.into_inner().next().unwrap();
            build_expr(inner)
        }
        Rule::number => {
            Expr::Number(pair.as_str().parse().unwrap())
        }
        Rule::identifier => {
            Expr::Ident(pair.as_str().to_string())
        }
        _ => unreachable!("unexpected rule: {:?}", pair.as_rule()),
    }
}

The pattern here is straightforward: match on the rule type, extract child pairs with into_inner(), and recursively process them. For binary operators, the grammar’s structure means children alternate between operands and operators, so we consume them in pairs.

Error messages

pest provides decent error messages automatically. When parsing fails, the error includes the position in the input and what the parser expected:

fn parse_or_report(input: &str) {
    match DslParser::parse(Rule::program, input) {
        Ok(pairs) => { /* process */ }
        Err(e) => {
            eprintln!("{}", e);
            // Prints something like:
            //  --> 1:5
            //   |
            // 1 | x = + 3;
            //   |     ^---
            //   |
            //   = expected term
        }
    }
}

The error message points to line 1, column 5, shows the problematic line with a caret, and explains what was expected. This is far better than the generic “parse failed” you might get from a hand-rolled parser. For users of your DSL, good error messages are often the difference between productive debugging and frustrated abandonment.

Best practices

Writing effective pest grammars requires attention to a few patterns.

First, use atomic rules for tokens. An identifier should be atomic so that whitespace between characters causes a parse failure rather than silent acceptance:

// Good: identifier is atomic, no internal whitespace
identifier = @{ ASCII_ALPHA ~ ASCII_ALPHANUMERIC* }

// Bad: allows "my identifier" to match
identifier = { ASCII_ALPHA ~ ASCII_ALPHANUMERIC* }

Second, define WHITESPACE for implicit handling. The underscore makes it silent and automatic:

// The underscore makes it silent and automatic
WHITESPACE = _{ " " | "\t" | "\n" }

Third, use negative lookahead to prevent keyword conflicts. Without lookahead, a rule matching identifiers might consume keywords:

// Prevent "ifx" from matching "if" + "x"
keyword_if = { "if" ~ !ASCII_ALPHANUMERIC }
identifier = @{ !keyword ~ ASCII_ALPHA ~ ASCII_ALPHANUMERIC* }
keyword    = _{ "if" | "else" | "while" | "for" }

The !keyword lookahead ensures identifiers don’t start with reserved words. The !ASCII_ALPHANUMERIC after “if” ensures the keyword ends at a word boundary.

ANTLR for Rust

When to choose ANTLR

ANTLR (ANother Tool for Language Recognition) has been the go-to parser generator for complex languages since the 1990s. Major projects including the Kotlin compiler, Twitter’s search query parser, and numerous SQL dialects use ANTLR-generated parsers. The tool generates parsers in multiple languages from a single grammar specification, includes sophisticated error recovery, and handles left-recursive grammars naturally.

Choose ANTLR when your grammar’s complexity exceeds what PEGs handle comfortably. If you find yourself fighting pest’s ordered choice semantics or manually eliminating left recursion, ANTLR might be the better fit. ANTLR also makes sense when your project spans multiple programming languages: write the grammar once, generate parsers for Rust, Java, Python, and others.

The downside is toolchain complexity. ANTLR requires Java to run the grammar compiler, adds build-time dependencies, and the Rust runtime is less mature than the Java or C# runtimes. For simple to medium grammars, pest’s native Rust integration is usually smoother.

Setup

Install ANTLR4 on your system:

# macOS
brew install antlr4

# Or download directly
curl -O https://www.antlr.org/download/antlr-4.13.1-complete.jar

Add the Rust runtime to your project:

[dependencies]
antlr-rust = "0.3"

[build-dependencies]
antlr-rust = "0.3"

Writing an ANTLR grammar

Create src/Dsl.g4:

grammar Dsl;

// Parser rules (lowercase)
program: statement* EOF;

statement
    : assignment
    | expressionStmt
    ;

assignment: IDENTIFIER '=' expression ';';
expressionStmt: expression ';';

expression
    : expression ('*' | '/') expression  # MulDiv
    | expression ('+' | '-') expression  # AddSub
    | '(' expression ')'                 # Parens
    | NUMBER                             # Number
    | IDENTIFIER                         # Ident
    ;

// Lexer rules (UPPERCASE)
IDENTIFIER: [a-zA-Z_][a-zA-Z0-9_]*;
NUMBER: [0-9]+;
WS: [ \t\r\n]+ -> skip;
COMMENT: '//' ~[\r\n]* -> skip;

Several things distinguish this grammar from the pest version.

ANTLR separates lexer rules (UPPERCASE names) from parser rules (lowercase names). The lexer runs first, tokenizing the input stream. The parser then works with tokens rather than raw characters. This two-phase approach mirrors traditional compiler architecture.

The expression rule is left-recursive: expression ('*' | '/') expression refers to expression on both sides. ANTLR handles this automatically, using the rule order to determine precedence. Earlier alternatives bind tighter, so multiplication and division (listed first) have higher precedence than addition and subtraction.

The # MulDiv labels after alternatives generate distinct visitor methods. Instead of one visitExpression method, you get visitMulDiv, visitAddSub, and so on. This makes AST construction cleaner.

Structural differences from pest

The fundamental parsing algorithms differ between pest and ANTLR:

PEG parsers try alternatives sequentially, backtracking on failure. ANTLR uses lookahead to predict which alternative will succeed before committing. This prediction enables ANTLR to detect ambiguities in your grammar and report them, rather than silently picking the first match.

+------------------+-----------------------------+----------------------------------+
| Feature          | pest (PEG)                  | ANTLR (LL)                       |
+==================+=============================+==================================+
| Ambiguity        | Impossible (ordered choice) | Detected and reported            |
| Left recursion   | Not supported               | Supported                        |
| Lookahead        | Manual (&, !)               | Automatic (LL(*))                |
| Error recovery   | Basic                       | Sophisticated                    |
| Target languages | Rust only                   | Many (Java, C#, Python, Rust…)   |
+------------------+-----------------------------+----------------------------------+

Generating the Rust parser

Create a build.rs file to invoke ANTLR during compilation:

use std::process::Command;

fn main() {
    println!("cargo:rerun-if-changed=src/Dsl.g4");

    let status = Command::new("antlr4")
        .args(&[
            "-Dlanguage=Rust",
            "-visitor",
            "-o", "src/parser",
            "src/Dsl.g4"
        ])
        .status()
        .expect("Failed to run ANTLR");

    if !status.success() {
        panic!("ANTLR failed");
    }
}

The cargo:rerun-if-changed directive tells Cargo to re-run the build script only when the grammar changes. ANTLR generates several Rust files in src/parser/: a lexer, a parser, and visitor/listener interfaces.

Using the generated parser

mod parser;

use antlr_rust::common_token_stream::CommonTokenStream;
use antlr_rust::tree::ParseTreeVisitor;
use antlr_rust::InputStream;
use parser::dsllexer::DslLexer;
use parser::dslparser::DslParser;

fn main() {
    let input = "x = 1 + 2 * 3;";
    let lexer = DslLexer::new(InputStream::new(input));
    let token_stream = CommonTokenStream::new(lexer);
    let mut parser = DslParser::new(token_stream);

    let tree = parser.program().expect("parse failed");

    // Use visitor pattern
    let mut visitor = MyVisitor::new();
    visitor.visit(&*tree);
}

The parsing pipeline has more explicit stages than pest. You create an input stream, feed it to the lexer to produce tokens, wrap the tokens in a CommonTokenStream for the parser to consume, and finally call the entry point rule. This verbosity reflects the underlying architecture: lexing and parsing are genuinely separate phases.

The visitor pattern

ANTLR generates visitor interfaces based on your grammar’s rules and labels:

use parser::dslvisitor::DslVisitor;

struct AstBuilder {
    // accumulator state
}

impl<'i> DslVisitor<'i> for AstBuilder {
    fn visit_assignment(&mut self, ctx: &AssignmentContext<'i>) {
        let name = ctx.IDENTIFIER().unwrap().get_text();
        let value = self.visit_expression(ctx.expression().unwrap());
        // build AST node
    }

    fn visit_addSub(&mut self, ctx: &AddSubContext<'i>) -> Expr {
        let left = self.visit(ctx.expression(0).unwrap());
        let right = self.visit(ctx.expression(1).unwrap());
        let op = if ctx.ADD().is_some() { Op::Add } else { Op::Sub };
        Expr::BinOp(Box::new(left), op, Box::new(right))
    }
}

The context objects provide typed access to the matched components. ctx.IDENTIFIER() returns the identifier token if present. ctx.expression(0) returns the first expression child. The visitor methods return values that propagate up the parse tree, letting you construct your AST bottom-up.

Error handling

ANTLR provides sophisticated error recovery. The default behavior attempts to continue parsing after errors, collecting multiple error messages rather than stopping at the first problem. You can customize this with your own error listener:

use antlr_rust::error_listener::ErrorListener;

struct MyErrorListener;

impl ErrorListener for MyErrorListener {
    fn syntax_error(
        &self,
        _recognizer: &dyn antlr_rust::recognizer::Recognizer,
        _offending_symbol: Option<&dyn antlr_rust::token::Token>,
        line: isize,
        column: isize,
        msg: &str,
        _e: Option<&antlr_rust::errors::ANTLRError>,
    ) {
        eprintln!("Error at {}:{}: {}", line, column, msg);
    }
}

// Attach to parser
parser.remove_error_listeners();
parser.add_error_listener(Box::new(MyErrorListener));

Removing the default listeners before adding your own prevents duplicate messages. The error recovery strategies are configurable: you can tell ANTLR to bail out immediately, to try single-token insertion or deletion, or to use more aggressive recovery that skips to synchronization points.

Choosing the right tool

The choice between nom, pest, and ANTLR depends on your grammar’s complexity, your performance requirements, and your team’s preferences.

Choose nom when parsing is simple or medium complexity and you want maximum control. Parser combinators excel when you need zero-copy parsing for performance, when you want to integrate parsing deeply with Rust’s type system, or when the grammar is simple enough that a dedicated file feels like overkill.

Choose pest when you want readable, declarative grammars and PEG semantics fit your language. pest works well for configuration files, markup languages, and DSLs where ordered choice resolves ambiguity naturally. The Rust-native integration is seamless, and automatic whitespace handling reduces boilerplate.

Choose ANTLR when your grammar is complex enough that left recursion is natural, when you need cross-language support, or when sophisticated error recovery matters. ANTLR’s tooling is mature: IDE plugins, debuggers, and visualization tools have been refined over decades. The grammar file serves as authoritative documentation.

Real-world example: a configuration language

Let’s implement a practical configuration language in both pest and ANTLR. The language looks like this:

# Comment
database {
    host = "localhost"
    port = 5432
    ssl = true
}

server {
    workers = 4
    timeout = 30s
}

Blocks contain properties. Properties have values of different types: strings, numbers, booleans, and durations. Comments start with #. This is similar to formats like TOML or Nginx configuration.

pest implementation

// config.pest
WHITESPACE = _{ " " | "\t" | "\n" | "\r" }
COMMENT    = _{ "#" ~ (!"\n" ~ ANY)* }

config = { SOI ~ block* ~ EOI }

block = { identifier ~ "{" ~ property* ~ "}" }

property = { identifier ~ "=" ~ value }

value = { string | duration | number | boolean }

identifier = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "_")* }
string     = @{ "\"" ~ (!"\"" ~ ANY)* ~ "\"" }
number     = @{ "-"? ~ ASCII_DIGIT+ }
duration   = @{ ASCII_DIGIT+ ~ ("s" | "ms" | "m" | "h") }
boolean    = { "true" | "false" }

The grammar is compact and readable. Silent rules handle whitespace and comments automatically. Note that duration must come before number in the value rule’s alternatives. Because PEGs use ordered choice, if number came first, it would match the digits in “30s” and leave “s” unparsed.

nom implementation

The same language in nom requires more code but offers finer control:

use nom::{
    IResult,
    branch::alt,
    bytes::complete::{tag, take_while1, take_until},
    character::complete::{char, multispace0, alpha1, alphanumeric0, digit1},
    combinator::{map, opt, recognize, value},
    multi::many0,
    sequence::{delimited, pair, preceded, terminated, tuple},
};

#[derive(Debug)]
struct Config {
    blocks: Vec<Block>,
}

#[derive(Debug)]
struct Block {
    name: String,
    properties: Vec<Property>,
}

#[derive(Debug)]
struct Property {
    name: String,
    value: Value,
}

#[derive(Debug)]
enum Value {
    String(String),
    Number(i64),
    Bool(bool),
    Duration(u64, DurationUnit),
}

#[derive(Debug)]
enum DurationUnit { Seconds, Milliseconds, Minutes, Hours }

fn ws<'a, F, O>(f: F) -> impl FnMut(&'a str) -> IResult<&'a str, O>
where
    F: FnMut(&'a str) -> IResult<&'a str, O>,
{
    preceded(multispace0, f)
}

fn config(input: &str) -> IResult<&str, Config> {
    let (input, blocks) = many0(preceded(multispace0, block))(input)?;
    Ok((input, Config { blocks }))
}

fn block(input: &str) -> IResult<&str, Block> {
    let (input, name) = identifier(input)?;
    let (input, _) = ws(char('{'))(input)?;
    let (input, props) = many0(property)(input)?;
    let (input, _) = ws(char('}'))(input)?;
    Ok((input, Block { name: name.to_string(), properties: props }))
}

fn property(input: &str) -> IResult<&str, Property> {
    let (input, _) = multispace0(input)?;
    let (input, name) = identifier(input)?;
    let (input, _) = ws(char('='))(input)?;
    let (input, val) = value_parser(input)?;
    Ok((input, Property { name: name.to_string(), value: val }))
}

fn value_parser(input: &str) -> IResult<&str, Value> {
    alt((
        map(string_literal, Value::String),
        map(duration, |(n, unit)| Value::Duration(n, unit)),
        map(number, Value::Number),
        map(boolean, Value::Bool),
    ))(input)
}

fn identifier(input: &str) -> IResult<&str, &str> {
    recognize(pair(
        alpha1,
        take_while1(|c: char| c.is_alphanumeric() || c == '_'),
    ))(input)
}

fn string_literal(input: &str) -> IResult<&str, String> {
    let (input, _) = char('"')(input)?;
    let (input, content) = take_until("\"")(input)?;
    let (input, _) = char('"')(input)?;
    Ok((input, content.to_string()))
}

fn number(input: &str) -> IResult<&str, i64> {
    let (input, neg) = opt(char('-'))(input)?;
    let (input, digits) = digit1(input)?;
    let n: i64 = digits.parse().unwrap();
    Ok((input, if neg.is_some() { -n } else { n }))
}

fn duration(input: &str) -> IResult<&str, (u64, DurationUnit)> {
    let (input, digits) = digit1(input)?;
    let (input, unit) = alt((
        value(DurationUnit::Milliseconds, tag("ms")),
        value(DurationUnit::Seconds, tag("s")),
        value(DurationUnit::Minutes, tag("m")),
        value(DurationUnit::Hours, tag("h")),
    ))(input)?;
    let n: u64 = digits.parse().unwrap();
    Ok((input, (n, unit)))
}

fn boolean(input: &str) -> IResult<&str, bool> {
    alt((
        value(true, tag("true")),
        value(false, tag("false")),
    ))(input)
}

The nom version is roughly four times longer but makes every parsing decision explicit. The AST types are defined in the same file. Error handling can be customized at each parser. Performance-critical applications might prefer nom’s zero-copy parsing, where identifier returns a &str slice into the original input rather than allocating a new String.

Tradeoffs

+----------------+-----------------------------------+----------------------------------+
| Aspect         | pest                              | nom                              |
+================+===================================+==================================+
| Lines of code  | ~15 (grammar) + ~50 (AST builder) | ~100 (combined)                  |
| Readability    | Grammar is self-documenting       | Parser logic visible but verbose |
| Performance    | Good                              | Excellent (zero-copy possible)   |
| Error messages | Automatic, decent                 | Manual, customizable             |
| Debugging      | Grammar visualizers               | Standard Rust debugging          |
+----------------+-----------------------------------+----------------------------------+

For this configuration language, pest is probably the better choice. The grammar is clear enough that anyone familiar with the format can understand it. Automatic whitespace handling eliminates a whole category of bugs. Error messages require no extra work.

nom would make sense if you needed to embed this parser in a performance-critical path where allocations matter, or if the configuration format had unusual requirements that pest’s semantics couldn’t express naturally.

What a grammar file buys you

Grammar-driven parsing changes your relationship with syntax. Instead of building up parsing logic incrementally in code, you design the grammar as a coherent specification. The grammar file becomes documentation that other tools can analyze, that IDEs can highlight, and that colleagues can read.

This declarative approach has the same tradeoffs as declarative approaches elsewhere in software. You gain clarity and let the tool handle common concerns. You lose fine-grained control over exactly what happens at each step. When the abstraction fits your problem, you write less code and make fewer mistakes. When it doesn’t fit, you spend time working around constraints rather than solving your actual problem.

pest brings PEG semantics to Rust with minimal friction. Ordered choice eliminates ambiguity, automatic whitespace handling reduces boilerplate, and the derive macro integrates cleanly with Cargo’s build system. For configuration files, markup languages, and DSLs where the grammar is the authoritative specification, pest is an excellent choice.

ANTLR brings industrial-strength parsing when your needs outgrow simpler tools. Left recursion, sophisticated error recovery, and cross-language generation justify the additional setup complexity. If you’re building a parser that will evolve over years, ANTLR’s mature tooling becomes an asset.

From grammars to state machines

Having settled how a DSL recognizes valid syntax, the series turns its attention from parsing to state machines. We’ll explore the typestate pattern, which uses Rust’s type system to enforce valid state transitions at compile time. Then we’ll build runtime state machines with macros, creating DSLs that specify states and transitions declaratively while generating efficient dispatch code.

Parsing and state machines complement each other. A DSL’s grammar defines what inputs are syntactically valid. A state machine defines what sequences of operations are semantically valid. Master both and you can create DSLs that guide users toward correct usage while providing clear feedback when they stray.

Further reading

• pest book provides comprehensive documentation for pest’s grammar syntax and usage patterns
• pest grammar syntax details every grammar construct
• ANTLR 4 documentation covers the full parser generator
• ANTLR Rust runtime documents the Rust-specific API
• PEG paper is Bryan Ford’s original paper defining Parsing Expression Grammars

The series so far

• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6 (this post)

Want more like this?

I write regularly about Rust, design patterns, and performance tips. Follow me here on Medium to stay updated.