Aquin DevKit: SDK & CLI

[embed]Client Challenge Edit descriptionpypi.org

If you’ve ever trained a model and wondered what actually changed. Aquin’s new SDK and CLI are worth a look.

Aquin is a training observability tool for ML engineers. You instrument your Python training loop with a few lines, push the run to their platform, and get a full post-hoc inspection: loss curves, learning rate, grad norm, epoch summaries, and two unique analyses: an SAE diff (which sparse autoencoder features strengthened or suppressed during fine-tuning) and a model diff (how the fine-tune shifted the model’s behaviour on a shared prompt set).

How it works

1. Install with pip install aquin and save your API key via aquin login.

2. Wrap your training loop with aquin.init(), run.log() per step, and run.checkpoint() at the end.

3. Run aquin package then aquin push: your run appears in the dashboard under CLI runs.

It’s framework-agnostic, but there’s a clean TrainerCallback pattern for HuggingFace Trainer and TRL that wires everything in without touching your training logic.

What makes it interesting

Most training dashboards stop at loss curves. Aquin goes further by loading your base model and fine-tuned checkpoint side by side on their VM and running mechanistic analyses automatically. The SAE diff surfaces which concepts got stronger or weaker. The model diff scores consistency, suppression, and robustness, giving you a behavioural fingerprint of what the fine-tune actually did.

Currently supported base models: Llama-3.2-1B-Instruct, pythia-2.8b, pythia-70m-deduped, and gpt2.

The SDK is at v0.0.1, actively developed by Aquin Labs. Worth watching if you care about understanding your fine-tunes beyond the loss curve.