ClearCast : Architecture notes from a privacy-first audio enhancer

A few months ago I noticed a pattern in my own workflow: the “recording” part is easy, but the “make it listenable” part is where time disappears.

Trim dead air. Reduce noise. Remove the “um / like / actually” ticks. Normalize loudness. Export something you’re not embarrassed to publish.

Most tools that do this well are cloud-based; meaning you upload raw audio (often the most sensitive part of what you’re making) to a server you don’t control. I wanted the opposite: a tool that keeps everything local, is inspectable end-to-end, and is simple enough that I can maintain it without turning it into a research project.

So I built clearcast: a local-first voice editor that runs on your machine, shells out to FFmpeg and whisper.cpp, and gives you a transcript-driven UI to decide what gets kept, cut, muted, or attenuated.

Code: [github.com/AakashKJha/clearcast](https://github.com/AakashKJha/clearcast)

What clearcast does (the 30-second tour)

The UI is intentionally minimal: record, upload, review, render.

Record or upload, everything stays on disk, local-first.

Then clearcast runs a pipeline:

1. Denoise / cleanup (FFmpeg filter chain)
2. Transcribe with Whisper (word-level timestamps)
3. Build an EDL (Edit Decision List): a full timeline of segments with actions like KEEP/CUT/ATTENUATE
4. You review the transcript, click words to override decisions, then render a final MP3 (loudness-normalized)

Click-to-edit transcript: the UI edits a timeline, not the raw file.

Pipeline runs once; renders can happen many times.

Stack:

• Java 21 + Spring Boot 3.4
• Vanilla JS frontend (served as static assets)
• FFmpeg + ffprobe + whisper.cpp
• Storage: filesystem (no DB); might consider later

The design decision that made everything else easier: the EDL

The first real architecture question wasn’t “how do I call FFmpeg?” it was: How do I represent “edited audio” so it’s easy to render and easy to change from the UI?

The naive model is “original audio + list of cut ranges.” That works for basic trimming, but it breaks as soon as you want anything more expressive:

• “keep this, but quieter”
• “mute this section (for video) without changing timing”
• “attenuate breaths instead of cutting them (cutting all breaths sounds robotic)”

So clearcast uses an Edit Decision List (EDL) as the central abstraction.

Instead of tracking edits as exceptions (“cut these ranges”), the EDL is a contiguous timeline:

• covers [0,duration][0,duration]
• no gaps, no overlaps
• each segment has an action and a reason

Conceptually:

public record EdlSegment(
    String id,
    double start,
    double end,
    EdlAction action,   // KEEP | CUT | MUTE | ATTENUATE
    EdlReason reason,   // SPEECH | FILLER | SILENCE | BREATH | MANUAL
    String text,
    List<String> wordIds,
    double gainDb       // only meaningful for ATTENUATE
) {}

This choice paid off in three ways:

• One invariant keeps you honest. If the EDL is gap-free, rendering becomes deterministic.
• Edits are local. Clicking a word changes one segment’s action; no juggling between “kept list” and “cut list.”
• Rendering is straightforward. Iterate segments in order and build an FFmpeg filter graph that applies the chosen actions.

If I had to summarize the lesson: pick the representation that makes the operations cheap and obvious, not the one that looks the smallest.

Extensibility: two seams, not one “clever” abstraction

Auto-editing lives or dies on detectors: fillers, silence trimming, breaths, stutters, mouth clicks. I wanted to add new detectors without re-wiring the pipeline every time, so there’s a seam for word-level classification:

public interface WordClassifier {
    Optional<EdlReason> classify(Word word);
}

Spring injects all implementations as a list; the EDL builder runs them in order and the first match wins. Adding a detector is “drop in a @Component.”

Then I hit the breath problem: breaths aren’t words. Whisper won’t hand you a token for them. Breath detection wants to analyze gaps between words, and often wants the audio.

At that point I could have forced everything into one “unified classifier interface” that accepts a kitchen sink of inputs. It would look consistent, but every implementation would ignore half the parameters and document the other half. That’s a classic leaky abstraction.

So clearcast uses two extension seams:

• Word classifiers: cheap, pure, transcript-based
• Gap/audio analyzers: operate on silence segments / audio-derived features

It’s slightly less “uniform,” but it’s more honest, and easier to test in isolation.

An interface with one implementation isn’t always over-engineering

Denoising is the part I expect to tweak the most: different rooms, different microphones, different noise profiles.

So it’s a strategy boundary:

public interface DenoiseStrategy {
    String name();
    String filterChain();
}

Right now there’s one strategy (aggressive cleanup). That can look like over-engineering, but the boundary is useful because it keeps other code stable:

• The pipeline doesn’t learn what “denoise” means
• The FFmpeg runner stays generic
• Adding a “light cleanup” strategy later is additive, not a refactor

Heuristic that worked for me: interfaces earn their keep when they freeze change at a boundary, not only when you have many implementations today.

State modeling: two lifecycles, two enums

A project has two distinct lifecycles:

1. processing: ingest → denoise → transcribe → EDL-ready
2. rendering: render → edit → re-render → repeat

It’s tempting to collapse them into one enum, but that creates weird questions like: “what’s the state after render, then edit?” It’s not “rendered” (output is stale) and it’s not “ready” either.

So clearcast keeps them separate: processing status vs render status. It mirrors reality and avoids the “combined state explosion.”

Persistence: a folder per project (no database)

For a local-first tool, a database felt like unnecessary complexity. Each project is a directory:

~/.voice-editor/projects/{uuid}/
  metadata.json
  original.{ext}
  denoised.wav
  transcript.json
  edl.json
  output.mp3

Benefits:

• portable (copy one folder)
• debuggable (open the JSON, inspect what happened)
• crash-safe with atomic writes (temp + rename)

Databases are great when you need concurrency, queries, or transactions across many records. For “documents keyed by ID,” the filesystem is simpler.

What’s next (and what I’d redo)

A few things I’d improve with more time/users:

• Undo: EDL edits are perfect for an undo stack; it’s just not implemented yet.
• Streaming pipeline: long recordings could be processed chunk-by-chunk so you can start reviewing early.
• Diff-based EDL updates: today you can replace the whole EDL; diffs matter only at much larger scales.

Try it

Repo: [github.com/AakashKJha/clearcast](https://github.com/AakashKJha/clearcast)

If you try it and it breaks on your machine, that’s useful feedback — I’m especially interested in portability issues around FFmpeg/whisper installs and performance differences across platforms.