Building an OTA System Changed How I Think About Engineering

A little different from my usual Medium posts.

If you’ve read my previous articles, you probably know they’re mostly about VLSI, FPGA development, and digital design.

This one is different.

For the past few months, I’ve been working on an engineering project during my internship that I never really spoke about publicly. I wanted to finish it first and reflect on what it actually taught me.

Without getting into company or product details, I spent these months designing and implementing an Over-the-Air (OTA) update system for embedded edge devices.

I thought I would be learning how software updates work.

I ended up learning how systems are designed.

I had the wrong idea about OTA

Before this project, OTA sounded simple.

Build a new version.

Upload it.

The device downloads it.

Restart.

Done.

That was pretty much my understanding.

As I got involved in the architecture, I realized that the update itself is probably the easiest part.

The difficult part is everything around it.

How do devices know an update exists?

What happens if they lose connection halfway through?

How do you know an update actually succeeded?

How do you avoid interrupting someone who’s actively using the device?

Those questions turned out to be much more interesting than writing the update logic itself.

One small decision changed everything

One feature taught me more than I expected.

Initially, whenever a new version was available, the client downloaded it and restarted immediately.

Technically, it worked.

From a user’s perspective, it was awful.

Imagine being in the middle of a session and suddenly the application restarts because a new version arrived.

The software wasn’t broken.

The experience was.

So I changed the flow.

The client now downloads the update in the background and waits until the device becomes idle before switching to the new version.

It sounds like a small change.

It completely changed how I think about software.

Good engineering isn’t just making something work.

It’s making it work without getting in the user’s way.

Bugs sometimes improve the design

One bug completely changed part of the architecture.

While testing, I uploaded a new version through the dashboard.

Everything looked fine.

Except the binary never reached cloud storage because my local environment wasn’t authenticated correctly.

That bug exposed another problem.

Our fallback approach for storing binaries wasn’t something that would scale well in the long run.

Instead of patching around it, I redesigned that part of the system and separated binary storage from version metadata.

That same incident also led me to build version management into the dashboard.

Sounds simple until you start asking questions.

Can you delete a version that’s currently running on a device?

What if another device is downloading it?

What if an update is already in progress?

Engineering is full of features that look tiny until you start thinking about edge cases.

The hardest problem wasn’t what I expected

When I started, I thought networking would be the difficult part.

Keeping devices connected.

Maintaining communication.

Handling unreliable internet.

Those problems certainly exist.

But they weren’t the hardest ones.

The real challenge was state management.

Designing what happens when updates fail.

Recovering cleanly.

Handling retries.

Making sure every device eventually reaches the correct state.

Production systems aren’t defined by the happy path.

They’re defined by everything that happens when things go wrong.

Hardware and software started feeling like one thing

This project was the first time I constantly switched between completely different ways of thinking.

One hour I was working on version management, cloud storage, dashboards, and fleet management.

The next hour I was thinking about embedded Linux, application restarts, symbolic links, and device behavior.

It never felt like backend engineering or embedded engineering.

It felt like building one system where every layer depended on the others.

I enjoyed that more than I expected.

Looking back

Something funny happened after working on this project.

Whenever my phone says there’s a software update available, I don’t just press “Update Tonight” anymore.

I start wondering.

When did it download?

How does it know it’s safe to install?

What happens if the battery dies halfway through?

What if something fails?

Building one system completely changed how I look at every other system around me.

That’s probably the biggest thing I take away from this experience.

I still love VLSI and digital design, and that’s the direction I want to continue pursuing.

But this project introduced me to edge computing and reliable system design in a way I never expected.

And I’m really glad it did.

Thanks for reading.

I’d love to hear what engineering project unexpectedly changed the way you think about technology.