I Built a Production ML Feedback Loop with FastAPI and Render. Here’s What Actually Broke.

The story of deploying a real ML backend — Store and Forward, image quality gates, automated retraining triggers, and the three things I had to fix before it worked in the wild.

Part 4 of 5 in the PlantSnap series — building a production herb classifier for iOS

When PlantSnap’s iOS app identifies a herb incorrectly, something needs to happen with that mistake.

Not just “log it somewhere.” The correction needs to survive offline conditions in a forest, sync reliably when signal returns, pass a quality check before it contaminates the training data, and eventually trigger a retraining run that improves the model.

That’s a lot of moving parts for what sounds like a simple feedback form.

This is the story of how I built that system — with FastAPI, deployed on Render.com, and connected to an iOS app running CoreML in the middle of nowhere.

Why FastAPI, Not Flask?

I’ve used Flask before. It works. But for a system where iOS is constantly sending inference requests and feedback corrections — sometimes hundreds in a burst when a user gets back to WiFi — I needed async by default.

FastAPI gave me three things immediately:

1. Automatic API docs. The moment I defined my Pydantic models, /docs showed up with a full interactive Swagger UI. Every endpoint, every schema, every expected input and output — live and testable without writing a single line of documentation. This turned out to be critical for debugging the mobile client integration.

2. Pydantic validation. Every request to my API is validated before my code even runs. Wrong field name? Missing required field? Wrong type? FastAPI returns a clear 422 error with the exact problem. Compare this to Flask, where you’d write your own validation logic and probably miss edge cases.

3. Async handlers throughout. S3 uploads, SQLite writes, background retraining triggers — none of these block the response. The iOS app gets its acknowledgement immediately, and the heavy work happens in the background.

from fastapi import FastAPI, BackgroundTasks
from pydantic import BaseModel

app = FastAPI()

class FeedbackRequest(BaseModel):
    predicted_herb: str
    correct_herb: str
    confidence: float
    image_base64: str
    device_id: str

@app.post("/feedback")
async def submit_feedback(
    req: FeedbackRequest,
    background_tasks: BackgroundTasks
):
    # Respond immediately
    background_tasks.add_task(process_feedback, req)
    return {"status": "queued", "message": "Feedback received"}

The iOS app gets a response in milliseconds. The actual image processing — decoding, quality checking, S3 upload — happens after.

The 3-Layer Image Quality Gate

The first version of my feedback endpoint accepted everything. Any image, any quality, any content.

After a week of testing, I had:

• 47 blurry images where you couldn’t identify anything
• 12 screenshots of other apps (yes, really)
• 8 images that were clearly not plants at all

Garbage data is worse than no data. A retraining run using these images would make the model worse, not better. I needed a quality gate before anything touched S3.

The gate has three layers, each rejecting a different failure mode:

Layer 1 — Blur Detection (OpenCV)

import cv2
import numpy as np

def is_blurry(image_array: np.ndarray, threshold: float = 100.0) -> bool:
    gray = cv2.cvtColor(image_array, cv2.COLOR_BGR2GRAY)
    laplacian_var = cv2.Laplacian(gray, cv2.CV_64F).var()
    return laplacian_var < threshold

The Laplacian variance measures edge sharpness. A sharp herb photo has lots of defined edges — stem, leaf veins, petal borders. A blurry photo has low variance because everything bleeds together. Below 100? Rejected.

Layer 2 — Brightness Check

A photo taken in a dark forest pocket or directly into the sun is useless for training. I check the mean pixel value on the value channel in HSV space:

def check_brightness(image_array: np.ndarray) -> tuple[bool, float]:
    hsv = cv2.cvtColor(image_array, cv2.COLOR_BGR2HSV)
    brightness = hsv[:, :, 2].mean()
    # Reject below 30 (too dark) or above 240 (overexposed)
    return 30 < brightness < 240, brightness

Layer 3 — CLIP Zero-Shot Verification

This is the most interesting layer. Even if an image is sharp and well-lit, it might not be a plant at all.

I use OpenAI’s CLIP to ask “is this image a plant?” — without any training, without any task-specific fine-tuning. CLIP encodes both the image and the text description “a photo of a plant” as vectors in a shared space. High cosine similarity means the image looks like what the text describes.

import torch
import clip
from PIL import Image

model, preprocess = clip.load("ViT-B/32")
text_labels = ["a photo of a plant", "not a plant"]
text_tokens = clip.tokenize(text_labels)

with torch.no_grad():
    text_features = model.encode_text(text_tokens)
    text_features /= text_features.norm(dim=-1, keepdim=True)

def verify_is_plant(image: Image.Image) -> tuple[bool, float]:
    image_tensor = preprocess(image).unsqueeze(0)
    with torch.no_grad():
        image_features = model.encode_image(image_tensor)
        image_features /= image_features.norm(dim=-1, keepdim=True)
        similarities = (image_features @ text_features.T).squeeze()
    plant_score = similarities[0].item()
    return plant_score > 0.25, plant_score

If the similarity to “a photo of a plant” is below 0.25 — rejected. This caught every screenshot and non-plant image in my test set.

Only images that pass all three layers reach S3.

The Confidence Threshold Decision

Not every correction is equally valuable for retraining.

If ResNet18 said “chamomile” with 94% confidence and the user says “actually it’s feverfew” — that’s a genuinely hard case. High-confidence wrong predictions are the most valuable training examples because they show the model’s blind spots.

If ResNet18 said “something” with 23% confidence and the user corrects it — that’s expected. Low confidence means the model already knows it doesn’t know. Less valuable.

I added a confidence threshold: only save corrections where the original prediction confidence was below 70%. These are the uncertain predictions that benefit most from correction signals.

CONFIDENCE_THRESHOLD = 0.70

async def process_feedback(req: FeedbackRequest):
    # Only save high-value corrections
    if req.confidence > CONFIDENCE_THRESHOLD:
        # Log it but don't save to training pool
        logger.info(f"High confidence correction skipped: {req.confidence}")
        return

    # Pass through quality gate
    image = decode_base64_image(req.image_base64)
    passed, reason = run_quality_gate(image)

    if not passed:
        logger.info(f"Image rejected by quality gate: {reason}")
        return

    # Save to S3 and SQLite
    await save_correction(req, image)

This felt counterintuitive at first — shouldn’t I save everything? But every noisy image in the training pool degrades the signal. I’d rather have 200 clean, high-value corrections than 1,000 mixed ones.

Store and Forward — The Core Mobile Pattern

PlantSnap users are in forests. Forests have no signal.

The feedback from a correction made at 2pm in the middle of Yosemite might not reach my server until the user drives back to the trailhead at 6pm. The iOS app needs to queue corrections locally and sync them when connectivity returns.

I implemented this with a simple queue in local storage on iOS:

// iOS side — queue corrections locally
func submitCorrection(_ correction: HerbCorrection) {
    var queue = loadQueue()
    queue.append(correction)
    saveQueue(queue)
    attemptSync()
}

func attemptSync() {
    guard NetworkMonitor.shared.isConnected else { return }

    let queue = loadQueue()
    for correction in queue {
        Task {
            do {
                try await api.submitFeedback(correction)
                removeFromQueue(correction)
            } catch {
                // Will retry on next connection
            }
        }
    }
}

On the server side, I made the feedback endpoint idempotent. Each correction gets a UUID generated on the device. If the same UUID arrives twice (because of a retry), the second is silently ignored:

@app.post("/feedback")
async def submit_feedback(req: FeedbackRequest):
    # Idempotency check
    existing = db.get_correction(req.correction_id)
    if existing:
        return {"status": "already_received", "id": req.correction_id}

    # Process normally
    background_tasks.add_task(process_feedback, req)
    return {"status": "queued", "id": req.correction_id}

Without idempotency, a user with flaky connectivity could submit the same correction 5 times as their phone keeps retrying. Five copies of the same image in your training data is actively harmful.

Deploying to Render — What I Expected vs What Happened

I chose Render.com for a specific reason: free tier with zero configuration. No Kubernetes, no Docker Compose, no nginx config. You point it at a GitHub repo and it deploys.

That’s what I expected. Here’s what actually happened:

Problem 1: Cold starts killed my response times

On the free tier, Render spins down your service after 15 minutes of inactivity. The first request after spin-down takes 30–60 seconds to respond — Render is booting the entire container from scratch.

For a health check endpoint, that’s fine. For an iOS app that shows a loading spinner to a user, that’s terrible.

My fix: a lightweight keep-alive ping from the iOS app. Every 10 minutes while the app is in the foreground, it hits /health. The service stays warm, response times stay under 200ms.

@app.get("/health")
async def health_check():
    return {"status": "ok", "timestamp": datetime.utcnow().isoformat()}

// iOS — keep service warm while app is active
func startKeepAlive() {
    Timer.scheduledTimer(withTimeInterval: 600, repeats: true) { _ in
        Task { try? await api.healthCheck() }
    }
}

Problem 2: CLIP model loading time

CLIP takes ~8 seconds to load the ViT-B/32 model. On my local machine this was barely noticeable. On Render’s free tier, this meant every cold start took 8+ seconds just loading the model — before serving any requests.

Fix: load the model once at startup, not per-request.

# Load ONCE when the app starts
@app.on_event("startup")
async def load_models():
    global clip_model, clip_preprocess, text_features
    clip_model, clip_preprocess = clip.load("ViT-B/32")
    # Pre-compute text embeddings for quality gate
    labels = ["a photo of a plant", "not a plant"]
    tokens = clip.tokenize(labels)
    with torch.no_grad():
        text_features = clip_model.encode_text(tokens)
        text_features /= text_features.norm(dim=-1, keepdim=True)

Now the 8-second load happens once at deployment, not on every cold start request.

Problem 3: SQLite on Render’s ephemeral filesystem

This one hurt. Render’s free tier uses an ephemeral filesystem — every redeploy wipes it clean. I was storing my SQLite database on the local filesystem.

After my first redeployment, all my feedback metadata was gone.

Fix: metadata stays in SQLite (cheap, fast for queries), but the actual correction images and a backup of the SQLite file both live in S3. On startup, the service checks S3 for an existing database backup and restores it before accepting requests.

@app.on_event("startup")
async def restore_database():
    if not os.path.exists("feedback.db"):
        try:
            s3.download_file(BUCKET, "backups/feedback.db", "feedback.db")
            logger.info("Database restored from S3")
        except ClientError:
            logger.info("No backup found, starting fresh")

The Retraining Trigger

The pipeline is: user corrects herb → image passes quality gate → saved to S3 → triggers retraining when enough corrections accumulate.

“Enough” is 50 new corrections since the last training run. I check this count every time a correction is saved:

async def maybe_trigger_retraining():
    count = db.count_corrections_since_last_run()
    if count >= RETRAINING_THRESHOLD:
        logger.info(f"Retraining threshold reached: {count} corrections")
        background_tasks.add_task(run_retraining_pipeline)

async def run_retraining_pipeline():
    # 1. Download all S3 corrections
    corrections = await download_s3_corrections()

    # 2. Add to training data
    merge_with_training_set(corrections)

    # 3. Retrain model
    new_model_path = train_model()

    # 4. Upload new CoreML model to S3
    await upload_model_to_s3(new_model_path)

    # 5. Bump version number
    increment_model_version()

    # 6. iOS will pick it up on next /version check
    logger.info("Retraining complete, new model deployed")

The iOS app checks /version on every launch. If the server version is higher than the local version, it downloads the new CoreML model silently in the background and swaps it in on next launch. Zero App Store submission required.

17/17 Tests Passing

Before I deployed anything, I wrote regression tests for every endpoint. Not “unit tests” in the abstract sense — actual integration tests that hit the running API with real payloads and check real responses.

def test_feedback_blurry_image_rejected():
    blurry_image = create_synthetic_blurry_image()
    response = client.post("/feedback", json={
        "predicted_herb": "chamomile",
        "correct_herb": "feverfew",
        "confidence": 0.45,
        "image_base64": encode_image(blurry_image),
        "device_id": "test_device"
    })
    assert response.status_code == 200
    assert response.json()["quality_check"]["passed"] == False
    assert "blur" in response.json()["quality_check"]["reason"]

def test_feedback_idempotency():
    correction_id = str(uuid.uuid4())
    payload = {..., "correction_id": correction_id}

    # First submission
    r1 = client.post("/feedback", json=payload)
    assert r1.json()["status"] == "queued"

    # Duplicate submission
    r2 = client.post("/feedback", json=payload)
    assert r2.json()["status"] == "already_received"

17 tests, all passing, all run automatically on every GitHub push via Actions. The CI/CD pipeline deploys to Render only if all 17 pass.

The live API is at computer-vision-yin8.onrender.com/docs — you can hit it right now and see every endpoint.

What I’d Do Differently

Use PostgreSQL instead of SQLite + S3 backup. The SQLite workaround works, but it’s fragile. Render offers a managed PostgreSQL free tier. I’d use that from the start.

Rate limit per device_id. Right now, nothing stops a device from submitting 1,000 corrections in a second. I’d add per-device rate limiting in a production deployment.

Add monitoring. The /metrics endpoint tracks confidence distributions per herb, but I have no alerting if the service goes down. Render has basic health check monitoring but I'd add a proper observability layer.

The Feedback Loop, Complete

Here’s the full picture of what we built:

The model gets smarter from real-world usage. Every user correction in the field becomes training data. The feedback loop closes.

That’s what production ML actually looks like — not a Jupyter notebook, not a demo that works on your laptop. A system that handles offline conditions, validates data quality, deploys automatically, and improves without you touching it.

References

[1] FastAPI Documentation. Background Tasks. https://fastapi.tiangolo.com/tutorial/background-tasks/

[2] Radford, A. et al. (2021). Learning Transferable Visual Models From Natural Language Supervision. ICML. arXiv

[3] OpenCV Documentation. Laplacian Edge Detection. https://docs.opencv.org/4.x/d5/db5/tutorial_laplace_operator.html

[4] Paszke, A. et al. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. NeurIPS. arXiv

[5] Render Documentation. Free Instance Types. https://docs.render.com/free

[6] Pydantic Documentation. Data Validation using Python Type Annotations. https://docs.pydantic.dev/

Links

• 🌿 Live API: https://computer-vision-yin8.onrender.com/docs
• 📱 Article 1: I Trained ResNet18 to Identify 70 Medicinal Herbs
• 📱 Article 2: I Built an ML App for Foragers in Forests. Server-Side Inference Would Have Killed It.
• 📱 Article 3: I Ran My Herb Classifier 4 Times. The Third Run Made It Worse. Here’s Why.
• 💻 GitHub: https://github.com/rachana1101/plantsnap
• 💼 LinkedIn: https://www.linkedin.com/in/rachana1101/

If this resonated, follow for upcoming Articles. Each one goes deeper into the PlantSnap stack.

Building something similar? I’d love to hear about it — drop a comment or connect on LinkedIn!