Stop Scrapping and Start Synthesizing: My Python Pipeline for Custom AI Training

The most soul-crushing part of Computer Vision isn’t the math — it’s the labeling. We’ve all been there: sitting for hours, clicking boxes around cars, faces, or trees until our eyes blur. But in 2026, we have a better way. I decided to build a pipeline that does the work for me, and the results were a game-changer.

By merging Synthetic Data for Computer Vision with automated foundation models, I created a “self-labeling” ecosystem. This isn’t just a shortcut; it’s a fundamental shift in how we build AI. Instead of hunting for data, we synthesize it. Instead of drawing boxes, we describe them.

In this story, I’ll walk you through the exact Python logic I used to generate high-fidelity images with Stable Diffusion and auto-annotate them using GroundingDINO. We are moving from being “data cleaners” to “data architects.”

Why Synthetic Data for Computer Vision is the Future of AI Development

The traditional approach of “collect, clean, and label” is rapidly becoming obsolete as the demand for massive, high-quality datasets outpaces human capacity. Synthetic Data for Computer Vision offers a scalable alternative that allows developers to simulate rare edge cases, controlled environments, and diverse lighting conditions that are often impossible to capture in the wild. By shifting the focus from manual data collection to algorithmic data generation, teams can iterate on their models much faster, testing new classes and scenarios with almost zero marginal cost.

At its core, the goal is to create “digital twins” of the objects or environments your model needs to understand. When we use tools like Stable Diffusion, we aren’t just making pretty pictures; we are generating mathematically complex visual information that serves as a ground-truth foundation for neural networks. This approach is particularly effective for classification and object detection, where the model needs thousands of examples to distinguish between similar categories. By generating this data synthetically, you ensure that your dataset is balanced, diverse, and completely free from the human errors often found in manual annotations.

On a high level, this methodology represents a transition toward “Data-Centric AI,” where the quality and programmatic control of your data are prioritized. Using foundational models like GroundingDINO to label synthetic images creates a closed-loop system where the AI essentially teaches itself. This pipeline allows you to move from a concept to a functional, trained model in a single afternoon. Whether you are building a system to monitor industrial equipment or a mobile app for wildlife identification, mastering these automated pipelines is the most effective way to stay competitive in the modern AI landscape.

Building an Automated Data Factory: From Pixels to Labels

The technical core of this tutorial is centered around a multi-stage Python pipeline designed to eliminate the manual labor traditionally associated with building Computer Vision datasets. The primary target of the code is to create a seamless bridge between Generative AI and Object Detection. By the end of the script execution, you move from having nothing but a list of category names to possessing a fully structured, labeled dataset ready for training state-of-the-art models like YOLOv8 or EfficientDet.

This workflow is achieved by first leveraging the diffusers library to tap into specialized Stable Diffusion models. Instead of relying on existing, potentially biased datasets, the code programmatically generates photorealistic images based on highly specific text prompts. This level of control allows you to define exactly what your model sees—specifying lighting, camera angles (like "medium-shot" or "front view"), and environmental context. This is particularly valuable for rare objects or specific industrial use cases where real-world images are difficult or expensive to acquire.

Once the synthetic images are generated and organized into their respective directories, the pipeline transitions into the annotation phase using the autodistill ecosystem. The script implements GroundingDINO, a zero-shot object detection model that uses natural language to "find" objects within an image. By defining an "ontology"—a simple mapping of text descriptions to class labels—the code instructs the model to scan the newly created synthetic images and automatically calculate precise bounding box coordinates ($x, y, w, h$).

The final stage of the code handles the data serialization and directory management. It doesn’t just show you detections on a screen; it writes the results into standardized formats like YOLO or COCO, ensuring that the output is immediately compatible with training scripts. By integrating these disparate technologies into a single, cohesive Python environment, the tutorial provides a blueprint for an automated data factory. This approach effectively shifts the developer’s role from a manual labeler to a high-level data architect, focusing on prompt engineering and model optimization rather than clicking mouse buttons thousands of times.

[embed]

Link to the video tutorial here

Download the code for the tutorial here or here

Best AI Photo Tools (Backgrounds, Objects, Headshots)

✅ Phot-AI packs more than 30 AI‑powered tools into one place — covering background and object removal/replacement, image extension and a suite of creative generators for art, icons and logos.

follow the link and start creating : https://phot.ai?ref=eran33

✅ Create and remix stunning AI art and photos with community-driven creativity. tap the link and start creating today! : https://www.remixai.io/?ref=eran

✅ PhotoGPT AI acts as your personal photographer — just describe what you need and the platform generates high‑quality headshots or casual images within minutes.

Its built‑in photo editor lets you remove objects, replace backgrounds and make studio‑quality corrections with a single click.

You can even train your own AI model using a few selfies, receive context‑aware prompt suggestions and upscale images for print‑ready results.

Dive into this all‑in‑one AI photo studio : https://www.photogptai.com/?ref=eran

My Blog

You can follow my blog **here **.

Link to the full post and code here : https://eranfeit.net/create-synthetic-data-for-computer-vision-pipelines/

Want to get started with Computer Vision or take your skills to the next level ?

Great Interactive Course : “Deep Learning for Images with PyTorch” here

If you’re just beginning, I recommend this step-by-step course designed to introduce you to the foundations of Computer Vision — Complete Computer Vision Bootcamp With PyTorch & TensorFlow

If you’re already experienced and looking for more advanced techniques, check out this deep-dive course — Modern Computer Vision GPT, PyTorch, Keras, OpenCV4

This tutorial explains how to build a professional-grade Python pipeline that uses Stable Diffusion to generate high-quality synthetic images and GroundingDINO to automatically label them for object detection and classification tasks.

The process of manual data annotation has long been the most significant bottleneck in developing high-performance machine learning models. This tutorial focuses on a revolutionary shift in the industry: leveraging Synthetic Data for Computer Vision to bypass the tedious weeks spent in labeling software. By combining the generative power of Stable Diffusion with the intelligent labeling capabilities of GroundingDINO, you will learn how to create a self-sustaining data factory that produces training-ready datasets in minutes rather than months.

The value of this workflow lies in its ability to solve the “data scarcity” problem that many developers and researchers face when working on niche or custom object detection tasks. Instead of searching for the perfect dataset or hiring a team of annotators, you gain the freedom to generate high-fidelity images that perfectly match your specific requirements. This guide provides a production-ready Python pipeline that handles everything from environment setup to final dataset exportation, ensuring you can scale your projects without increasing your manual workload.

To achieve this, we bridge the gap between two of the most powerful advancements in AI: Generative Models and Foundation Detection Models. We will start by configuring a Diffusion pipeline to synthesize photorealistic images of specific objects, such as African wildlife, based on text prompts. By controlling the generation process, we ensure that the visual variety — lighting, angles, and backgrounds — is optimized for training robust models that generalize well in real-world scenarios.

Once the images are generated, the pipeline utilizes Autodistill and GroundingDINO to perform zero-shot object detection. This means the system “looks” at the synthetic images and automatically draws precise bounding boxes around the objects based on text definitions you provide. By the end of this article, you will have a complete, labeled dataset ready for YOLO training, having successfully automated the most expensive and time-consuming part of the computer vision lifecycle.

1. Constructing the Virtual Laboratory (WSL & Conda)

Every great experiment needs a controlled environment. To ensure high-speed GPU performance without Windows-related library conflicts, we start within WSL (Windows Subsystem for Linux). This gives us the stability of Linux while keeping our Windows workflow intact.

I use Conda to isolate this specific project. By creating a dedicated environment, we ensure that our “Generative AI” stack doesn’t interfere with our “Detection” stack. It’s about building a clean, reproducible workspace.

### Enter the Linux subsystem for better ML compatibility
wsl

### Create a dedicated Python 3.11 environment
conda create -n diffusers311 python=3.11

### Switch to your new environment
conda activate diffusers311

2. Assembling the AI Toolkit

To make this work, we need three pillars: Generation (Diffusers), Logic (Transformers), and Vision (Autodistill). These libraries represent the cutting edge of the 2026 AI landscape. We aren’t just installing code; we are installing pre-trained “brains” that already understand the world.

We also fix specific versions of OpenCV to prevent the common “GUI display” errors that often plague WSL setups. This curated list of requirements is designed to get you from installation to execution with zero friction.

### Install the generative engines and image processing tools
pip install diffusers[torch]==0.16.1 
pip install transformers==4.28.1 
pip install huggingface-hub==0.14.1 
pip install accelerate==0.19.0
pip install opencv-python==4.13.0.92

### Grab the auto-labeling and detection frameworks
pip install autodistill==0.1.29
pip install autodistill-grounding-dino==0.1.4
pip install roboflow==1.2.15

### Finalize the environment with core data science utilities
pip uninstall -y opencv-python
pip install opencv-python==4.13.0.92
pip install scikit-learn==1.8.0

### Launch your editor to begin the build
code .

3. From Text to Reality: Your First Synthetic Image

This is the “Eureka” moment. We initialize a DiffusionPipeline pointing to a specialized nature model. By leveraging torch.float16, we drastically reduce VRAM usage, allowing high-quality generation even on mid-range GPUs.

The prompt is our “camera lens.” We don’t just ask for an animal; we define the lighting, the depth of field, and the resolution. The negative prompt acts as a quality filter, stripping away common AI artifacts like “plastic textures” or “deformed limbs.”

import os 
from diffusers import DiffusionPipeline
import torch
import cv2
import numpy as np

### Configure the model and target the GPU
animals_models = "VuDucQuang/nature-and-animals"
device = "cuda" if torch.cuda.is_available() else "cpu"

### Load the pre-trained weights with high-performance settings
pipline = DiffusionPipeline.from_pretrained(
    animals_models, torch_dtype=torch.float16 ) 

pipline.to(device)

object_name = "African elephant"

### Define the photorealistic prompt and the negative constraints
prompt = f'Medium-shot of a {object_name}, front view, color photography, photorealistic, hyperrealistic, realistic, incredibly detailed, digital art, crisp focus, depth of field, 50mm, 8k'
negative_prompt = '3d, cartoon, anime, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)) Low Quality, Worst Quality, plastic, fake, disfigured, deformed, blurry, bad anatomy, blurred, watermark, grainy, signature'

### Trigger the generation process
print("Generating image...")
result = pipline(prompt=prompt, negative_prompt=negative_prompt).images[0]

### Convert and save the output for inspection
img = cv2.cvtColor(np.array(result), cv2.COLOR_RGB2BGR)
output_dir = "/mnt/d/temp"
file_path = os.path.join(output_dir, f"{object_name}.png")
cv2.imwrite(file_path, img)

### Open the result to verify visual fidelity
cv2.imshow("Generated Image", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

4. Automating the Dataset Factory (Classification)

One image is a proof of concept; a thousand images is a dataset. In this section, we automate the directory management. The script creates a hierarchical structure, ensuring that every “Lion” or “Leopard” generated has a home that a standard CNN or Transformer model can understand.

This programmatic approach ensures class balance. In the real world, finding 100 perfect photos of a rare animal is hard. Here, it’s just a loop. We are essentially “farming” data with perfect consistency.

Once the single image generation is verified, we can automate the creation of an entire classification dataset. This script iterates through a list of categories — from Lions to Wildebeests — and creates a dedicated folder for each. This organizational structure is exactly what deep learning frameworks like TensorFlow and PyTorch expect for training.

By wrapping our generation logic in a loop, we can create dozens or even thousands of images per category while you walk away from your computer. The code handles the file naming and path management, ensuring that every image is saved in the correct sub-directory. This is the first step in building a Synthetic Data for Computer Vision factory that works for you

import os 
from diffusers import DiffusionPipeline
import torch
import cv2
import numpy as np

### Re-initialize the pipeline for our batch generation task
animals_models = "VuDucQuang/nature-and-animals"
device = "cuda" if torch.cuda.is_available() else "cpu"
pipline = DiffusionPipeline.from_pretrained(
    animals_models, torch_dtype=torch.float16 ) 
pipline.to(device)

### Define the list of animals we want in our classification dataset
categories = ['Lion', 'African Elephant', 'Leopard', 'Rhinocerous', 'Cape Buffalo', 'Cheetah' , 'Giraffe', 'Zebra' , 'Hippo' , 'Crocodile' , 'Wildebeest' , 'Warthhog' ]

### Create the directory structure for our synthetic dataset
base_dir = "/mnt/d/Data-sets/synthetic/Animals-Classification"
os.makedirs(base_dir, exist_ok=True)

### Loop through each category and create a folder for it
for category in categories:
    category_path = os.path.join(base_dir, category)
    os.makedirs(category_path, exist_ok=True)
    print(f"Created directory: {category_path}")

num_images_per_category = 10

### Automatically generate and save the images into their folders
for category in categories:
    for j in range(num_images_per_category):
        print(category + ":Image no." + str(j))
        prompt = 'Medium-shot of a {} , front view, '.format(category) + \
                'photorealistic, hyperrealistic, realistic, incredibly detailed, digital art, crisp focus, depth of field, 50mm, 8k'
        negative_prompt = '3d, cartoon, anime, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ' + \
                        '((grayscale)) Low Quality, Worst Quality, plastic, fake, disfigured, deformed, blurry, bad anatomy, blurred, watermark, grainy, signature'

        ### Generate and save the image directly using PIL
        img = pipline(prompt=prompt, negative_prompt=negative_prompt).images[0]
        file_name = category + str(j) + ".png"
        full_path = os.path.join(base_dir, category, file_name)
        print(f"Saving image to: {full_path}")
        img.save(full_path)

5. Preparing the Ground Truth for Detection

For Object Detection, we need a flat file structure. This script pools all our synthetic creations into a single images directory. This is the "raw input" that our upcoming auto-labeler will feast upon.

By centralizing the data, we make it easier to run batch inference. We are preparing a diverse visual buffet — different species, different angles — all waiting for their bounding boxes to be drawn by the AI.

Object detection requires a slightly different approach than classification. Instead of sorted folders, we usually want all our training images in a single “images” directory, with a corresponding set of label files. This script prepares that “raw” pool of images by generating various animals and saving them into one central location.

By generating a diverse set of images into a single folder, we create the perfect input for our upcoming auto-labeler. We want the images to be high-quality and centered, which makes it easier for the detection model (GroundingDINO) to find them. This phase is about building the “raw material” for your detection model.

This automated generation is a lifesaver for projects involving rare objects or specific viewpoints that don’t exist in standard datasets like COCO or Pascal VOC. You are essentially creating your own custom “mini-universe” of data, perfectly tailored to the needs of your Synthetic Data for Computer Vision detector.

import os 
from diffusers import DiffusionPipeline
import torch

### Initialize the same high-end pipeline for detection image generation
animals_models = "VuDucQuang/nature-and-animals"
device = "cuda" if torch.cuda.is_available() else "cpu"
pipline = DiffusionPipeline.from_pretrained(
    animals_models, torch_dtype=torch.float16 ) 
pipline.to(device)

### Define our list of targets for the detection task
categories = ['Lion', 'African Elephant', 'Leopard', 'Rhinocerous', 'Cape Buffalo', 'cheetah' , 'Giraffe', 'Zebra' , 'Hippo' , 'Crocodile' , 'Wildebeest' , 'Warthhog' ]

### Create a single folder to hold all images for the detection pipeline
base_dir = "/mnt/d/Data-sets/synthetic/Animals-Object-Detection/images"
os.makedirs(base_dir, exist_ok=True)

num_images_per_category = 10

### Loop through categories and save all generated images to the same folder
for category in categories:
    for j in range(num_images_per_category):
        print(category + ":Image no." + str(j))
        prompt = 'Medium-shot of a {} , front view, '.format(category) + \
                'photorealistic, hyperrealistic, realistic, incredibly detailed, digital art, crisp focus, depth of field, 50mm, 8k'
        negative_prompt = '3d, cartoon, anime, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ' + \
                        '((grayscale)) Low Quality, Worst Quality, plastic, fake, disfigured, deformed, blurry, bad anatomy, blurred, watermark, grainy, signature'

        ### Generate the image and save it with a unique name
        img = pipline(prompt=prompt, negative_prompt=negative_prompt).images[0]
        file_name = category + str(j) + ".png"
        full_path = os.path.join(base_dir, file_name)
        print(f"Saving image to: {full_path}")
        img.save(full_path)

6. The End of Clicking: Zero-Shot Auto-Labeling

This is the crown jewel of the pipeline. We use GroundingDINO, a model that understands language as well as pixels. We define an “Ontology” — a map that tells the AI “If you see a large gray animal with a trunk, label it ‘African Elephant’.”

With one command, the base_model.label() function scans every image and writes the annotation files ($x, y, w, h$). No human intervention. No manual errors. Just a professional, training-ready dataset generated while you grab a coffee.

This is the peak of the tutorial. We define a CaptionOntology, which maps our animal names to the labels we want the computer to use. We then initialize the GroundingDINO model, which is a powerful “foundation model” that can detect almost anything described in text without being specifically trained on it.

The base_model.label() command is where the magic happens. The script scans your entire "images" folder, identifies the animals using the ontology, and automatically writes the bounding box coordinates into a new "dataset" folder. This effectively replaces hundreds of hours of manual clicking with a single command that runs in the background.

By the time this script finishes, you have a professional, training-ready dataset in standard formats (like YOLO). This is the true power of Synthetic Data for Computer Vision: you can go from an idea to a fully labeled dataset without ever manually labeling a single image. You are now a data architect, not a manual annotator.

import torch 
import os 
from autodistill.detection import CaptionOntology
from autodistill_grounding_dino import GroundingDINO 

### Define the mapping between what the AI "sees" and your dataset labels
ontology = CaptionOntology({
    "Lion": "Lion",
    "African Elephant": "African Elephant",
    "Leopard": "Leopard",
    "Rhinocerous": "Rhinocerous",
    "Cape Buffalo": "Cape Buffalo",
    "Cheetah": "Cheetah",
    "Giraffe": "Giraffe",
    "Zebra": "Zebra",
    "Hippo": "Hippo",
    "Crocodile": "Crocodile",
    "Wildebeest": "Wildebeest",
    "Warthhog": "Warthhog"
})

### Set the confidence thresholds for detection and text matching
BOX_THRESHOLD = 0.3
TEXT_THRESHOLD = 0.3

### Define the input image folder and the target output directory
base_dir = "/mnt/d/Data-sets/synthetic/Animals-Object-Detection/images"
DATASET_DIR_PTH = "/mnt/d/Data-sets/synthetic/Animals-Object-Detection/dataset"

### Initialize the GroundingDINO base model with our ontology
base_model = GroundingDINO(ontology=ontology, 
                           box_threshold=BOX_THRESHOLD,
                           text_threshold=TEXT_THRESHOLD)

### Run the labeling process to generate a complete, annotated dataset
dataset = base_model.label(input_folder=base_dir, extension=".png", output_folder=DATASET_DIR_PTH)

FAQ

Q: What is synthetic data for computer vision? A: It refers to images generated by AI models (like Stable Diffusion) specifically for training machine learning models. It allows you to create balanced datasets even when real-world photos are difficult or expensive to obtain.

Q: Why use GroundingDINO for auto-labeling? A: GroundingDINO is a “zero-shot” detector, meaning it can find objects based on text prompts without needing prior training on those specific objects. This makes it perfect for annotating synthetic images automatically.

Q: Do I need a powerful GPU to run this code? A: Yes, a modern NVIDIA GPU with at least 8GB of VRAM is highly recommended. Stable Diffusion and GroundingDINO are computationally heavy and will be extremely slow on a CPU.

Q: Can I use this for non-animal datasets? A: Absolutely. You can change the prompts and the ontology to detect anything from industrial parts and medical anomalies to specific types of vehicles or furniture.

Q: Is synthetic data as good as real data? A: While real data is ideal, synthetic data is excellent for pre-training or augmenting real datasets. Modern generative AI produces images realistic enough to teach models the core features of an object effectively.

Q: What is Autodistill? A: Autodistill is a framework that uses large foundation models (like GroundingDINO) to “teach” smaller, faster models (like YOLOv8) by automatically labeling data for them.

Q: Why do we use WSL for this tutorial? A: WSL provides a native Linux environment on Windows, which is much more stable for AI libraries like PyTorch and Diffusers, which are often developed primarily for Linux systems.

Q: What does the BOX_THRESHOLD do? A: It controls how “sure” the model must be before it draws a bounding box. A higher value leads to fewer but more accurate detections, while a lower value finds more objects but may include errors.

Q: How many images should I generate? A: For basic classification, 100–200 images per class is a good start. For complex object detection, you might want 500+ images per category to ensure the model generalizes well.

Q: Can I run this on Google Colab? A: Yes, the code is Python-based and will work on Colab as long as you have a GPU runtime active. You will just need to adjust the file paths to save to Google Drive.

Summary: The Future is Automated

By combining Generative AI with Foundation Models, we’ve bypassed the most expensive part of machine learning development. We went from a blank screen to a fully labeled, multi-class object detection dataset in a single Python session. This is the power of Synthetic Data for Computer Vision — it levels the playing field, allowing any developer to build world-class models without a massive labeling budget.

Connect :

☕ Buy me a coffee — https://ko-fi.com/eranfeit

🖥️ Email : feitgemel@gmail.com

🌐 https://eranfeit.net

🤝 Fiverr : https://www.fiverr.com/s/mB3Pbb

Enjoy,

Eran