Simulating Reality: Why Your Tests Need Real APIs, Not Just Fake Strings.

Ever felt like you’re playing a high-stakes game of Jenga with your test data? One wrong move, one invalid email format or a missing dependency, and the whole tower comes crashing down.

As engineers, we live and breathe tests. They’re our safety net and confidence booster, ensuring that every line of code we push doesn’t accidentally break something critical. But here’s the catch: writing good tests, especially end-to-end (E2E) tests, often means tackling something far more challenging than the code itself: data for testing. When data setup is manual and brittle, it becomes a massive drag on software implementation velocity. Instead of shipping features, we’re stuck debugging environments, effectively trading our development speed for data maintenance.

Mocking data for testing quickly became a bottleneck — a tedious, error-prone exercise that drained precious time from actual feature development. Strict validation rules meant that critical fields, from contact details to financial identifiers, had to be perfect; a single malformed email address like myEmail.com instead of myEmail@gmail.com could halt an entire test suite. Crucially, we didn't just need valid individual fields; we needed to create these entities at scale while maintaining the complex, interconnected relations between them—all without compromising the integrity of our testing environments.

Beyond Basic Mocks

At Melio, we decided to tackle this data friction head-on by rethinking our entire approach to test orchestration. Our system, like many modern financial platforms, relies on a web of interconnected entities — users, vendors, transactions, and bank accounts — each bound by strict business logic. We realized that simple, static mocks weren’t enough.

We needed a tool that truly understood our business domain and could orchestrate the creation of these entities exactly as our production system does. It wasn’t just about generating data; it was about simulating state. This led to the birth of Smoker (Simulator + Mocker 🤝), a tool designed to mirror reality within our test environments.

The Power of Data Simulation

If you’re a software developer or a QA engineer drowning in manual data setup or battling inconsistent test data, you need a system like Smoker that offers a lifeline:

• Reliable & realistic test data: Say goodbye to tests failing because of invalid emails or malformed IDs. You can generate data that looks and feels real, adhering to your system's constraints.
• Accelerated test setup: Quickly spin up complex, multi-entity scenarios with a single command, dramatically cutting down on test preparation time.
• Confidence in your tests: Trust that your E2E tests are validating your business logic against valid data, reflecting real-world conditions.
• Focus on what matters: Spend less time crafting mock data and more time building and testing your core features.
• Effortless scenario reproduction: Easily recreate specific data states to debug tricky bugs or validate complex workflows.
• Clean up your data easily: You have the ability to monitor all the entity IDs that were created and easily clear them from your DB whenever you want.

Our Solution: A Lambda-Powered Data Orchestrator

During the design phase, I prioritized the most straightforward and reliable architecture for our needs (though the best approach always depends on your specific resources and timeline). I opted for a central service — yes, a monolith, which is often exactly what an orchestrator requires — to interact with our various internal APIs. To handle asynchronous processes where entities take time to materialize, the service implements a robust polling loop. This ensures each entity is successfully created and verified in the database before the orchestrator moves to the next step in the simulation.

Throughout this entire flow, leveraging a package like faker.js is paramount for generating near-real data, providing the foundational realism for our intelligent mocks.

Here’s how we built it and how it works:

The Architecture: A Client-Server Duo

The Smoker system is divided into two main parts:

1. The Client (CLI Tool): This is a Node.js-based command-line interface. It’s the user’s entry point. When you want to create data, you interact with this CLI. For example, you might run smoker create-vendor --count 10. The client's job is to take your request, format it, and invoke the smoker-api Lambda function.
2. The Server (AWS Lambda — smoker-api): This is where the magic happens. Its primary responsibility is to orchestrate the creation process by calling our existing internal APIs. This is a crucial design decision: it’s the ultimate form of dogfooding. Instead of Smoker bypassing the system to write directly to the database, it uses the same pathways as our users. This ensures that every validation rule and side effect is triggered, proving that our own internal tools can “eat” the same logic we serve to our customers. This ensures that:

• Business logic is respected: All existing validations, data transformations, and side effects within our core services are naturally applied.
• Data consistency is maintained: Interconnected entities are created and linked by the services themselves, preserving referential integrity and complex relationships.

Inside the Lambda: Simulators and Smart Mockers

Within the smoker-api Lambda, we have two key concepts:

• Simulators: Each simulator is responsible for a specific data generation workflow. For instance, CreateVendorsSimulator handles the creation of vendor entities and its connected entities that should be created as a prerequisite to vendor creation.
• Smart mockers: This is where we got clever with our data generation. While we leverage faker.js for basic data patterns, our custom smart mockers for properties like emails, bank accounts, or tax IDs encapsulate our specific business rules and constraints.

This approach ensures that every piece of data created by Smoker passes through our actual system's pipelines, guaranteeing its validity and consistency, just as if a real user had created it.

From Manual Build to AI Blueprint

While building Smoker was a rewarding engineering challenge, the rise of AI has changed the game. Today, you don’t need to spend days or weeks architecting these orchestrators from scratch. If you provide an AI agent with your system’s API documentation and database schemas, it can act as a co-pilot to generate the entire simulation framework for you.

Here’s how I envision building a data simulation system with AI tools and a potential prompt for an AI agent:

AI’s Role in Data Simulation:

1. Schema understanding & constraint inference: LLMs excel at understanding complex structures. Provide your AI tool with OpenAPI specs, database schemas, and even existing code to automatically infer data types, relationships, and implicit business constraints (e.g., “email fields usually follow a specific regex,” “IDs are UUIDs”).
2. Smart mocker generation: Instead of manually writing EmailMocker, BankAccountMocker, etc., an AI could generate these based on inferred constraints and common data patterns. For example, after scanning your code and encountering a field called emailAddress, it should know to generate something like faker.internet.email().
3. Simulator orchestration (workflow generation): This is where AI could truly shine. Given a high-level goal (e.g., “create a fraudulent vendor entity scenario”), an AI could:

• Identify necessary entities
• Determine creation order and dependencies
• Infer API call sequences
• Suggest “unhappy path” scenarios

The “Build My Simulation Engine” AI Prompt:

Copy and paste this into an AI tool to start building:

---
  Test Data Generator - Interactive Setup Prompt

Help me create a test data generation service for my project. This tool will generate realistic mock data
  and persist it to my storage layer for testing purposes.

Guide me through the setup by asking these questions one section at a time:

---

## 1. Project Basics
  - What should this project/service be called?
  - What programming language? (TypeScript, Python, Go, Java, C#, Rust, etc.)
  - What's the target runtime version? (e.g., Node 20, Python 3.11, Go 1.21)

## 2. Data Schema
  Ask me to provide my schema in ONE of these formats:
  - OpenAPI/Swagger spec (file path or URL)
  - Database schema (Prisma, SQL DDL, SQLAlchemy models, GORM structs, etc.)
  - Type definitions (TypeScript interfaces, Python dataclasses/Pydantic, Go structs, etc.)
  - GraphQL schema
  - JSON Schema
  - Protobuf definitions
  - Or I'll describe my entities manually

## 3. Data Storage
  - What's my primary data store? (PostgreSQL, MySQL, MongoDB, DynamoDB, CosmosDB, Redis, Elasticsearch,
  Neo4j, Firestore, Supabase, PlanetScale, etc.)
  - What client/ORM am I using? (Prisma, TypeORM, SQLAlchemy, GORM, Entity Framework, Diesel, raw client,
  etc.)
  - Any secondary storage or external APIs that need mock data?

## 4. Deployment Target
  - How will this run?
    - **Serverless**: AWS Lambda, Azure Functions, GCP Cloud Functions, Vercel, Cloudflare Workers
    - **Containers**: Docker, Kubernetes, ECS, Cloud Run
    - **Server**: Express, FastAPI, Gin, Spring Boot, ASP.NET
    - **CLI tool**: Local execution, CI/CD pipeline
    - **Other**: Describe your setup

## 5. Simulation Scenarios
  Based on my entities, ask me:
  - Which entities need bulk creation?
  - What are the typical test scenarios?
  - What entity relationships must be maintained?
  - What validation constraints exist? (counts, required fields, business rules)

## 6. Environment & Configuration
  - What environments do I need? (local, dev, staging, prod, etc.)
  - How do I manage configuration? (env files, AWS SSM, HashiCorp Vault, config files, etc.)
  - Any secret management requirements?

## 7. Existing Infrastructure (optional)
  - Do I have existing client libraries or SDKs?
  - Existing logging framework? (Winston, Pino, Logrus, Zap, Serilog, Log4j, etc.)
  - Code style/linting config to follow?
  - Monorepo or standalone project?

---

After gathering inputs, generate a project following these principles:

### Architecture Pattern
  src/ (or appropriate language convention)
  ├── handlers/          # Entry points (Lambda/HTTP/CLI)
  ├── simulators/        # Simulation logic (validate + execute)
  ├── services/
  │   ├── generators/    # Fake data generators
  │   └── persisters/    # Orchestrate generation + storage
  ├── data-access/       # Storage layer abstraction
  ├── types/             # Type definitions & custom errors
  └── utils/             # Logger, config, helpers

### Core Pattern (adapt syntax to chosen language)
  Simulator interface/trait/protocol:
  - validate(request) → throws/returns error if invalid
  - run(request) → executes simulation, persists data

Generator functions:
  - Use language-appropriate faker library
  - Return properly typed mock objects

Persister functions:
  - Call generator to create mock data
  - Save to storage layer
  - Return saved entity with generated IDs
  - Log created resources

### Faker Libraries by Language
  - **TypeScript/JavaScript**: @faker-js/faker
  - **Python**: Faker
  - **Go**: gofakeit
  - **Java**: JavaFaker, DataFaker
  - **C#**: Bogus
  - **Rust**: fake-rs
  - **Ruby**: Faker
  - **PHP**: FakerPHP

### Key Requirements
  - Strongly typed (no type escape hatches like `any`, `object`, `interface{}` unless necessary)
  - Input validation with sensible bounds
  - Custom error types/exceptions
  - Structured logging with entity IDs
  - Environment-based configuration
  - README with setup and usage instructions

Conclusion

Ultimately, Smoker proved that the best way to move fast is to build solid foundations. By investing in our data infrastructure, we reclaimed countless hours previously lost to environment debugging. With the added power of AI, the barrier to building these tools has never been lower. Don’t let your data be the bottleneck — simulate reality and get back to shipping.

visit our career website