Introducing JsonT: The Schema-Driven Data Language for Scale

JSON is excellent. But at scale, it’s expensive

JsonT Engine — Conceptual View

We all love JSON. It’s the lingua franca of the web—readable, flexible, and supported everywhere. But if you have ever built a high-volume data pipeline, a batch ingestion system, or a low-bandwidth mobile API, you know the hidden costs of JSON:

• Redundancy: Why are we sending the string "customer_id" million times in a single batch file?
• Ambiguity: Is that field optional? Is it a string or a number? You won't know until the parser crashes.
• Validation Hell: You end up writing the same validation logic in your producer, your consumer, and your database

Enter JsonT

What is JsonT?

JsonT (JSON-Typed) is a schema-driven, positional data language designed for compact, validated, and efficient large-scale data transfer.

Think of it as the child of JSON and CSV, raised by Protocol Buffers. It keeps the hierarchical structure of JSON but strips away the repetitive field names by using a strict schema and positional tuples.

How it works?

The magic of JsonT lies in separating the structure from the data.

In standard JSON, a list of users looks like this:

[
  { "id": 1, "name": "Alice", "role": "admin" },
  { "id": 2, "name": "Bob", "role": "user" }
]

In JsonT, we define the structure once in a schema, and then stream the data as compact tuples

1. The Schema

{
    schemas: {
        User: {
            int: id,
            str: name,
            str: role?  // The '?' marks this field as optional
        }
    }
}

2. The Payload

{
    data-schema: User,
    data: [
        { 1, "Alice", "admin" },
        { 2, "Bob", "user" }
    ]
}

The Result?

• Smaller payloads for large datasets.
• Zero ambiguity about data types.
• Zero redundancy in field names.

Key Features

• Massive Compression: By removing repeated keys, JsonT achieves payload sizes comparable to binary formats like BSON or Protobuf, but remains human-readable.
• Built-in Validation: Validation isn’t an afterthought; it’s baked into the parser. If a payload parses, it is valid. Constraints like types (int, str), nullability, and optionality are enforced by the language.
• Streaming Friendly: Designed for modern architecture, JsonT supports incremental validation. You don’t need to load a 1GB file into memory to parse the first record.
• Type-safe: It borrows strict typing concepts from Protobuf and Avro, ensuring that a number is always a number and a boolean is always a boolean

When do you use JsonT?

JsonT isn’t trying to kill JSON. JSON is still perfect for config files, small payloads, and ad-hoc debugging. However, JsonT shines where JSON struggles:

• Large Batch APIs: When you are transferring thousands or millions of records.
• Bandwidth-Constrained Environments: IoT devices, mobile apps on patchy networks, or expensive satellite links.
• Data Lakes & Analytics: Storing massive amounts of structured event logs.
• Strict Contracts: When the producer and consumer need a hard guarantee on the data shape

Getting Started

JsonT is open source and currently supports Java. You can easily integrate it using Maven

<dependency>
    <groupId>io.github.datakore</groupId>
    <artifactId>json-t</artifactId>
    <version>0.0.1</version>
</dependency>

Parsing data is as simple as defining your adapter and streaming the content

JsonTContext ctx = JsonT.builder()
    .withAdapter(new UserAdapter())
    .parseCatalog(Paths.get("schema.jsont"));

// Stream massive datasets efficiently
ctx.withData(dataStream).as(User.class).stream()
    .subscribe(user -> System.out.println("Received: " + user.getName()));

Join the project

JsonT is currently in active development. We are building a future where data transfer is efficient, strictly typed, and developer-friendly.

We are looking for contributors to help with:

• Rust and TypeScript implementations.
• IDE tooling and syntax highlighting.
• Performance benchmarking.

Check out the repo, give it a star ⭐, and let’s make data transfer better together.

👉 Refer: **GitHub: datakore/json-t**