Data Formats for Token Efficiency in AI Systems

How data formats evolved from system integration to AI-driven efficiency

Photo by Markus Winkler on Unsplash

The Evolution of Data Formats

Data formats have quietly shaped every era of software architecture. Whether it was the use of XML to create SOAP-based services or the adoption of JSON (JavaScript Object Notation) to build RESTful architectures, each transition was motivated by simplicity, performance, and developer efficiency.

But today, as we move into an AI-first world, a new constraint is emerging: token efficiency. For Large Language Models (LLMs), any choice of data format will affect your application's cost, latency, and scalability.

This document provides an overview of the evolution of data formats — from XML to JSON and now TOON. It will be particularly useful for developers who have primarily worked with JSON or are beginning to work with TOON.

XML → JSON

I remember the early days of Web services, when the Simple Object Access Protocol (SOAP) used XML to exchange information. However, XML has its own challenges, like being complex to read due to tags and attributes, and overall complexity, which results in larger payloads. Refer below example

<?xml version="1.0" encoding="UTF-8"?>
<orders>
    <order>
        <orderId>ORD123</orderId>
        <customerName>Rahul</customerName>
        <product>Laptop</product>
        <quantity>1</quantity>
        <price>75000</price>
        <totalPrice>75000</totalPrice>
    </order>
    <order>
        <orderId>ORD124</orderId>
        <customerName>Amit</customerName>
        <product>Mouse</product>
        <quantity>2</quantity>
        <price>1500</price>
        <totalPrice>3000</totalPrice>
    </order>
</orders>

Then came the days of REST APIs, during which JSON became widely used due to its simple key/value structure, lightweight nature, and ease of parsing.

What about other data formats like YAML and Protobuf?

YAML is designed for human readability and is widely used in configuration files (e.g., Kubernetes, CI/CD pipelines). In LLM scenarios, YAML can sometimes increase token usage due to whitespace and formatting.

Protocol Buffers (Protobuf), developed by Google, are designed for high-performance data exchange. Protobuf is not human-readable and requires encoding/decoding, making it unsuitable for direct interaction with LLMs.

TOON (Token-Oriented Object Notation)

Today, in the era of AI, we frequently send JSON as input to LLMs. The cost of calling an LLM depends on the number of tokens processed. More tokens mean a higher cost. JSON, with its repeated keys, quotes, braces, and commas, can generate many tokens — especially for large datasets.

TOON is a new compact, human-readable data serialization format used to reduce the number of tokens while exchanging structured or JSON-like data with language models. Let’s take the JSON example below:

{
  "orders": [
    { "orderId": "ORD123", "customerName": "Rahul", "product": "Laptop", 
      "quantity": 1, "price": 75000, "totalPrice": 75000 },
    { "orderId": "ORD124", "customerName": "Amit", "product": "Mouse", 
      "quantity": 2, "price": 1500, "totalPrice": 3000 }
  ]
}

Tokens are essentially sub-words, and many punctuation marks are also treated as separate tokens. In JSON, the number of tokens increases due to repeated keys in a collection of order objects, as well as additional elements such as quotes, braces, and commas. The cost of these tokens in a JSON response is low, often just a micro-fraction of a cent in bandwidth. However, they can significantly increase LLM usage costs. To enhance data sharing with LLMs, TOON has been introduced. It compresses data structures and offers a more efficient way to share information. Below is the resultant TOON format of the above JSON:

orders[2]{orderId,customerName,product,quantity,price,totalPrice}
  ORD123,Rahul,Laptop,1,75000,75000
  ORD124,Amit,Mouse,2,1500,3000

I used the online JSON-to-TOON converter tool, available at https://lnkd.in/dRVds_yv. This tool gives me the result below:

TOON tokens ≈ 24 JSON tokens ≈ 37

This is a 35.1% reduction in tokens. You can experiment with your own data to see how much token reduction you can achieve.

Since TOON values depend on a predefined column structure and their position in that structure, avoid manual rule validation. At minimum, validate that the row and column counts match the schema, that any required fields are present, that there are no extra columns, etc. Below are the differences between these commonly used data formats.

XML vs JSON vs TOON

Understanding When to Use and Avoid TOON

TOON can be considered when your system is moving from traditional API communication into AI-first architectures, where token efficiency matters. The places where we can use TOON are large LLM prompts and feeding logs to AI agents.

Please note that TOON is not a replacement for JSON. Hence, we can avoid TOON in the following places where JSON is widely used and more suitable in public facing API’s, logging, debugging, tooling, mobile/web clients, and third-party integrations.

Best Practices for using TOON

TOON can be used efficiently when working with LLMs, especially when minimizing token usage. This will enable one to reduce token usage and minimize costs.

However, for general-purpose programming or standard APIs, you can use JSON or another standard format to communicate between programs or systems. The new TOON concept is better suited for introduction when interacting with LLMs.

While working with TOON, keep column order consistent and explicitly declared. e.g., orderId, customerName, product, quantity, price, and totalPrice columns are clearly defined in the above structure.

Conclusion

TOON does not replace JSON. TOON is a complement to JSON. Use JSON when building public APIs and third-party integrations. Use TOON when working with LLMs, AI agents, or large structured datasets where minimizing the number of tokens has a direct impact on cost, performance, and scalability. The process of transforming JSON to TOON and vice versa increases the complexity of processing and testing.

Learn More / Tools

Refer to the official TOON GitHub repository: https://github.com/toon-format/toon.

For hands-on experimentation, you can also use JSON → TOON online converter: https://scalevise.com/json-toon-converter