Virtuquill: Why not Protocol Buffers?

My new app, Babel Class Builder, is a low-code option for storing flat file definitions and generating classes in multiple programming languages to read and write the data. A recent visitor asked whether we are using Protocol Buffers as our the “intermediate format” for coding the classes. The simple answer is “no”, but keep reading if you’d like to know why.

First off, Protocol Buffers (or protobufs) “are a language-neutral, platform-neutral extensible mechanism for serializing structured data.” This aligns with the Babel Class Builder’s goal. They support several data mapping and validation goals like using enums, plus tagging required, optional, and repeating fields. However, protobufs primarily focus on the efficiency of storing strongly-typed and structured data transmitted as message. For example, a string of data like “T2023 999999999” could be defined as :

message Transmitter {
  string rec_type = 1;
  int32 payment_year = 2;
  bool prior_year = 3
  string transmitter_tin = 4;
}

At first glance, this could describe the beginning of the IRS electronic tax filing Transmitter record.

This mapping aligns with the Babel Class Builder’s goal, but protobufs primarily focus on the efficiency of storing strongly-typed and structured data. Protobufs allow for using LEN bits in reading a fixed amount of data in “over the wire”, so a parser could determine that the first field is “T”, not “T2023”, or “T2023 999999” et cetera. However, serializing this data requires customized transformations for writing the data into fixed width, justified fields with differing pad characters. The prior_year field is logically a boolean, but as expressed in the IRS Transmitter record, “P” means true and space or empty indicate false.

Additionally, Babel Class Builder focuses more on the preservation of hierarchy and formatting rules of the data. To continue with our IRS example, the Pub 1220 file specification is structured with several nested “envelopes” opened by a header record, closed by a trailer record and filled with one or more records / envelopes within them.

In general, the record types are determined by their first character, “T”, “A”, “B”, “C”, “K”, or “F”. These identify “Transmitter”, “Issuer”, “Payee”, “Issuer summary”, “State Totals” and “End of File”, respectively. That’s relatively straightforward. However, the “B” records report different information in varying layouts based on which IRS Form you are reporting. A 1099-INT form for interest received is different than a 1098 form reporting Mortgage Interest paid. In a vacuum, one cannot determine whether a given “B” record represents a 1098 or a 1099-Int (or a 5498 or any of the dozens of other forms). To correctly identify the type of the “B” record, a parser needs to know not only that it is the child of an “A” record, but that parent “A” record also contains a specific code in positions 26–27:

This is not a task for which protobufs are well-suited. Babel Class Builder was purpose-built to address the issues of:

• Hierarchical, nested data with complex identifiers
• Fixed Width input / output
• Value Lists & Enumerations
• Non-standard Boolean values
• Field-specific padding and justification rules

I hope this whets your appetite to check out Protocol Buffers and Babel Class Builder, depending on your use case.