Open Standard for Benchling Configuration Management

Configuration and change management form the bedrock of all digital platforms. Certainly this is true for systems that need to meet regulatory standards (think GxP, 21 CFR Part 11, ISO/IEC 20000) but this equally applies to any solution that users need to trust for its quality, accuracy, and reliability. This is such a fundamental principle that there are over a billion git repos worldwide.

Why then is it okay for us to not manage the Benchling configurations for the platforms that underlie the work of many tens of thousands of scientists and billions of dollars in investments — platforms that ultimately deliver medicines to improve our collective health?

The current state of configuration management in Benchling is that admins configure by clicking through the UI and then rely on the current deployed state to represent the design. This is configuration, not configuration management. And without configuration management, we don’t have real change management.

The stakes are high. The lack of rigor that we expect from all other data platforms risks lower quality data capture and integrity, missed insights, longer time to insights, reduced productivity, increased costs, failed audits (when applicable), broken integrations, and higher friction to making our data available for AI and ML.

How do you implement configuration management on Benchling?

• “It is what it is” — the current state of Prod is the system of record
• Document in Word or Excel documents
• Maintain an exact representation of your configuration in a structured data format that is version controlled and available for collaborative peer review

I’ve worked my way through these stages of maturity over the years with Benchling. I started by clicking around and getting stuff done. Even though that was the norm (and often still is) at the time, my internal engineer klaxon was blaring at me.

I then quickly adopted a series of process improvements over various client engagements experimenting with different standardized formats in Word and Excel. Each was an improvement over the previous, but still wholly insufficient and inefficient. Like a paper lab notebook, these documents did record what was done at a point in time, and that’s a start, but they did not support many of the attributes that are needed for proper configuration management in the same way that a paper notebook is not a good basis for an AI drug discovery strategy. There’s a reason no one records their AWS infrastructure configuration as a library of Word docs! I’m very disciplined — nonetheless, like a car loosing 10% of its value as soon as it drives off the lot, these documents quickly drifted from the actual implementation and worse, it was not possible to know to what extent.

Where are you in this journey?

What if we all jump to the next maturity level and treat the Benchling configuration like any other professionally managed data platform?

Let’s first define our goals for a good configuration management recording format that is high on the maturity curve:

• Human and machine readable
• Structured
• Version controllable
• Accurate
• Complete
• Unambiguous
• Reviewable
• Traceable
• Shareable

To address these needs I created a structured specification, dubbed Kenfigure™, that I’ve been using with my clients for over 2 years with extraordinary success. Since this is a common problem across all Benchling admins, I’ve now published the Kenfigure specification on GitHub (using an Open Data Commons Attribution License — meaning free to use).

The following sections will provide an overview of what Kenfigure is and some of the myriad benefits for adopting it.

What is Kenfigure?

Kenfigure is a YAML specification that defines all aspects of a Benchling tenant’s configuration. (For those who are not familiar with YAML, it’s a text file with a simple defined structure — see the example below). The specification includes a publicly hosted JSON Schema which formally defines the structure of the YAML files and can be used by IDEs (VS Code, Cursor, PyCharm, etc.) to automatically validate the contents and provide typing completion and inline documentation.

A full tenant description is represented by a folder tree of YAML files (a “kenfiguration™”; sorry I don’t get out much). The files can be in any structure to help keep objects organized by object type, scientific function/grouping, or a combination. The entire tree (including supplemental artifacts and documentation as needed) should be managed in a git (or other) version control repository.

Here’s an annotated example of molecule entity called Chemical.

Entity_schemas:
- Name: Chemical
  Description: A generic small molecule.
  Entity type: Molecule
  Prefix: CHEM
  System name: chemical
  Containable type: None
  Name template: []
  Constraint:
  - Canonical SMILES
  - CAS RN
  Naming options:
  - NEW_IDS
  # - Generate new registry IDs  # Long form is accepted, but I prefer the short form
  RegID chips: false
  RegID display: false
  Icon: small-molecule
  Access type: Registry-based
  Diagram:
    Group: Chemicals  # All entities with the same name are rendered in a group of that name
  Fields:
  - Name: CAS RN
    Description: If we don't have a SMILES, then a real or pseudo CAS RN drives uniqueness
    Tool tip: Create a pseudo CAS RN if no official number exists
    System name: cas_rn
    Type: Text

Check out Kenfigure.com for the full details including the annotated documentation and the full specification.

What this enables

Kenfigure is much more than a way to document a Benchling configuration. It’s a powerful enabler of multiple quality and productivity practices and tools. I’ll touch on a subset here.

Git management

With git management, you can

• Draft changes and review differences before committing
• Create feature branches and circulate for reviews and approvals
• Run automated style, rule, or impact checks using using CI/CD pipeline tools (e.g., GitHub Actions, GitLab Runners)
• Get full traceability, auditability, history, and even rollback if needed
• Maintain discrete branches, one for each tenant
• Collaborate with other contributors (view kenfigurations and even create pull requests or merge requests)
• Collaborate with read-only stakeholders (e.g., data science, computational teams, AI/ML teams, dashboard builders) so they have a definitive definition of the data model and the system names needed for SQL queries.
• Create release notes from commit logs for better change communication

Productivity

The format lends itself to efficient editing in an IDE. Some notable examples include:

• Clone schema files or copy & paste large sections (Benchling configurations often have a lot of similar contents that is replicated with minor variations)
• Search across all information in one search. This includes searches across schema types, all fields, documentation, etc.
• Typing completion and automatic validation saves time and maintains quality
• AI integration (e.g., GitHub Copilot, Cursor) can generate fields or entire sets of fields with just the press of the tab key. AI can also provide excellent suggestions for tooltips.
• Global search and replace (sometimes we need to make broad changes due to shifts in the modeling or due to an M&A.

Documentation

There are multiple places where you can add documentation beyond what is stored in Benchling. This helps to capture why things were done, tradeoffs, caveats, todos, notes for the future and other information helpful to the long-term maintenance of the environment. Effective use of these capabilities is a powerful hedge against staff turnover so that institutional knowledge sticks with the platform and doesn’t evaporate upon inevitable staff or contractor turnover.

• Add documentation files (markdown, text, Word, Excel, etc.) to any part of the directory tree. This is useful for capturing theory of design documents, stakeholder requirements, and other such notes
• Add comment blocks to the top of files or as inline comments
• Use the Description keyword to document the top-level object (e.g., Dropdown, Entity schema, etc.) or at the individual field level.

Other cool stuff

All the above delights me to no end daily (one reason I’m eager to share this). I should probably wrap this up, but I need to share just a few more benefits that are enabled by Kenfigure.

Ken on the Tron Lightcycle Power Run roller coaster… or am I in the grid? In either case, I look happy with Kenfigure.

AI assistance

As noted above, out of the box VS Code + GitHub Copilot and Cursor (others too, but I haven’t tried them) do a great job with typing completion and hover hints. You also can get help with bulk editing (it often does a good job anticipating the next change or helping with bulk changes) and tooltip generation. However, I literally jumped out of my chair and yelled to my family to come look (they didn’t care truth be told, but that didn’t stop me) when I ran my first full tests after providing a deliberate set of rules/prompts to guide the Cursor on how to answer questions about the data model. This include the mundane like “tell me if this proposed entity prefix is unique and visually distinct from other prefixes” to more elaborate queries like “list all entities that directly relate to LNP production and generate an ER diagram for just that set”. I could go on, but this will have to wait until a separate post. I do plan to publish my Cursor rules, so stay tuned!

Impact assessment

At one of my clients with a lot of software integrations we require two approvals or all Kenfigure merge requests, one from the Benchling platform team and one from the software team. The goal of the software review is to assess if any proposed changes might require a coordinated change to any of our integrated software services. This works well since the merge request shows exactly what’s changing. However, this is still a manual review. An awesome engineer on the team suggested that we automate the analysis during a team meeting. By the afternoon he had added a Python script to the CI/CD pipeline to compare the changes in the merge request to our codebase and then issue an email if any potential software impacts were detected. He even automatically added a comment with the impact notice into the merge request comment stream. Cool.

Diagrams

Because we have a well-structured format, I’ve been able to create a tool to automatically create an ER diagram from a kenfiguration. For me (I’m a visual thinker) this is a game changer for my ability to keep track of some very complex and dynamic systems. The fact that this is easy means the diagram is always up to date (can also be part of the CI/CD pipeline). My implementation drives a commercial database tool for the rendering (which is interactive and has all details of the kenfiguration) and is also able to output an interactive HTML file that I can share with data science and AI/ML stakeholders.

Infrastructure-as-code

I’ve created a tool (kenfigure-tool) that can extract from Benchling into Kenfigure format, or… wait for it…, can import a kenfiguration to Benchling!

I just heard you say “WHAT!!! This is the holy grail of Benchling administration. Why did you bury the lead, Ken!?”. Well, here’s the story. The purpose of this article is to describe the open Kenfigure standard. Without any additional automation, it is incredibly powerful and accessible. I honestly believe that every Benchling customer should be using it. Kenfigure-tool is a tool I use in my consulting practice (a substantial quality and productivity amplifier) but not generally available (so far). It’s important to mention its existence since it demonstrates a key aspect (among many) of the value of Kenfigure, but beyond that, the attention should be fully on Kenfigure.

Conclusion

Benchling configurations are complex and ever changing. As Benchling platform owners (admins) it’s our responsibility to manage our platforms to the same standards as any other modern data platform. This is especially critical considering the incredibly central role that Benchling plays in recording our scientific data and making it available for analysis and eventually regulatory filings or scientific publications.

Kenfigure is an open standard that takes almost no effort to start using and yet reaps tremendous and ongoing rewards. I encourage everyone to try it out. Once started, I’m confident you’ll never contemplate not using Kenfigure in the future. If there is ever any barrier to adoption, let me know and I’ll be happy to improve the standard where needed and provide advice on its application.

“Follow” to be notified of new articles since I’ll be use this venue to communicate changes and to provide details on the various use cases, automations, integrations, and other ways to gain even more value from using a standard for documenting your Benchling configurations.