Dynamically derivate JSON traits and case classes with Circe in Scala

In our scenario to decode and model a Json string (e.g. a configuration file) that may have different data structures for its fields.

First of all, the required libraries:

libraryDependencies ++= Seq(
  "io.circe" %% "circe-core",
  "io.circe" %% "circe-generic",
  "io.circe" %% "circe-parser"
).map(_ % "0.12.3")
libraryDependencies += "io.circe" %% "circe-derivation-annotations" %"0.12.0-M7"

libraryDependencies += "io.circe" %% "circe-generic-extras" % "0.13.1-M4"

A Scala common approach to model this kind of data is reflected by traits and case classes data structures.

Let’s suppose we expect a configuration file structured like the following:

{
  "reader": "myReader",
  "writer": "myWriter"
  "firstConf": {
    "broker": "broker",
    "topic": "myTopic"
  },
  "secondConf": {
    "schema": {
      "type": "record",
      "name": "myName",
      "namespace": "myNamespace",
      "fields": [
        {
          "name": "FirstName",
          "type": [
            "null",
            "string"
          ],
          "default": null
        },
        {
          "name": "LastName",
          "type": [
            "null",
            "string"
          ],
          "default": null
        }
      ]
    },
    "numOfShards": 1
  },

  "thirdConf": {
    "windowDuration": 900
  }
}

As you can see, each _Conf field is structured in a different way, since we want to provide each time a customized configuration file, but keeping a generic approach.

We want to model the result of our Circe parsing like this:

import io.circe.JsonObject
import io.circe.generic.JsonCodec

@JsonCodec
case class AppConfig(
  reader: String,
  writer: String,
  firstConf: Conf,
  secondConf: Conf,
  thirdConf: Option[Conf] = None
)

With the Conf trait extended by the case classes:

sealed trait Conf
@JsonCodec
case class FirstConf(broker: String,
                     kafkaTopic: String,
                     ) extends Conf

@JsonCodec
case class SecondConf(
                    schema: Json,
                    numOfShards: Int
                  ) extends Conf

@JsonCodec
case class ThirdConf(
                          windowDuration: Int
                          ) extends Conf

And parse the config file like this:

io.circe.parser.decode[AppConfig](jsonString)

In order to let Circe dynamically infer the type of the data that extends the trait, we must modify our trait in this way:

import io.circe.generic.extras.{Configuration, ConfiguredJsonCodec}
@ConfiguredJsonCodec
sealed trait Conf

object Conf {
  implicit val config: Configuration =
    Configuration.default.withDiscriminator("type")
}

And add a field “type” to the data structures that should be modeled under the trait:

"firstConf": {
  "type": "FirstConf",
  "broker": "broker",
  "topic": "myTopic"
},
"secondConf": {
  "type": "SecondConf",
...
},
"thirdConf": {
    "type": "ThirdConf",
    "windowDuration": 900
  }