Creating your own Subgraph: A Step-by-Step Hands-On Guide With Mocaverse Example

In our previous articles (An Overview Of On-chain Event Logs Processing), we explored various architectures for handling on-chain events. Now, let’s delve into the subgraph solution and learn how to build a subgraph for Mocaverse NFTs step by step.

One common challenge when building a DApp is that we can only fetch the current state of a contract and not its historical transactions for display. For example, if you have your own ERC20 token, you can easily retrieve the balance using the “balanceOf” function. However, there is no straightforward way to access the historical transactions by solely interacting with the smart contract.

This is where The Graph comes into play. The Graph is an easily managed indexing protocol that provides a solution to this challenge. So, what can The Graph do?

1. Instead of hosting a full node, which can be costly, The Graph builds a node for you when you deploy it. This node can directly access transactions. When an event is emitted from the smart contract, it will be handled by the corresponding event handler.
2. The Graph generates basic event handlers and entity schemas, while still allowing flexibility for customizing entities and handlers.
3. It supports a public accessible GraphQL endpoint, making it easy for all DApps to retrieve the data they need without additional integration efforts.
4. It provides a playground for testing your queries.

Now, let’s use Mocaverse as an example to customize and deploy a subgraph!

Off-topic: Mocaverse one of the most trending NFTs recently, has launched their on-chain decentralized identity (DID) called Moca ID, and the floor price of their NFTs has increased from 0.6 ETH to 3 ETH in just two months and it is still going up!

Setting Up the Development Environment

Prerequisite: The Graph CLI is built in JavaScript, so make sure you have Node.js installed on your machine. Also, ensure that you have either yarn or npm installed.

Step 1: Register an account with The Graph.

• Go to The Graph Studio and connect your wallet to register.
• Click “Create a Subgraph” to create a dashboard for managing your subgraph.
• Let’s name this project “MocaverseDemo”.

• Once the subgraph project is created, you will see a screen like the one above. The left side is for configuring the basic information of the project, such as tags, website URL, and description. The right side displays the version and ID of the deployed subgraph, as well as deployment guidelines. Simply follow the deployment instructions to create the repository.

Step 2: Install The Graph Client

• Run npm install -g @graphprotocol/graph-cli or yarn global add @graphprotocol/graph-cli to install the graph client globally. We will need this client to create a subgraph repository later.

Note: The version used in this example is @graphprotocol/graph-cli/0.60.0.

Step 3: Initialize a Subgraph Repository

If you are only following the guidelines, the command would look like this: graph init --studio mocaversedemo. However, we can provide additional information as arguments. For example, we can configure the contract address, network, ABI file path, and subgraph slug of your project. The command would look like this:

graph init --product subgraph-studio --from-contract <CONTRACT_ADDRESS> --network <NETWORK_NAME> --abi <PATH_TO_ABI> --contract-name=<CONTRACT_NAME> --start-block=<STARTING_BLOCK_NUMBER> <SUBGRAPH_SLUG>

Let’s start by obtaining the contract ABI of Mocaverse. Normally, we can export the contract ABI from Etherscan or Polygonscan. Go to the “Contract” tab on Etherscan, scroll all the way down, and look for the “Contract ABI” row. Click on “export abi” to obtain it.

Mocaverse contract address: **0x59325733eb952a92e069c87f0a6168b29e80627f**

Before exporting the ABI, let’s take a look at it. The ABI does not appear to be a typical NFT contract ABI, as there are no transfer or mint functions and events.

After some investigation, we found some clues.

Firstly, if we go to the first page on the “internal transaction” tab, we can see that the Mocaverse contract was deployed by “Thirdweb: TWFactory”. This affected the ABI detection so the NFT contract ABI cannot be shown correctly.

But what kind of NFT contract is it? We can find the answer from the second clue. Let’s go back to the “Contract” tab and check the contract name. After doing so, we discover that the Mocaverse contract is an ERC721 contract inherited from the TieredDrop contract of Thirdweb.

Luckily, we found a similar contract called TieredDrop, and all the events match the Thirdweb documentation. Thank goodness we don’t have to type all the events ourselves!

Now, let’s export the ABI.json, extract the result data, and store it in the folder we will use for the subgraph.

In this example, the init command would look like this:

graph init --product subgraph-studio --from-contract 0x59325733eb952a92e069c87f0a6168b29e80627f --network mainnet --abi mocaverse-abi.json --contract-name=MocaverseDemo --start-block=16744931 mocaversedemo

When we run the command, a few more options will be asked for configuration. Choose Ethereum as the protocol, and use the default values since we have already set them in the command arguments.

Reminder: We chose the starting block 16744931 because the Mocaverse contract was created at this block level. Be careful not to let the subgraph start synchronizing from block 0, as it would take a long time to sync up millions of blocks without processing any data we need, since the contract didn’t exist before it was deployed.

After the setup, the repository will look like this:

Anatomy of Subgraph Repository

1. abis folder: stores contract ABI files.
2. src/mocaverse-demo.ts: contains the logic for all handlers. The graph client has generated basic handlers for each event listed in the ABI file.
3. tests/mocaverse-demo-utils.ts: functions for generating mock events. Basic events have already been pre-generated here.
4. tests/mocaverse-demo.test.ts: test cases to validate the entity structure.
5. networks.json: contains information about the network, contract name, block number, and contract address that we set.
6. schema.graphql: the GraphQL schema of the project. Basic event entities have been pre-generated here. We can add, remove, or modify entities as needed.
7. subgraph.yaml: contains all project configurations. If we add or remove any entity or handler, we need to update this file as well.

Customizing the Subgraph

Although basic event entities have already been pre-generated, we are surely not satisfied with just obtaining the historical transfer records of Mocaverse. Let’s customize it to also include the latest contract state, such as the current holders and token information.

The customizations are similar to the official tutorial, but you may encounter a compilation error if you strictly follow the official tutorial. Therefore, here is a bug-free and tailored version for the Mocaverse contract.

First, we need to append the following code to the schema.graphql file. This defines the User entity for the holders’ state, and the Token and TokenMetadata entities for the token information.

// schema.graphql
type Token @entity {
  id: ID!
  tokenID: BigInt!
  tokenURI: String!
  ipfsURI: TokenMetadata
  updatedAtTimestamp: BigInt!
  owner: User!
}
type TokenMetadata @entity {
  id: ID!
  image: String!
  name: String!
  tribe: String!
}
type User @entity {
  id: ID!
  tokens: [Token!]! @derivedFrom(field: "owner")
}

Next, we need to authorize and generate the schema by running the following command:

graph auth --studio <DEPLOY_KEY> && cd mocaversedemo && graph codegen

Ensure that the User, Token, and TokenMetadata entities are present in the generated/schema.ts file. Now, we can start modifying the handlers.

In common practice, if we want to get the metadata of a token, we can fetch it from the IPFS node by concatenating the IPFS hash and the token ID (ipfs://<HASH>/<TOKEN_ID>). However, the design of the Thirdweb TieredDrop contract allows users to set a different IPFS hash for each batch of tokens. So, we cannot simply concatenate the hash and ID.

To work around this, I have created a script to fetch the metadata of each token using the Moralis NFT metadata endpoint (https://docs.moralis.io/web3-data-api/evm/reference/get-nft-metadata) and map it to its IPFS URI.

Please download this file here and place it under the src folder.

We also need a function to map the corresponding IPFS URI with the token ID. I have added this function in moca_metadata.ts, which should also be placed under the src folder.

//src/moca_metadata.ts
import { json, JSONValueKind, BigInt, JSONValue, Result } from '@graphprotocol/graph-ts'
import { dataStr } from './mocaTokenUri'

export function getIpfsHashFromTokenId(tokenId: BigInt): string {
    const mapForIpfsHash: Result<JSONValue, boolean> = json.try_fromString(dataStr);
    if (mapForIpfsHash.isOk) {
        return  mapForIpfsHash.value.toObject().get(tokenId.toString())!.toString() as string;
    }
    return "";
}

After that, we need to create a new handler to process metadata files. Create a new file named metadata.json under the src folder with the following code. This handler stores the token name, image URL, and tribe of the Moca.

// metadata.json
import { json, Bytes, dataSource } from '@graphprotocol/graph-ts'
import { TokenMetadata } from '../generated/schema'

export function handleMetadata(content: Bytes): void {
  let tokenMetadata = new TokenMetadata(dataSource.stringParam())
  const value = json.fromBytes(content).toObject()
  if (value) {
    /* using the metadata from IPFS, update the token object with the values  */
    const image = value.get('image')
    const name = value.get('name')
    const attributes = value.get('attributes')
  if (name) {
      tokenMetadata.name = name.toString()
    } else {
        // if the metadata is not valid, set the token to invalid
        tokenMetadata.name = "invalid"
    }
    if (image) {
      tokenMetadata.image = image.toString()
    } else {
        // if the metadata is not valid, set the token to invalid
        tokenMetadata.image = "invalid"
    }
    let tribe = ""
    if (attributes) {
      const attributesArray = attributes.toArray()
      for (let i = 0; i < attributesArray.length; i++) {
        const attribute = attributesArray[i].toObject()
        if (attribute) {
          const traitType = attribute.get('trait_type')
          if (traitType && traitType.toString() == 'Tribe') {
            tribe = attribute.get('value')!.toString()
          }
        }
      }
    }
    tokenMetadata.tribe = tribe
    tokenMetadata.save()
  }
}

We also need to add a new templated data source with kind: file/ipfs to subgraph.yaml. This data source will be spawned when a file of interest is identified.

templates:
  - name: TokenMetadata
    kind: file/ipfs
    mapping:
      apiVersion: 0.0.7
      language: wasm/assemblyscript
      file: ./src/metadata.ts
      handler: handleMetadata
      entities:
        - TokenMetadata
      abis:
        - name: MocaverseDemo
          file: ./abis/MocaverseDemo.json

Lastly, it’s time for the main task. Let’s go back to mocaverse-demo.ts and customize our handlers. We will modify the “handleTransfer” function only, and not the “handleTokensClaimed”, “handleTokensLazyMinted”, or “handleTokenURIRevealed” functions, to reduce the sync time. All tokens have already been minted and revealed for months. The ipfs uris we got from Moralis are already the revealed version.

For token entities, we can use the token ID as the entity ID and simply fill in all the data the entity needs.

let token = Token.load(event.params.tokenId.toString())
  if (!token) {
    token = new Token(event.params.tokenId.toString())
    token.tokenID = event.params.tokenId

    const tokenIpfsHash = getIpfsHashFromTokenId(event.params.tokenId)
    //This creates a path to the metadata for a single Crypto coven NFT. It concats the directory with "/" + filename + ".json"
    token.tokenURI = tokenIpfsHash
    token.ipfsURI = tokenIpfsHash
    TokenMetadataTemplate.create(tokenIpfsHash)
  }
  token.updatedAtTimestamp = event.block.timestamp
  token.owner = event.params.to.toHexString()
  token.save()

To add token metadata, create a TokenMetadataTemplate and the handler will automatically run.

TokenMetadataTemplate.create(tokenIpfsHash)

For the User entity, we only need to use the owner address as the entity ID. The holding tokens will be derived from the “owner” field of Token.

/* if the user does not yet exist, create them */
  let user = User.load(event.params.to.toHexString())
  if (!user) {
    user = new User(event.params.to.toHexString())
    user.save()
  }

Since we have added a new data source, we need to re-run the codegen command for the new datasource and handler.

graph codegen

Deploying the Subgraph

Before deploying, we need to run the build command.

graph build

Afterwards, you may encounter an error related to the line entity.tier = event.params.tier in the handleTokensLazyMinted function. This error occurs because the graph client is unable to handle the tier field correctly. To resolve this issue, you can update the tier getter of TokensLazyMinted__Params in the generated/MocaverseDemo/MocaverseDemo.ts file as shown below:

get tier(): string {
    // somehow subgraph recognize this as FIXED_BYTES, but it's actually STRING
    if (this._event.parameters[0].value.kind == ethereum.ValueKind.BYTES || this._event.parameters[0].value.kind == ethereum.ValueKind.FIXED_BYTES) {
      return this._event.parameters[0].value.toBytes().toHexString();
    }
    return this._event.parameters[0].value.toString();
  }

To see the Build completed message, rerun the build command.

Once that is done, we can deploy our subgraph by running the following command and inputting the version as v0.0.1:

graph deploy --studio mocaversedemo

After executing the command, a successful message will be displayed.

To check the sync up status, we can click on the “thegraph” link, which is the same page as the project dashboard we created earlier.

Once the progress reaches 100%, we can start using it!

Querying data

Please feel free to test out my subgraph by using the sample queries provided below.

development query url: https://api.studio.thegraph.com/query/57363/mocaversedemo/version/latest

1. Fetch 5 Holders and their holdings

{
  users(first: 5, where: {tokens_: {}}) {
    id,
    tokens {
      tokenID,
      tokenURI,
      ipfsURI {
        name,
        tribe
      }
    }
  }
}

2. Fetch the metadata of token ID 1234.

{
  tokens(first: 5, where: {tokenID: 1234}) {
    id
    tokenID
    tokenURI
    ipfsURI {
      id,
      image,
      tribe
    }
  }
}

3. Fetch transfer history of token ID 1234

{
  transfers(first: 5, where: {tokenId: 1234}){
    from,
    to,
    tokenId,
    transactionHash,
    blockNumber
  }
}

Full code here: https://github.com/evanwhl508/subgraph-mocaverse-demo/tree/master

Follow me on medium if you are interested in learning more about web3 technologies with practical examples.