Deep dive into metadata
What is metadata?
As outlined in our Deep dive into smart contracts, an NFT is created by a smart contract and identified by a token ID. Without metadata, an NFT's ID is the only thing that distinguishes it from another NFT in the same collection.
If we look at its asset details using the Immutable X API, we can see that
"token_id": "160036".However, looking at the Properties section in its Immutable Marketplace listing, you can see a list of traits. This comes from the
"metadata" field in its asset details, which includes fields like "element": "Dark", "faction": "Horde" and "tagline": "Bringer of Despair".To enable NFTs to be more descriptive, smart contracts can contain metadata, which is information about the special characteristics of each token. The aim of metadata is to allow applications to utilize this data.
Other examples of how games or applications can use asset metadata:
- An in-game asset has characteristics that provide its owner specific utility within the game, like being able to defeat opponents whose assets have lesser characteristics. The game needs to be able to read the attributes of the asset in order to execute the game mechanics accordingly.
- An NFT marketplace will want to display attributes of an asset that is for sale, like the asset's name, rarity, image (using the URL specified in the asset's metadata), etc.
Metadata is most commonly in JSON format.
Example metadata from the above Guild of Guardians example, Umbar:
{
"name": "Umbar",
"type": 27,
"class": "Melee",
"series": "Founder",
"assetId": "618ede175d73c80016940936",
"element": "Dark",
"faction": "Horde",
"tagline": "Bringer of Despair",
"image_url": "https://gog-art-assets.s3-ap-southeast-2.amazonaws.com/Content/Thumbnails/Heroes/Umbar/Thumbnail_Hero_Umbar_Base.png",
"rarityStr": "Rare",
"serialNumber": "8120",
"animation_url": "https://guildofguardians.mypinata.cloud/ipfs/QmNqbvR6aKJnwJkvAotXnaVFNYuCAvedvtxhvhVxmQz62f/HLS/Base/CollectionAsset_Hero_Umbar_Base.m3u8",
"specialEditionStr": "Normal",
"animation_url_mime_type": "application/vnd.apple.mpegurl"
}
There are two major aspects of NFT metadata:
- How is it referenced? - where is the metadata retrieved from? This is typically some kind of URI, ie. URL, IPFS CID.
- How is it stored? - where is this URI stored? (On-chain or off-chain?)
The above two factors play an important role in determining the immutability of an NFT's metadata.
What does "immutability" of metadata mean and why is it important?
When an NFT's metadata is said to be "immutable", it means that it cannot be changed. This is important because metadata specifies the characteristics of an NFT, highlighting desired qualities (ie. like the qualities of a game item providing its user with special advantages), or providing information about the "rarity" of certain traits, thereby making certain assets more valuable.
Since an asset's traits can have a big impact on its utility or value, a major concern for NFT consumers is to ensure that the traits that they have purchased remain as they expect. This also suggests that NFT project creators would want to enhance trust in their projects and released assets by assuring consumers that those traits cannot be changed.
Ways that NFT metadata can be referenced
There are two main ways that NFT collections reference metadata:
| Reference method | Description | Immutable? | Example |
|---|---|---|---|
| URL | The metadata JSON is hosted at an endpoint | No - the data at the endpoint can be changed at anytime by the owner | Friendship Bracelets collection (token ID 1) - metadata URL |
| Content identifier (CID) stored on IPFS | See below | Yes - because the hash is directly generated from specific content. The content cannot be changed without the hash also changing. | Bored Ape Yacht Club collection (token ID 1296) - metadata URL |
How do CIDs on IPFS (inteplanetary file system) work?
The most common way that NFT collection creators ensure the immutability of an asset's metadata is by using content addressing of this data. Content addressing is when a hash is created from data (meaning that if this data changes, the hash will also change) and assigned as its content identifier (CID). Then, this identifier is used to organize and locate this data in a storage network. The most common storage network used by Web3 collections is IPFS (interplanetary file system), which is a peer-to-peer storage network based on content addressing.
We don't suggest using Pinata in production. Our systems will timeout after 5 seconds if your server does not respond; ensure your providers respond faster. For better performance, use services like AWS S3 to host metadata in production.
How does it work?
- Create a hash from the content, called a CID (content identifier).
- Upload the content to the IPFS network
- IPFS uses a distributed hash table (key-value database) to store the information about which node (computer) in the network holds the content represented by a particular CID.
- The DHT can then be queried with the hash to find the node hosting the content, and then connecting to the node to get the content required.
- The IPFS hash (CID) is stored in the token smart contract, and since the content represented by this hash cannot change without the hash changing, then it provides assurance that a collection's attributes are fixed. Of course, token contract owners can structure the contract in such a way that they can change the hash stored in the contract, however, since a smart contract's code is clearly visible on the blockchain, this behaviour can be publicly observed.
Ways that NFT metadata is stored
The NFT metadata reference can be stored either on-chain or off-chain:
| Storage method | Description | Immutable? | Examples |
|---|---|---|---|
| On-chain | The metadata is stored on the blockchain, typically as a value in a smart contract | Yes - in the sense that the value stored on the blockchain cannot be changed | The value stored in the tokenURI() of an L1 smart contractThe blueprint string that is assigned during the minting of an L1 token when its corresponding L2 token is withdrawn |
| Off-chain | The metadata is stored in a database, typically by the application, game or marketplace that is using the metadata (Why?) | No - the database owner has full control over what values are stored and referenced | The metadata_api_url provided when a collection is created on Immutable XThe blueprint string that is provided when a token is minted on L2 but has not yet been withdrawn to L1 |
On L1:
In the L1 smart contract
Store the URL or URI in the tokenURI field in the token smart contract as part of the ERC721 standard. Most marketplaces and applications on L1 know how to access the NFT metadata using this function.
function tokenURI(uint256 _tokenId) external view returns (string);
// Returns "https://mynftcollection.com/elephants" if hosted URL is provided
// Returns "ipfs://bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi"
// if IPFS URI is provided
// Returns "https://dweb.link/ipfs/bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi"
// if a HTTP gateway for an IPFS URI is provided
On L2 (Immutable X):
There are two ways to store metadata about tokens on Immutable X:
- Provide a URL in the
metadata_api_urlrequest body when creating a collection - Provide a
blueprintstring for a token that is minted on Immutable X
Comparison:
| Storage method | Stored on-chain or off-chain? | What happens when the token is withdrawn to L1 |
|---|---|---|
Providing a metadata_api_url | Off-chain - stored on Immutable X and accessible via API endpoints | That URL is not specified on-chain (L1) |
Providing a blueprint string when token is minted on L2 | Both - off-chain on Immutable X until the token is withdrawn to L1, then that value is written to the blockchain | The value is written to the L1 blockchain |
Providing a metadata_api_url
After a user has uploaded their off chain metadata to a provider, they will need to provide Immutable X with an API endpoint for us to retrieve the metadata. This is the value for metadata_api_url when registering a collection. This endpoint must be accessible via HTTPs and need to return JSON. The IPFS gateway in the previous section is an example of this:
https://bafybeiaujofuzawix6cgwawua6rkil2j4gurhp3jzwux7gdf5iwx4onmy4.ipfs.nftstorage.link
Our metadata crawler will access <metadata_api_url>/<token_id> at the time of minting a new token. It appends /<token_id> to the metadata_api_url, for example: https://metadata_api_url.com/1
Tips
- Ensure your endpoint returns valid JSON for each tokenID
- Ensure your endpoint sets the response header
Content-Type: application/json - Use a dedicated gateway if using IPFS
Users can modify off chain data via an asset metadata refresh.
Providing a blueprint string when token is minted on L2
The blueprint is a required field defined at the time of minting on Immutable X for each NFT. This represents the on-chain Immutable metadata of the NFT that will be written to the blockchain when it is withdrawn from Immutable X L2 to L1.
The blueprint can be a string of any format - examples include values like attack, an identifier, or an IPFS hash. This is passed to the mintFor() function in your smart contract where you could add some logic to decode it on-chain, or just save it unchanged. Check out our minting assets guide to see an example of this.
Why is metadata stored off-chain?
There are some situations that necessitate NFT metadata being stored off-chain. Additionally, NFT metadata can be stored off-chain temporarily (like the blueprint string when an L2 asset is first minted) before it is written on-chain for permanent storage.
The two main reasons metadata is stored off-chain are:
- When NFTs are used in games or applications that make rapid changes to an NFTs characteristics and need to access these updates quickly, it becomes infeasible to constantly make on-chain updates as:
- The cost of numerous updates will accumulate
- The time taken to store and retrieve may be too slow for the needs of the game/application
- The NFT is being transacted with on L2, and has not yet been withdrawn to L1, so no data has yet been written to L1.
Costs of metadata storage
When metadata is stored on-chain, blockchain state is updated, so transaction costs apply. These costs increase if the amount of data stored is larger, like those of images, videos and media. This is why links to IPFS hashes representing metadata JSON values is typically what is stored, and within those metadata JSONs are links to other files stored on IPFS, like images or videos.
Persisting metadata between L1 and L2
L2 tokens are merely representations of their corresponding L1 smart contract tokens. When a token is minted on L2, the L1 contract doesn't know anything about it. The link between the L1 and L2 token is only created when the L2 token is "withdrawn" to L1, which means that its corresponding L1 token is minted on the L1 smart contract. However, this token is purely derived from its L1 contract, and, as such, only contains data that the L1 contract contains. This means that its metadata will be that associated with the L1 contract, not anything that was specified on L2.
However, the way we understand that an L2 token represents a particular L1 smart contract token is because, when registering a collection on Immutable X, the L1 contract owner signs a transaction linking the L2 collection on Immutable X to a L1 smart contract address. These nuances of this relationship highlights the nature of the L1-L2 relationship, in that the L1 "base chain" operates autonomously and independently of L2, while the L2 chain relies on L1 for transaction validation and finalization and can choose to update its state based on the L1 state.
How to ensure that L2 metadata is persisted when the token is withdrawn to L1?
Immutable X provides a way for tokens to provide a string that is written to the L1 blockchain (Ethereum) when that token is withdrawn to L1. This is the blueprint parameter when an L2 token is minted and has the following characteristics:
- This can only be set when the token is minted on L2
- It is a required field (should be a string with at least one empty space, ie.
" ") - It cannot be updated once set
This can be used by collections that want to provide token owners with certainty regarding the attributes of their tokens on both L2 and L1.
Best practices for Immutable NFT metadata
- Use a content hash (CID) representation of the token's metadata
- Store this CID in the token's contract on-chain where a record of it remains forever
Store this CID in the token's contract on-chain where a record of it remains forever
As stated above, typically the link to a collection's metadata is stored in the tokenURI() of the L1 smart contract. For maximum immutability, it would be ideal for the contract to be constructed such that this value cannot be updated once set.
However, sometimes developers want to ensure that they are able to make updates to the metadata, ie. fix errors, update stated features, etc. that are not detrimental to token owners.
How to be able to update metadata whilst maintaining trust?
This comprises of a couple of elements:
- The tokenURI, which is the standard function to retrieve an NFT's metadata should be able to be updated
- "Maintaining trust" needs to be defined. Is it ensuring that:
- The original metadata that a token owner received at the time of minting should remain relatively unchanged?
- The original metadata should be always be accessible, regardless of what it has been changed to?
In order to determine the best metadata storage pattern for a collection, the creators need to decide on these factors.
There are a couple of pattern options (all require the tokenURI() to be updated):
- Simply update the tokenURI of a contract with a new IPFS link, however, as this is an on-chain state update, there will be a blockchain record and previous values can always be accessed.
- In the original metadata file (stored on IPFS), include a link to an IPNS URI, which points to the latest collection metadata JSON. Each time metadata is updated, the IPNS pointer can be pointed to a new IPFS URI. Thus, the latest metadata JSON file can always be accessed via the IPNS link. However, since the original metadata file contains the original metadata JSON, it is still always accessible. If applications and marketplaces adopted this pattern, they can display to users both the original and most updated metadata.
What is IPFS?
InterPlanetary File System (IPFS) is a distributed system for storing and accessing files, websites, applications, and data. If that sounds like a mouthful, don't worry, you can think of IPFS as decentralized file storage. Here are some characteristics of IPFS:
- Decentralized and not owned by a single party
- Censorship resistant
- Easier to backup files
IPFS uses a method called content addressing to access files. This involves generating a hash every time a file is uploaded and instead of using a URL like www.test.com/file we use the hash to locate the file. This hash also enables us to verify where the content has come from and whether it has been modified. NFTStorage and Pinata are examples of IPFS providers.
When you upload a folder to an IPFS provider, you will often receive an IPFS Gateway which allows web browsers to access IPFS. There will also be an option to specify the type of gateway you use.
- Public gateway - Available for everyone, free, slower, subject to rate limiting - use for testing
- Private(dedicated) gateway - Ability to control access, costs money, generallty more uptime - use in production and mainnet launches
Here's an example of a gateway you can copy into your browser:
https://bafybeiaujofuzawix6cgwawua6rkil2j4gurhp3jzwux7gdf5iwx4onmy4.ipfs.nftstorage.link
- This is often referred to as a
baseURIin smart contracts and on L1s. - On Immutable X this is the
metadata_API_URL
Any files inside the folder will be accessible by appending the filename to the gateway. Example:
https://bafybeiahnvizkbk5ni234sbqc572ejdpy5627d63jojr7lvygw4bcq6jny.ipfs.nftstorage.link/1
In this example, the name of the file is 1
Alternative storage options
While IPFS is the most popular method of storing NFT data, there are alternatives:
Metadata compatibility
It's also worth mentioning that while most NFTs follow the ERC721 standard, marketplaces can have different standards for metadata. This means metadata needs to be structured in a certain way to be accessed. For example, in all marketplaces built on Immutable X, metadata is not nested.
{
"name": "Rufus",
"animation_url": "https://guildofguardians.mypinata.cloud/ipfs/QmQDee8BPDfAH2ykRX375AWJwYZcbbJQa8wHokrSnMLLUC/HLS/Base/CollectionAsset_Hero_Rufus_Base.m3u8",
"animation_url_mime_type": "application/vnd.apple.mpegurl",
"image_url": "https://gog-art-assets.s3-ap-southeast-2.amazonaws.com/Content/Thumbnails/Heroes/Rufus/Thumbnail_Hero_Rufus_Base.png",
"attack": 4,
"collectable": true,
"element": "Water"
}
whereas in a marketplace like OpenSea, fields like attack, collectable and element would be nested under the attributes field.
{
"description": "Rufus but on a different marketplace",
"external_url": "https://gog-art-assets.s3-ap-southeast-2.amazonaws.com/Content/Thumbnails/Heroes/Rufus/Thumbnail_Hero_Rufus_Base.png",
"image": "https://gog-art-assets.s3-ap-southeast-2.amazonaws.com/Content/Thumbnails/Heroes/Rufus/Thumbnail_Hero_Rufus_Base.png",
"name": "Rufus",
"attributes": [
{ "trait_type": "Example 1", "value": "Black" },
{ "trait_type": "Example 2", "value": "Red" },
{ "trait_type": "Example 3", "value": "Yellow" }
]
}
Read more about our metadata schema here.