API pagination guide

This document explains how to implement cursor-based pagination in API requests. It's designed to help developers efficiently retrieve data using our API.

Concepts

Cursor

In our API, a cursor is a base64 encoded string that represents a position in a dataset. It's used to fetch the next set of results in a paginated API query.

After decoding, a cursor might look like this:

{
    "id": "0x6f304a275...",
    "name": null,
    "updated_at": "2022-11-15T14:13:02.591287Z"
}

Page size

Page size refers to the number of results returned in a single API response.Its purpose is to control the volume of data retrieved per request to manage server load and client-side data handling.

Remaining field

The remaining field indicates if more data is available (1: yes, 0: no).

Implementing pagination

1. Initial request:

• Make your first API call without a cursor to retrieve the first set of data.
• Specify the page_size to control the number of results.

2. Subsequent requests:

• Use the cursor provided in the previous response to request the next set of data.
• Repeat this step until remaining is 0.

Practical example

This script demonstrates fetching a list of filled orders from the Immutable X API for specified collections, using pagination with throttling in the Typescript SDK.

// Import SDK modules
import { config, x } from "@imtbl/sdk";

const { Environment } = config;

const {
    IMXClient,
    imxClientConfig, // helper method to create a client config
    ImmutableX,      // alias for IMXClient
} = x;

// Function to fetch orders in a collection
async function listOrdersInCollections(cursor: string, collection: string) {
    const environment = Environment.SANDBOX; // or Environment.PRODUCTION
    const client = new IMXClient(imxClientConfig({ environment }));;
    const ordersRequestParam: x.OrdersApiListOrdersV3Request = {
        cursor: cursor,
        pageSize: 20,
        sellTokenAddress: collection,
        status: "filled"
    };
    return await client.listOrders(ordersRequestParam);
}

// Function to paginate through orders with throttling
async function enumerateOrdersCursor(collection: string) {
    let orders = [];
    let remaining = 1;
    let cursor = '';
    let throttleCount = 0;

    while (remaining === 1) {
        const response = await listOrdersInCollections(cursor, collection);
        remaining = response.remaining;
        cursor = response.cursor;
        orders = [...orders, ...response.result];

        // Throttling mechanism: pause after every 5 requests
        throttleCount++;
        if (throttleCount % 5 === 0) {
            await new Promise(resolve => setTimeout(resolve, 1000)); // 1-second delay
        }
    }
    return orders;
}

// Main function for processing orders from multiple collections
async function main() {
    const collections = ['0x07f68c5...', '0xb40a0df...'];
    let orders = [];
    for (const collection of collections) {
        const result = await enumerateOrdersCursor(collection);
        orders = [...orders, ...result];
    }
    console.log(orders);
}

// Execute and handle errors
main()
    .then(() => console.log('Completed listing orders'))
    .catch(err => {
        console.error(err);
        process.exit(1);
    });

Conclusion

This guide provided a comprehensive overview of implementing cursor-based pagination, complete with a practical example incoporating throttling to ensure a smooth interaction with the API.