duniter-indexer

Description

G1 Duniter indexer is an indexer for the G1 Duniter blockchain. It is a nodejs application that uses the duniter-v2s. The goal of this application is to index the blockchain and provide a GraphQL API to query the blockchain.

Installation

Requirements

First, you need to install docker.

To verify that docker is installed correctly, run:

docker run hello-world

Depending on your installation method, you might need to run docker commands as sudo.

Clone the repository

Then, you need to clone the repository and go inside the newly created folder:

git clone https://git.duniter.org/nodes/duniter-indexer.git
cd duniter-indexer

Set up the environment

Copy the .env.example file to .env:

cp .env.example .env

You can then change the default values in the .env file. The most important value is the HASURA_GRAPHQL_ADMIN_SECRET which is the secret used to access the hasura console.

Usage for production

To run the indexer in production, you need:

• a duniter archive node on a live network (gdev, gtest, g1)
• postgres and hasura graphql engine dependencies
• the duniter indexer

You can get this with the example docker-compose.yml files:

docker compose -f docker-duniter-gdev.yml -f docker-dep-prod.yml -f docker-indexer.yml up -d

It will start four services on the following interfaces:

• a duniter-v2s archive node (ws rpc on 9944, p2p on 30333)
• a postgres database (on port 5432)
• a hasura graphql server (on port 8080)
• the indexer application (on port 3000)

Usage for development

Scheme

The indexer is made of several components. The following scheme shows the data flow between the components and the interfaces you can interact with.

Install developement dependencies

You will need pnpm, a better version of npm, "node package manager".

# install dependencies
pnpm install
# If you want to modify the database structure, you will also need hasura-cli:
pnpm install --global hasura-cli
# To get the schema with `pnpm schema`
pnpm install get-graphql-schema

Start development dependencies

TL;DR: ./load.sh

Start a local blockchain

To use the dev blockchain with custom data:

# start local dev blockchain
docker compose -f docker-duniter-dev.yml up

You can check this local Duniter node with polkadotjs on your browser

Start the PostgreSQL and Hasura GraphQL engine

The indexer is based on hasura graphql engine and a postgres database, that you can start them with:

# start postgres and hasura (can take 30 seconds to initialize)
docker compose -f docker-dep-dev.yml up

Hasura will start, you can check it with hasura console on your browser Your admin secret is the one you set in the .env file (default is "your_secret"). The postgresql database also starts, and you can interact with it in the Hasura console.

Start the indexer

On another terminal, you can start the indexer:

pnpm dev:test # uses genesis and history from `resources` folder

it should start indexing the chain into postgres database:

[nodemon] starting `vite-node src/server.ts`
[15:15:35.231] INFO: Server listening at http://0.0.0.0:3000
[15:15:35.275] INFO: Indexing Genesis
[15:15:35.314] INFO: Indexing past transactions
[15:15:35.318] INFO: Indexing past blocks...
[15:15:35.320] INFO: There are some blocks to index
[15:15:35.347] INFO: Last finalized from RPC: 5
[15:15:37.522] INFO: Last finalized from RPC: 6
[15:15:37.529] INFO: There are some blocks to index
[15:15:42.876] INFO: Last finalized from RPC: 7

Alternatively, use F5 for vscode debugging. At this point you are up to contribute to the indexer.

More

Indexer app

You can start the indexer app for a custom playground interface:

pnpm dev:front

this will expose the duniter indexer app, you should see here the last indexed block.

Modify the database

To modify the database structure, you need to start hasura console after the graphql engine with:

pnpm hc

this will open your browser on an other hasura console. The changes you make to the database structure will create migration files in hasura/migrations folder.