A starter [Squid](https://subsquid.io) project to demonstrate its structure and conventions.
It accumulates [kusama](https://kusama.network)account transfers and serves them via GraphQL API.
A [Squid](https://subsquid.io)-based indexer.
It takes [ĞDev](https://kusama.network)data and serves it via GraphQL API.
## Summary
## Dev requirements
-[Quickstart](#quickly-running-the-sample)
-[Public archives for Parachains](#public-archives-for-parachains)
-[Self-hosted archive](#self-hosted-archive)
-[Development flow](#dev-flow)
-[Database Schema](#1-define-database-schema)
-[Entity classes](#2-generate-typeorm-classes)
-[DB migrations](#3-generate-database-migration)
-[Typegen for Events, Extrinsics and Storage Calls](#4-generate-typescript-definitions-for-substrate-events-calls-and-storage)
-[Deploy the Squid](#deploy-the-squid)
-[Conventions](#project-conventions)
-[Type Bundles](#types-bundle)
## Prerequisites
* node 16.x
* node 18.x
* docker
* npm -- note that `yarn` package manager is not supported
* npm
## Quickly running the sample
## Run
Example commands below use [sqd](https://docs.subsquid.io/squid-cli/).
Please [install](https://docs.subsquid.io/squid-cli/installation/) it before proceeding.
...
...
@@ -43,38 +29,23 @@ sqd run .
```
A GraphiQL playground will be available at [localhost:4350/graphql](http://localhost:4350/graphql).
## Public archives for Parachains
Subsquid provides archive data sources [for most parachains](https://docs.subsquid.io/substrate-indexing/supported-networks/). Use `lookupArchive(<network name>, <lookup filters>)` from `@subsquid/archive-registry` to look up the archive endpoint by the network name, e.g.
If the chain is not yet supported, you can still index it using [RPC ingestion](https://docs.subsquid.io/substrate-indexing/setup/general/#set-data-source). If you take this route, use [metadata exporer](https://github.com/subsquid/squid-sdk/tree/master/substrate/substrate-metadata-explorer) with [Substrate typegen](https://docs.subsquid.io/substrate-indexing/squid-substrate-typegen/) for help with decoding.
You can also fill out this [form](https://forms.gle/Vhr3exPs4HrF4Zt36) to submit a request for an Archive/Subsquid Network dataset.
## Self-hosted archive
Self-hosted Archives are deprecated by the ArrowSquid release. Keep an eye on updates on [Subsquid Network](https://docs.subsquid.io/subsquid-network/) and use it instead once it is released.
## Dev flow
The rest was here in the squid substrate template.
### 1. Define database schema
...
...
@@ -132,93 +103,20 @@ each historical change in the spec version. See the [Substrate typegen](https://
## Deploy the Squid
After a local run, obtain a deployment key by signing into [Subsquid Cloud](https://app.subsquid.io) and run
```sh
npx sqd auth -k YOUR_DEPLOYMENT_KEY
```
Next, inspect the Squid CLI help to deploy and manage your squid:
```sh
npx sqd squid --help
```
For more information, consult the [Deployment Guide](https://docs.subsquid.io/deploy-squid/).
## Project conventions
Squid tools assume a certain project layout.
* All compiled js files must reside in `lib` and all TypeScript sources in `src`.
The layout of `lib` must reflect `src`.
* All TypeORM classes must be exported by `src/model/index.ts` (`lib/model` module).
* Database schema must be defined in `schema.graphql`.
* Database migrations must reside in `db/migrations` and must be plain js files.
*`squid-*(1)` executables consult `.env` file for a number of environment variables.
See the [full desription](https://docs.subsquid.io/basics/squid-structure/) in the documentation.
## Types bundle
Substrate chains that have blocks with metadata versions below 14 don't provide enough
information to decode their data. For those chains, external [type](https://polkadot.js.org/docs/api/start/types.extend)[definitions](https://polkadot.js.org/docs/api/start/types.extend) are required.
Subsquid tools include definitions for many chains, however sometimes external
definitions are still required.
You can pass them as a special json file (types bundle) of the following structure:
```json5
{
"types":{
"AccountId":"[u8; 32]"
},
"typesAlias":{
"assets":{
"Balance":"u64"
}
},
"versions":[
{
"minmax":[0,1000],// spec version range with inclusive boundaries
"types":{
"AccountId":"[u8; 16]"
},
"typesAlias":{
"assets":{
"Balance":"u32"
}
}
}
]
}
```
*`.types` - scale type definitions similar to [polkadot.js types](https://polkadot.js.org/docs/api/start/types.extend#extension)
*`.typesAlias` - similar to [polkadot.js type aliases](https://polkadot.js.org/docs/api/start/types.extend#type-clashes)
*`.versions` - per-block range overrides/patches for above fields.
All fields in the type bundle are optional and applied on top of a fixed set of well-known frame types.
Note, that although the structure of subsquid types bundle is very similar to the one from polkadot.js,
those two are not fully compatible.
## Differences from polkadot.js
Polkadot.js provides lots of [specialized classes](https://polkadot.js.org/docs/api/start/types.basics) for various types of data.
Even primitives like `u32` are exposed through special classes.
In contrast, the squid framework works only with plain js primitives and objects.
For instance, account data is passed to the handler context as a plain byte array. To convert it into a standard human-readable format one should explicitly use a utility lib `@subsquid/ss58`:
```typescript
// ...
from:ss58.codec('kusama').encode(rec.from),
to:ss58.codec('kusama').encode(rec.to),
```
## Graphql server extensions
It is possible to extend `squid-graphql-server(1)` with custom
[type-graphql](https://typegraphql.com) resolvers and to add request validation.
For more details, consult [docs](https://docs.subsquid.io/graphql-api/).
See https://duniter.org/wiki/duniter-v2/indexers/duniter-squid/
## TODO
- [ ] Refac way to process data from events (new, fetch needed...)
- [ ] Use cache abilities
- [ ] Look if it is possible to add relation without fetching endpoints
