Migrate to Duniter v2 ecosystem: Substrate, Subsquid/Hasura

Duniter v2 Substrate support with polkadot-sdk:

• APIs: RPC, and GraphQL Hasura

---

Resources

• py-polkadot-sdk repository
• Polkadot SDK documentation
• subxtpy repository
• subxtpy documenation
• Client developer doc
• Constants
• https://git.duniter.org/clients/cesium-grp/cesium2s/-/issues
• https://forum.duniter.org/t/vocabulaire-de-base-pour-comprendre-duniter-v2s-lecture-fortement-recommandee-pour-tous/9053
• https://forum.duniter.org/t/liste-des-parametres-protocolaires-de-duniter-v2s/9297/1

---

Transition plan

• Release and maintain v0.12.0 as BMA-only
• v0.20.0dev as Substrate-only

Allow non-related developments/experimentation to happen in parallel?: Which would not create much git conflicts:

• Configuration support (#79): Seems the best candidate, the class development it-self, not the integration in the commands
• Use a Terminal based and/or Graphical user int... (#18), Set console interface with Textual console/TUI ... (#440): PyGObject3, Textual TUI console: Just experimentation of the possibilities, interface design, no integration

---

Consider Hexagonal architecture

• duniterpy#117
• https://blog.octo.com/architecture-hexagonale-trois-principes-et-un-exemple-dimplementation/ en

---

Set up Duniter v2s and indexer on ĞDev

• Set-up Duniter-v2s
  • Become smith
• Set-up Indexer: Subsquid

Outline plan

• Set up authentication based on DEWIF converted to SR25519 substrateinterface.Keypair.
• Set up two network layers: RPC and Indexer GraphQL, make sure the two endpoints are sync
• Get currency, parameters, head: blockchain/tools.py, wot/tools.py identity helpers
• port commands one by one to v2 ecosystem (starting by the easy one, then ramping up the difficulty)
  • v1 and v2 ecosystems support would coexist during the transition
  • hiding commands and arguments can be done with Click
• remove v1 network layer, public key
• remove duniterpy dependency (probably later)

---

• Split changes in child items/sub-tickets. Put them in the Kanban board and/or milestone.
• Ideas from Tikka roadmap
• Matograine’s plan

• Delete code of commands which will be rewritten, keep structure, pick code in git history for the reimplementation
• Delete step-by-step commands, functions which have been rewritten

Delete features which don’t make sense anymore and which are not very coupled

• Delete blockchain verify cmd, doesn’t make sense anymore, algorithm can be retrieved in the git history for next blocks implementation (#283 (closed))
• Delete blockchain diffi, since the PoW is no longer a thing with BABE/Grandpa consensus algorithm
• Delete checksum command since the address integrity (checksum) and currency is integrated into the address.
• blockchain blocks? Does not make sense with https://duniter-portal.axiom-team.fr/ blocks exploration more interesting in a web interface
  • blockchain info is the only command remaining in blockchain group. Move one step above.

Set up network layers

• Introduce polkadot-sdk dependency
• Consider subxtpy (give it a try) (async, no Py3.13 support)
• Consider async-substrate-interface (which can be used without asynchronous property). Dependent on from bittensor_wallet.utils import SS58_FORMAT. Not standalone library.
• Check Tikka substrate own client library
• Set up RPC API network layer
• Set up GraphQL API network layer with gql, doc, repology
  • choose transporter (httpx, websocket): choosing requests because it is the library used by py-polkadot-sdk and pydiscourse
• Initiate clients in Click context, causing issue for testing and if we would like to use the functions as a library. The functions should initialize these clients on the need.
• Check whether RPC and GraphQL endpoints are syn... (#370): Check the RPC and the indexer APIs are sync: same block number and hash each time network requests are made. Create similar cmd: index check.

Pre-requisites

• Account storage support: authentication, revoca... (#477 - closed)

Authentication

• Not using SR25519 because this would reduce our possibilities. With ED25519 (used since Duniter v1), we can still access the seed and do everything. With SR25519 we are locked and can’t get the seed.
• Therefore we are not implementing the encrypted JSON authentication format provided by py-polkadot-sdk since it only supports ED25519. We therefore stay with (E)WIF authentication format
• Do we set mnemonic as default import method, instead of Scrypt?

Tools, functions, helpers

• Get currency code/name from RPC: substrate.chain for the ss58 format address
• Request uids from the indexer to set the link between the identity index and the identity username/alias
• Implement Click type checking when passing a ss58 format address
• Implement mnemonic authentication import
• Amount class: Amounts management: cent, unit, and relative fo... (#193)
• get_ud_value()
• Display fees and refund estimations on all kind of transactions: nodes/rust/duniter-v2s!331 (merged)

Port existing commands/features

• wot lookup: #389, #387
• wot status: missing distance, sentry info
• money history: minor topics to be considered, #339
• wot certify, Write unit tests for 'certify' command (#375)
• money balance: nodes/rust/duniter-v2s!328 (merged), nodes/duniter-squid!27 (merged)
  • Take into account UD (not yet claimed)? Are they not yet part of the balance returned by Substrate? 1 2
• money transfer:
  • basic transfer
  • transfer reference
  • multi-recipients with batch
  • Remove --outputBackChange option, rename --allSources?
• wot membership

---

Postpone to next release? (not so important, DR can be released standalone)

• wot revocation: (support reading v1 revocation format file until a deprecation date, support generating and reading v2 format)
• wot exclusions
• blockchain blocks

Important features to be ported

Only implemented by Silkaj, so once we switch the Ğ1 to v2s ecosystem, following features will stop functioning:

• multi-recipients transfers to Ğ1 technical ecosystem remunerations
• DeathReaper: Keep it similar listening to the blockchain events. Listen to exclusions from the indexer. Could be replaced with a feature onchain to report via email the exclusion. The datapod could store the email addresses. Having the emails accessible to everyone would be an issue regarding spam purposes. A private service/datapod should store the emails.

Post tasks after initial porting

• Get rid of DuniterPy usage as much as possible
  • What might remain is authentication support and v1 pubkey support. Copy code into Silkaj to get rid of DuniterPy dependency.
• Get rid of --display/--dry-run options Display option to display the generated documen... (#314)

Implement new features

• wot certify display expired certifications
• Endpoint discovery 1, 2, 3
• UD creation
• Display UDs in history command (claimed and unclaimed)
• identity: create, confirm, link-account, change-owner-key Implement the identity and all the process arou... (#90)
• Smith sub-WoT handling
  • Commands similar to normal wot?: smith: certify, lookup, membership, invite, accept status, update-key, set-session-key, go-offline, go-online, check
  • Integrate Smith license
  • Becoming Smith diagram
  • Check whether the RPC API connected to is in Unsafe, Safe mode
• Technical Committee: tech-committee: list-members, list-proposals, vote
• Mnemonic derivations (//0, //1,…) Ğcli

Later

• Display fees per transaction and ask for confirmation (some are reimbursed, so mention the fees are reimbursed). Document it on the documentation.
• Schedule transfer support