Skip to content
Snippets Groups Projects

WIP: WS2P v2

Open WIP: WS2P v2
ws2p_v2 into master
Open Éloïs requested to merge ws2p_v2 into master
Viewing commit aa9c3a84
Show latest version
1 file
+ 4
− 1
Compare changes
  • Side-by-side
  • Inline
rfc/0006_ws2p_v2.md 0 → 100644
+ 869
− 0
# Duniter WS2P API v2
This document details the specifications of WS2P v2.
## Contents
* [Contents](#contents)
* [FAQ](#faq)
* [What is WS2P ?](#what-is-ws2p)
* [Why create a synchronization process ?](#why-create-a-synchronization-process)
* [Why remove shared peer records ?](#why-remove-shared-peer-records)
* [Why not a total binarization?](#why-not-a-total-binarization)
* [What consequences for clients software ?](#what-consequences-for-clients-software)
* [Conventions](#conventions)
* [Raw format](#raw-format)
* [Binary format](#binary-format)
* [Message encapsulation](#message-encapsulation)
* [Payload type](#payload.type)
* [Endpoints v2](#endpoints-v2)
* [Endpoint binary format](#endpoint-binary-format)
* [Endpoint utf8 format](#endpoint-utf8-format)
* [Peer card](#peer-card)
* [Peer card Raw format](#peer-card-raw-format)
* [Peer card JSON stringified format](#peer-card-json-format)
* [Getting peer cards from other Duniter nodes](#getting-peer-cards-from-other-duniter-nodes)
* [Priority of WS2P out-coming connections](#priority-of-ws2p-outcoming-connections)
* [Establishing a WS2P connection](#establishing-a-ws2p-connection)
* [List of accepted v1 messages](#list-of-accepted-v1-messages)
* [CONNECT Message](#connect-message)
* [ACK Message](#ack-message)
* [SECRET FLAGS Message](#secret-flags-message)
* [OK Message](#ok-message)
* [Rules for accepting an incoming connection](#rules-for-accepting-an-incoming-connection)
* [WS2Pv2 Messages](#ws2pv2-messages)
* [List of accepted v1 messages](#list-of-accepted-v1-messages)
* [Requests](#requests)
* [Requests responses](#requests-responses)
* [HEADs v2](#heads-v2)
* [HEADs v3](#heads-v3)
* [Documents messages](#documents-messages)
## FAQ
### What is WS2P
WS2P means "**W**eb **S**ocket **To** **P**eer".
WS2P is the inter-node network layer, it is the only guarantor of synchronization between the different nodes of the network, so its role is critical.
WS2P is the network part of the Duniter protocol
WS2P is exclusively based on websocket technology.
### Why create a synchronization process
The download phase of the synchronization process is essential, it allows a new peer to download the entire blockchain quickly (then indexed in a second phase called "apply").
Historically, this process was managed by the BMA crawler, but since the arrival of WS2P v1 the BMA crawler is no longer in service, and we keep its code only for `sync` command. So for maintainability reasons, it is necessary to migrate this process outside the BMA crawler, then delete the BMA crawler code.
Without its crawling part, BMA has become a Client API, which is to be removed and replaced by a new Client API named GVA (see [GVA RFC](https://git.duniter.org/nodes/common/doc/blob/graphql_api_rfc/rfc/0003%20RFC%20GraphQL%20API%20for%20Duniter%20Clients.md)).
Finally, the synchronization process is more about the inter-node network than the interface with clients, so it makes sense to migrate this process into WS2P.
### Why remove shared peer records
Historically, shared peer files had been set up to allow the BMA crawler to automatically detect the use of multiple member nodes with the same key and automatically assign a different prefix to different nodes of the same member so that they calculate different nonces spaces.
Since Duniter-ts 1.6, the user can now manually assign a different prefix to each of his machines.
In addition, WS2P v1 has provided a new way to uniquely identify nodes (the node id, we will discuss this later). This new way of uniquely identifying nodes opens the way to possible new automatic prefix allocation mechanisms that will not require a shared peer record.
Finally, shared peer cards have a big drawback: a peer cannot prune obsolete endpoints of other peer sharing the same key because it cannot easily evaluate if these other endpoints are obsolete. In absolute terms, it is possible to bypass problems but it is simpler to directly delete shared peer records, and as we no longer need them: Occam's razor.
### What is the binary format used for serialization
To allow easy interfacing between different implementations, WS2P v2 defines network features (detailed later in the RFC) allowing each WS2P Public to declare in its endpoint the list of features it supports. The default format is [CBOR](https://en.wikipedia.org/wiki/CBOR), a widespread binarization format based on JSON. But for byte-efficiency reasons, Dunitrust defined an other serialization format based on the Rust crate *bincode*.
## Conventions
### Raw format
All documents in raw format are explained via a [PEG grammar].
The raw format is a plain text format used to sign a document or verify its signature.
Only documents whose signature must be verifiable by a client software are signed in this way. This applies in particular to peer cards and HEADs.
WS2pV2 messages are signed in binary format.
The PEG grammars of raw formats are presented according to the [pest syntax] (a particular implementation of PEG).
[PEG grammar]: https://en.wikipedia.org/wiki/Parsing_expression_grammar
[pest syntax]: https://pest.rs/book/grammars/syntax.html
## Common structures
A certain number of structure like hash appear repeatedly in the messages. Here we describe it once and refer to it later.
### 32 byte number
A 32-bytes number is represented in JSON by an array of 32 8-bits numbers.
```json
[8, 56, 211, ..., 12] // array of 32 8-bits numbers
```
### 64 byte number
A 64-bytes number is represented in JSON by an array of 64 8-bits numbers.
```json
[8, 56, 211, ..., 12] // array of 64 8-bits numbers
```
### Hash, Pubkey, Signature
Hashes or pubkeys or signatures are just numbers, most often 32B or 64B one.
## Message encapsulation
All WS2P v2 message are encapsuled under [PKSTL v1 protocol](./0011_pkstl_v1.md).
Each WS2Pv2 message is binarized and then placed in the CUSTOM_DATAS field of the PKSTL message that encapsulates it.
Several binary serialization formats are available: CBOR and Bincode. The message defined below are written in JSON, allowing straightforward serialization to CBOR. For the Bincode serialization, see [Bincode format reference](./0006_annex/bincode.md).
All WS2Pv2 messages are structured as follows (CBOR format) :
```json
{
"ws2p_version": 2,
"currency_name": "g1",
"issuer_node_id": 395,
"issuer_pubkey": {
"algo": "Ed25519",
"content": [52, 147, .., 93], // 32 bytes pubkey
},
"payload": {
"type": "MESSAGE_TYPE_NAME",
"content": {} // depends on payload.type
},
}
```
`ws2p_version` := This field is placed first so that future versions of WS2P are not constrained on the other fields,
the only constraint will be to start the message with the version number stored in u32.
`currency_name` := Empty string is allowed, it allow the user to synchronize his node without having to manually enter the currency on which he synchronizes. The node will then adopt the currency that is specified in the CONNECT message it will receive from the selected reference node.
_* The type of `payload` is determined by the content of `message_type`._
### payload.type
Exhaustive list of possible `payload.type` values:
| message type | `payload.type` value | Hashed | Signed |
|:--------------------:|----------------------|--------|--------|
| CONNECT | "Connect" | no | yes |
| ACK | "Ack" | no | yes |
| SECRET_FLAGS | "SecretFlags" | no | yes |
| OK | "Ok" | no | yes |
| KO | "Ko" | no | yes |
| REQUEST | "Request" | yes | no |
| REQUEST_RESPONSE | "ReqRes" | yes | no |
| PEERS | "Peers" | yes | no |
| HEADS_V2 | "HeadsV2" | yes | no |
| HEADS_V3 | "HeadsV3" | yes | no |
| BLOCKS | "Blocks" | yes | no |
| PENDING_IDENTITIES | "PendingIdentities" | yes | no |
| PENDING_MEMBERSHIPS | "PendingMemberships" | yes | no |
| PENDING_CERTS | "PendingCerts" | yes | no |
| PENDING_REVOCATIONS | "PendingRevocations" | yes | no |
| PENDING_TXS | "PendingTxs" | yes | no |
## Endpoints v2
```json
{
"api": "WS2P",
"api_version": 2,
"network_features": [], // array of bytes, defined below
"api_features": [], // array of bytes, defined below
"host": "subdomain.domaine.tld",
"ip_v4": "5.135.188.170",
"ip_v6": "2001:41d0:8:c5aa::1",
"port": 443,
"path": "ws2p",
}
```
### utf8 format
The utf8 format is used to display the endpoint in a human-readable format. It is also in this format that the user can manually enter an endpoint.
General utf8 endpoint format :
API_NAME VERSION NF1 .. NFn AF PORT HOST IP4 [IP6] PATH
VERSION := version number prefixed by `V` (example : `V1`)
NF := NETWORK_FEATURE
AF := API_FEATURES in hexadecimal prefixed by `0x` (example : `0x01`)
Example:
WS2P V2 TLS 0x7 443 g1.durs.info ws2p
### network_features
The 8 bits represent booleans to define the presence or absence of 8 network features. WS2Pv2 defines only 4 features, the remaining 4 are undefined and are in anticipation of future Ğfeatures.
Network features :
| bit | feature |
|:---------:|----------|
| 0000_0001 | HTTP |
| 0000_0010 | WS |
| 0000_0100 | S |
| 0000_1000 | TOR |
HTTP := This feature indicates that the endpoint should be contacted with http protocol.
WS := This feature indicates that the endpoint should be contacted with websocket protocol.
S := This feature indicates that the endpoint should be contacted with an SSL/TLS overlay (HTTPS or WSS or any other protocol that support TLS).
TOR := This feature indicates that the endpoint must be contacted via the tor network (hidden service).
Note about the network protocol to choose: Each API must define a default network protocol. If no network protocol is specified in the network features, the default API network protocol is chosen.
For example, the default network protocol for WS2P is `ws://`, it's the only one possible for this API, so there is no need to indicate it in the network features.
### api_features
The interpretation of this field depends on the API because it represents API-specific features. Here is the interpretation for the WS2P API :
WS2PFeatures type definition :
| bit | feature |
| :-------: | ------- |
| 0000_0100 | RBC |
| 0000_0010 | LOW |
RBC := Support Rust BinCode format
LOW := Accept low speed connection requests
WS2P v2 uses only 2 of 8 features. The 6 free bits can be used for future versions of WS2P.
## Peer card
Signed document declaring all the endpoints of a peer.
Peer card v11 is signed in raw format (without SIGNATURE itself of course).
A node's peer record can also contain signatures of the node's network key. This allows, for example, to sign mirror nodes with known and trusted keys.
The signed key must be signed with the blockstamp corresponding to the date of signature. More precisely, it is the text `PUBKEY:BLOCKSTAMP` that must be signed.
### CBOR format
```json
{
"version": "v11",
"currency_name": "g1",
"issuer": {
"algo": "Ed25519",
"content": [122, 11, .., 95] // 32 bytes pubkey
},
"node_id": 10623,
"created_on": 542,
"endpoints": [], // Array of endpoitns v2
"endpoints_str": [], // Array of Strings (endpoitn v2 in utf8 format)
"sig": {
"algo": "Ed25519",
"content": [187, 7, .., 23] // 64 bytes signature
},
"certifiers": [
{
"certifier": {
"algo": "Ed25519",
"content": [122, 11, .., 95] // 32 bytes pubkey
},
"blockstamp": {
"id": 327,
"hash": [85, 132, .., 27] // 32 bytes hash
},
"sig": {
"algo": "Ed25519",
"content": [187, 7, .., 23] // 64 bytes signature
},
}
]
}
```
### Peer card Raw format
Approximate description :
VERSION:CURRENCY:NODE_ID:PUBLIC_KEY:CREATED_ON
ENDPOINT 1
...
ENDPOINT n
SIGNATURE
_/!\ This description is approximate, it allows you to quickly understand the format but is not authoritative for developing a code to parse the format exactly. If you want to develop a code using this format, please refer to its PEG grammar._
Example :
11:g1:0:7iMV3b6j2hSj5WtrfchfvxivS9swN3opDgxudeHq64fb:50
WS2P V2 S 7 g1.durs.ifee.fr 443 ws2p
WS2P V2 S 7 84.16.72.210 443 ws2p
EQ2D5almq2RNUi3XZNtTpjo9nWtJF0PzsCW7ROAzCQKiEtpI7/fW8Z23GJ2a/SIxfYSzlq/cZqksE4EoVe1rAw==
Peer card v11 PEG grammar :
```pest
// Single character rules
nl = _{ "\n" }
no_zero_hexa_lower = @{ '1'..'9' | 'a'..'f' }
hexa_lower = @{ ASCII_DIGIT | 'a'..'f' }
hexa_upper = @{ ASCII_DIGIT | 'A'..'F' }
base58 = { !("O" | "I" | "l") ~ ASCII_ALPHANUMERIC }
base64 = { ASCII_ALPHANUMERIC | "+" | "/" }
// Numbers rules
tens = @{ '1'..'9' ~ ASCII_DIGIT }
u8_hundreds = @{ ("2" ~ ('0'..'4' ~ ASCII_DIGIT | ('0'..'5'){2})) | ("1" ~ ASCII_DIGIT{2}) }
u8 = @{ u8_hundreds | tens | ASCII_DIGIT }
no_zero_u_int = @{ '1'..'9' ~ ASCII_DIGIT* }
u_int = @{ "0" | no_zero_u_int }
// Usefull types rules
currency = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "-" | "_"){,255} }
block_id = @{ u_int }
hash = @{ hexa_upper{64} }
pubkey = @{ base58{43,44} }
ed25519_sig = @{ base64{88} | (base64{87} ~ "=") | (base64{86} ~ "==") }
// IP v6 rules
ip6_seg = _{ hexa_lower{1,4} }
ip6_full = _{ (ip6_seg ~ ":"){7} ~ ip6_seg }
ip6_no_full = @{
ip6_seg? ~
(":" ~ ip6_seg){0,6} ~
"::" ~
(ip6_seg ~ ":"){0,6} ~
ip6_seg?
}
ip6_inner = @{ ip6_full | ip6_no_full }
ip6 = _{ "[" ~ ip6_inner ~ "] " }
// Endpoint v2 rules
api_version_inner = @{ no_zero_u_int }
api_version = _{ "V" ~ api_version_inner ~ " " }
http = @{ "HTTP " }
ws = @{ "WS " }
tls = @{ "S " }
tor = @{ "TOR " }
network_features = _{ http? ~ ws? ~ tls? ~ tor? }
api_features_inner = @{ (hexa_lower{2})+ | no_zero_hexa_lower }
api_features = _{ "0x" ~ api_features_inner ~ " " }
domain_name_part = @{ ASCII_ALPHA_LOWER ~ (alphanum_lower | "-" | "_")* }
domain_name_ext = @{ alphanum_lower+ }
domain_name_parts = @{ (domain_name_part ~ ".")+ ~ domain_name_ext }
domain_name_onion = @{ alphanum_lower{16} ~ ".onion" }
domain_name_inner = @{ domain_name_parts | domain_name_onion | domain_name_part }
domain_name = _{ domain_name_inner ~ " " }
ip4_inner = { u8 ~ "." ~ u8 ~ "." ~ u8 ~ "." ~ u8 }
ip4 = _{ ip4_inner ~ " " }
path = _{ " " ~ path_inner }
endpoint_v2 = ${ api_name ~ " " ~ (api_version)? ~ (network_features)? ~ (api_features)? ~ ip4? ~ ip6? ~ domain_name? ~ port ~ path? }
// Peer v11 rules
peer_v11 = ${ "11:" ~ currency ~ ":" ~ node_id ~ ":" ~ pubkey ~ ":" ~ block_id ~ nl ~ (endpoint_v2 ~ nl)+ ~ ed25519_sig? }
```
### Peer card JSON stringified format
This human-readable stringified format will be used by all APIs that wish to provide peer cards in a human-readable format. For example, Client APIs.
```json
{
"version": 11,
"currency": 1,
"node_id": "15af24db",
"algorithm": "Ed25519",
"pubkey": "D9D2zaJoWYWveii1JRYLVK3J4Z7ZH3QczoKrnQeiM6mx"
"blockstamp": "128310-000002A569DCEED62227CAC0ABDFE6B2647B21B3193A40398CD12BC2A95C24D9",
"endpoinds": [
"WS2P 2 1 TLS 3 DEF LOW ABF 443 g1.domaine.tld ws2p"
],
"signature": "e2BtWtLs3Uqw80bvOx8YGfmDLfA44/apzpZA8YfH+WtnmPY5r4XUlgvctPsq2bHVw3iPWxuxx5oJh0JHITrECw==",
"certifiers": [
"Ez4huJahi6qfG8eVjyc84CwVtpHagmfUTtRFazXY4G2h:81234-0000047F1D3BB30C24CBD9D60977FC9474D3A985DF88BB9C9CAFE9160D94AC17:oP5iHvXRx1mI+Nic621/tU42kk8Mr+m5wgNZiBPJf4vGSlGSPkTSbzMJgHehnCcEF/UB+xPoLZ2WK1AMODlFDg=="
]
}
```
### Getting peer cards from other nodes
Initially, nodes are totally endpoint agnostic and don't know any node.
The ws2p module requires that at least one or more peer cards are already stored in the database when launching the node,
it's up to the `sync` command to handle this (see "WS2P synchronisation" part).
When node starts up, the WS2P module accesses the peer cards stored in a local database following synchronization.
## Priority of WS2P outcoming connections
The WS2P module attempts to establish outcoming connections at the kick-start of the node and then every 10 minutes.
At each attempt, all known endpoints are contacted until the outcoming quota is reached.
If the outcoming quota is already reached at the beginning of an attempt, the least priority connection is removed in order to keep the network dynamic and scalable.
The WS2P module does not attempt to connect randomly to any node among those of which it knows a WS2P endpoint,
it classifies them according to the following criteria :
### 1st criterion: network features
Some network features, imply that the node prioritizes endpoints having such or such network feature.
For example, if the node has the TOR feature, it will prioritize nodes that also have the TOR feature.
_Note: the first criterion takes precedence over the second one._
### 2nd criterion: priority score
Please register or sign in to reply
The priority score of a node depends on its public key and is calculated as follows :
by default the priority score is zero
If the key is a member: +1
If the key is preferred/privileged: +2
If the key is the same as yourself: +4 (for multi-node)
In case of equality, the order in which the endpoints will be contacted will depend on the sorting algorithm used according to the implementations, there are no rules, we can consider that it is pseudo-random.
## Establishing a WS2P connection
After sorting all WS2P endpoints according to the criteria described in the previous section,
Duniter tries to connect to the other nodes.
**normal connection**
1. Opening a websocket pointing to the remote endpoint. If the websocket has been successfully opened before the timeout\*, send a CONNECT message. Otherwise, log the error and close the websocket.
2. Wait to receive a CONNECT message from the remote node, checking the validity. If the CONNECT message is valid, then sending an ACK message. Otherwise, log the error and close the websocket.
3. Wait to receive an ACK message from the remote node, checking the validity. If the ACK message is valid, then sending a SECRET_FLAGS/OK message. Otherwise, log the error and close the websocket.
4. Wait to receive a SECRET_FLAGS/OK message from the remote node, checking the validity. If the message is valid, then sending a SECRET_FLAGS/OK message. Otherwise, log the error and close the websocket. If the receiving message is OK and sending OK message too, skip step 4 and consider the ws2p connection completely established.
5. Wait to receive an OK message from the remote node, checking the validity. If the OK message is valid, then sending an OK message and consider the ws2p connection completely established. Otherwise, log the error and close the websocket.
\*timeout : 15 seconds for connection to a conventional endpoint. 30 seconds for connection to a hidden tor service.
For the remote node, only the first step change :
1. Sudden receipt of a CONNECT message. If the CONNECT message is valid, then sending my own CONNECT message and an ACK message at the same time. Otherwise, log the error and close the websocket.
**sync connection**
The connection steps are the same for a sync connection excepts :
1. The CONNECT message contains a chunkstamp telling from which chunk the asking node wants to synchronize from
2. After having sent it's ACK message, the responding node also sends a SYNC_INFO message described below
WARNING: independently of all these rules, each implementation must integrate its own anti-spam protections. Any connection can be rejected if the situation requires it (attack by denial of service for example).
### CONNECT message
```json
{
"challenge": [...], // 32 bytes hash
"api_features": [...], // list of 4 integers corresponding to api_features (see in endpoint definition)
"connect_type": {...}, // see below
"peer_card": {...} // look at the peercard definition above
}
```
challenge := random hash generated by the sending node of the CONNECT message, the receiving node will then have to sign this challenge and then send this signature in its ACK message to prove that it has the corresponding private key to the public key it indicates.
api_features := This is exactly the same type as the field of the same name in the endpoints. But private WS2P nodes do not declare endpoints, so they must be able to indicate in the CONNECT message which features they support. Public WS2P nodes also fill this field, so any changes in the configuration of a public node will be applied on the 1st new connection. (If this was not the case, we would have to wait for the update of the peer record).
connect_type := See ConnectType below
peer_card := See PeerCard above
#### ConnectType type definition
ConnectType represent the connection type. It can be:
- Incoming: *connect message sent when receiving an incoming websocket connection*
- Client: *connect message sent by a client*
- Server: *connect message sent between nodes for a normal connection*
- Sync: *connect message sent for a synchronization request*
- SyncAskChunk: *connect message sent for a chunk request*
Incoming, Client and Server are represented by a string.
Sync is represented by an object
```
{
"Sync": {
"from_blockstamp": "128310-000002A569DCEED62227CAC0ABDFE6B2647B21B3193A40398CD12BC2A95C24D9"
}
},
```
SyncAskChunk is reprensented by an object
```json
{
"SyncAskChunk": "128310-000002A569DCEED62227CAC0ABDFE6B2647B21B3193A40398CD12BC2A95C24D9",
}
```
### ACK message
```json
{
"hash": [...] // 32 bytes hash
}
```
Each node must sign the challenge of the other to prove that it's in possession of the private key corresponding to the public key under which it identifies.
challenge := The challenge given by the other node in their CONNECT message.
The message is already signed at the container level, so there is no need to repeat the signature in the ACK message payload. On the other hand, the challenge to be signed must be in the payload.
### SECRET FLAGS message
TODO
If it is valued, the `member_proof` field must contain a signature of the challenge* send by other node in their CONNECT message.
_*It's the remote challenge for the signatory, and the local challenge for the verifier._
#### WS2PSecretFlags type definition
| bit | flag |
|:---------:|-----------------|
| 0000_0001 | LOW_FLOW_DEMAND |
LOW_FLOW_DEMAND := The sender node of this message indicates that it's behind a low speed connection and therefore care must be taken to transmit low volumes of data to it. All the code will do is to be careful to send only essential information and not already sent, which requires caching and pre-processing, that's why we don't do it for normal connections.
### OK message
In most cases, the OK message is empty, it simply indicates that the remote node accepts the connection establishment. If the local node has also sent an OK message, then it considers the connection as fully established.
But sometimes the OK message can also transmit additional informations :
```json
{
"prefix": ... // null or non-zero integer
}
```
prefix := In ws2p v2, member nodes have the option of not publicly communicating their prefix. It will only reveal their prefix to the node of their choice among those with which they establish a connection. For example, a member node may decide to share its prefix only with other member nodes of the same key.
If this field is zero, it means that the remote node does not want to reveal its prefix (the prefix being necessarily greater than or equal to 1).
#### SYNC_INFO message
```json
{
"chunk_size": 500,
"target_blockstamp": {
"id": 0,
"hash": [...] // 32 bytes hash
},
"milestones": [...], // list of 32 bytes hashes
"peer_cards": [...] // list of peercards
}
```
chunk_size := defines the size of the block used in milestones. A 1000-block chunk is a good compromise between efficiency of compression and duration of transfer
target_blockstamp := Indicates the current blockstamp of the message sender node. This blockstamp will be the target to reach for the node being synchronized.
milestones := Hash table of the last block of each chunk. We do not need the block numbers, we know them. Here the remote node sends the hashs of all these chunk, which correspond to the current hashs of all the blocks having a number in 250 module 249, in ascending order.
peer_cards := see [definition of binary peer card](#peer-card-binary-format)
### KO message
```json
0 // number representing reason
```
#### reason
| value | significance |
|:-----:|------------------|
| 0 | TIMEOUT |
| 1 | FULL |
| 2 | LOW_NO_SYNC |
| 3 | DIFFERENT_BRANCH |
## WS2Pv2 Messages
### REQUESTS
| data name | size in bytes | data type |
|:------------:|---------------|-------------|
| request_id | 4 | u32 |
| request_type | 4 | u32 |
| param_1 | ? | ? |
| param_2 | ? | ? |
#### request_type interpretation
| value | significance | param_1 (type) | param_2 (type) |
|:-----:|--------------------|-------------------------|--------------------|
| 0 | EMPTY_REQUEST | - | - |
| 1 | CURRENT_BLOCKSTAMP | - | - |
| 2 | BLOCKS_HASHS | begin_block_id (u32) | blocks_count (u16) |
| 3 | CHUNK | begin_block_id (u32) | blocks_count (u16) |
| 4 | CHUNK_BY_HASH | chunkstamp (Blockstamp) | - |
| 5 | WOT_POOL | folders_count (u16) | min_cert (u8) |
BLOCKS_HASHS := In case of fork, to quickly find the fork point, the node will request the hashes of the ForkWindowsSize of the local blockchains of the other nodes.
It would be counterproductive to ask directly for the entire blocks, when you will only need them if you actually decide to stack the corresponding branch.
CHUNK_BY_HASH : During synchronization, chunk is requested by Chunkstamp (= Blockstamp of the last block of the chunk).
WOT_POOL := For network performance reasons, a Duniter-Rust* node never shares its entire wot pool at once. It randomly selects `folders_count` folders among those having received at least `min_cert` certifications. It's the requesting node that sets the values of `min_cert` and `folders_count` according to its connection rate, its configuration and the rate of new folders it has obtained in these previous requests.
_*This is part of the specificities of WS2P requests in binary format, which will be introduced in all implementations in the future._
### REQUESTS RESPONSES
| data name | size in bytes | data type |
|:----------------:|---------------|------------|
| request_id | 4 | u32 |
| response_type | 4 | u32 |
| response_content | ? | ? |
#### response_type interpretation
| value | significance |
|:-----:|--------------------|
| 0 | EMPTY_RESPONSE |
| 1 | BAD_REQUEST |
| 2 | CURRENT_BLOCKSTAMP |
| 3 | BLOCKS_HASHS |
| 4 | CHUNK |
| 5 | WOT_POOL |
EMPTY_RESPONSE := `response_content` field is absent.
BAD_REQUEST := Used only in debug mode. In production, bad requests are simply ignored (The server does not respond).
#### BAD_REQUEST response
| data name | size in bytes | data type |
|:------------:|---------------|------------|
| reason | ? | String |
_*The total payload size of any WS2P message is indicated in the metadata, field `reason` being the only one of unknown size, its size is deduced by calculation._
#### CURRENT_BLOCKSTAMP response
| data name | size in bytes | data type |
|:------------:|---------------|------------|
| blockstamp | 36 | Blockstamp |
#### BLOCKS_HASHS response
| data name | size in bytes | data type |
|:------------:|----------------|---------------|
| hashs_count | 8 | u64 |
| hashs | 32*hashs_count | [[u8; 32]; ?] |
#### CHUNK response
| data name | size in bytes | data type |
|:------------:|---------------|---------------|
| blocks_count | 8 | u64 |
| blocks | ? | [BlockDocument; blocks_count] |
#### WOT_POOL response
| data name | size in bytes | data type |
|:-------------:|---------------|------------------------------|
| certs_count | 8 | u64 |
| certs | ? | CompactCertificationDocument |
| folders_count | 8 | u64 |
| folders | ? | WotPoolFolder |
CompactCertificationDocument type description :
| data name | size in bytes | data type |
|:-------------:|---------------|---------------------|
| issuer* | ? | PubKeyBox |
| target* | ? | PubKeyBox |
| block_id | 4 | u32 |
| signature | ? | SigBox |
WotPoolFolder type description :
| data name | size in bytes | data type |
|:---------------:|-----------------|------------------------------|
| identity | ? | CompactIdentityDocument |
| membership | ? | CompactPoolMembershipDoc |
| certs_count | 8 | u64 |
| certs | ? | CompactCertificationDocument |
CompactIdentityDocument type description :
| data name | size in bytes | data type |
|:-------------:|---------------|---------------------|
| uid | ? | String |
| blockstamp | 36 | Blockstamp |
| pubkey | ? | PubkeyBox |
| signature | ? | SigBox |
CompactPoolMembershipDoc type description :
| data name | size in bytes | data type |
|:-------------:|---------------|---------------------|
| blockstamp | 36 | Blockstamp |
| signature | ? | SigBox |
### HEADS v2
| data name | size in bytes | data type |
|:------------:|---------------|-----------|
| head_content | ? | String |
### HEADs v3
| data name | size in bytes | data type |
|:------------------:|-------------------|------------|
| api_outgoing_conf | 1 | u8 |
| api_incoming_conf | 1 | u8 |
| free_mirror_rooms | 1 | u8 |
| low_priority_rooms | 1 | u8 |
| node_id | 4 | u32 |
| pubkey | ? | PubkeyBox |
| blockstamp | 36 | Blockstamp |
| software_name | ? | String |
| soft_version | ? | String |
| signature | ? | SigBox |
| step | 1 | u8 |
api_outgoing_conf := See "api_outgoing_conf interpretation".
api_incoming_conf := See "api_incoming_conf interpretation".
free_member_rooms := Number of rooms available for incoming ws2p connections.
low_priority_rooms := Number of rooms occupied by nodes with lower priority than a member. The number of rooms available for a member is equal to `free_member_rooms` + `low_priority_rooms`.
node_id := Field randomly drawn by the node during its 1st configuration.
pubkey := Public key of the network keychain of the node.
blockstamp := Current blockstamp of the node.
signature := Direct signature of the byte vector from `software_size` to `soft_version` included.
step := Number of head rebounds (unsigned field).
#### api_outgoing_conf interpretation
| bit | signification |
|:---------:|---------------------------|
| 0000_0001 | WS2P_PRIVATE_CLEAR |
| 0000_0010 | WS2P_PRIVATE_TLS |
| 0000_0100 | WS2P_PRIVATE_TOR_TO_CLEAR |
| 0000_1000 | WS2P_PRIVATE_TOR_TO_TOR |
There are still 4 flags for future network layers that could be implemented in the future.
#### api_incoming_conf interpretation
| bit | signification |
|:---------:|-------------------|
| 0000_0001 | WS2P_PUBLIC_CLEAR |
| 0000_0010 | WS2P_PUBLIC_TLS |
| 0000_0100 | WS2P_PUBLIC_TOR |
There are still 5 flags for future network layers that could be implemented in the future.
#### HEAD v3 Raw format
Approximate description :
VERSION:CURRENCY:OUTGOING_CONF:INCOMING_CONF:FREE_MIRROR_ROOMS:FREE_MEMBER_ROOMS:NODE_ID:PUBLIC_KEY:BLOCKSTAMP:SOFTWARE:SOFT_VERSION
SIGNATURE
STEP
_/!\ This description is approximate, it allows you to quickly understand the format but is not authoritative for developing a code to parse the format exactly. If you want to develop a code using this format, please refer to its PEG grammar._
Example :
3:g1:0:0:0:0:0:7iMV3b6j2hSj5WtrfchfvxivS9swN3opDgxudeHq64fb:50-000005B1CEB4EC5245EF7E33101A330A1C9A358EC45A25FC13F78BB58C9E7370:durs:0.2.0-a
vwlxpkCbv83qYSiClYA/GD35hs0AsZBnqv7uoE8hqlarT2c6jVRKhjp8JBqmRI7Se4IDwC2owk0mF4CglvyACQ==
2
HEAD v3 PEG grammar :
```
// Single character rules
nl = _{ "\n" }
no_zero_hexa_lower = @{ '1'..'9' | 'a'..'f' }
hexa_lower = @{ ASCII_DIGIT | 'a'..'f' }
hexa_upper = @{ ASCII_DIGIT | 'A'..'F' }
base58 = { !("O" | "I" | "l") ~ ASCII_ALPHANUMERIC }
base64 = { ASCII_ALPHANUMERIC | "+" | "/" }
// Numbers rules
tens = @{ '1'..'9' ~ ASCII_DIGIT }
u8_hundreds = @{ ("2" ~ ('0'..'4' ~ ASCII_DIGIT | ('0'..'5'){2})) | ("1" ~ ASCII_DIGIT{2}) }
u8 = @{ u8_hundreds | tens | ASCII_DIGIT }
no_zero_u_int = @{ '1'..'9' ~ ASCII_DIGIT* }
u_int = @{ "0" | no_zero_u_int }
// Usefull types rules
currency = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "-" | "_"){,255} }
block_id = @{ u_int }
hash = @{ hexa_upper{64} }
pubkey = @{ base58{43,44} }
blockstamp = ${ block_id ~ "-" ~ hash }
ed25519_sig = @{ base64{88} | (base64{87} ~ "=") | (base64{86} ~ "==") }
// HEADv3 rules
api_outgoing_conf = @{ u8 }
api_incoming_conf = @{ u8 }
free_member_rooms = @{ u8 }
free_mirror_rooms = @{ u8 }
software = @{ ASCII_ALPHA ~ ASCII_ALPHANUMERIC* }
soft_version = @{ ASCII_DIGIT ~ "." ~ ASCII_DIGIT ~ "." ~ ASCII_DIGIT ~ ("-" ~ ("a" | "b" | "rc") ~ ASCII_DIGIT ~ ("." ~ ASCII_DIGIT){0,2})? }
step = @{ u_int }
head_v3 = ${
"3:" ~ currency ~ ":" ~
api_outgoing_conf ~ ":" ~
api_incoming_conf ~ ":" ~
free_member_rooms ~ ":" ~
free_mirror_rooms ~ ":" ~
node_id ~ ":" ~
pubkey ~ ":" ~
blockstamp ~ ":" ~
software ~ ":" ~
soft_version ~ nl ~
ed25519_sig? ~
(nl ~ step)?
}
```
#### HEADv3 JSON format
This human-readable format will be used by all APIs that wish to provide heads in a human-readable format. For example, Client APIs.
```json
{
"content": "VERSION:CURRENCY:OUTGOING_CONF:INCOMING_CONF:FREE_MIRROR_ROOMS:FREE_MEMBER_ROOMS:NODE_ID:PUBLIC_KEY:BLOCKSTAMP:SOFTWARE:SOFT_VERSION",
"signature": "SIGNATURE", // Signature in base64
"step": 3
}
```
Example :
```json
{
"content": "3:g1:0:0:0:0:0:7iMV3b6j2hSj5WtrfchfvxivS9swN3opDgxudeHq64fb:50-000005B1CEB4EC5245EF7E33101A330A1C9A358EC45A25FC13F78BB58C9E7370:durs:0.2.0-a",
"signature": "vwlxpkCbv83qYSiClYA/GD35hs0AsZBnqv7uoE8hqlarT2c6jVRKhjp8JBqmRI7Se4IDwC2owk0mF4CglvyACQ==",
"step": 2,
}
```
### Documents messages
All WS2Pv2 documents are necessarily in Binary Format. When document binarization is coded in Duniter-ts, then it will be necessary to define a common binary format for each document. For performance reasons, we will probably opt for the format in which each document is signed.
We must therefore wait for the binarization of the blockchain protocol.
In the meantime, Durs nodes exchange documents directly as portable binary objects (this is a Rust feature).