Skip to content
Snippets Groups Projects
Select Git revision
  • ws2p_v2
  • master default
  • feature/protocol-v2s
  • 0016_checksum
  • tx_comment_encrypt
  • dubp_v13
  • hd_wallet
  • dubp-mnemonic
  • dubp_v12
  • tx-out-hash
10 results

0006_ws2p_v2.md

Blame
  • Duniter WS2P API v2

    This document details the specifications of WS2P v2.

    Contents

    FAQ

    What is WS2P

    WS2P means "Web Socket To Peer".

    WS2P is the inter-node network layer, it is the only guarantor of synchronization between the different nodes of the network, so its role is critical.

    WS2P is the network part of the Duniter protocol

    WS2P is exclusively based on websocket technology.

    Why create a synchronization process

    The download phase of the synchronization process is essential, it allows a new peer to download the entire blockchain quickly (then indexed in a second phase called "apply").

    Historically, this process was managed by the BMA crawler, but since the arrival of WS2P v1 the BMA crawler is no longer in service, and we keep its code only for sync command. So for maintainability reasons, it is necessary to migrate this process outside the BMA crawler, then delete the BMA crawler code.

    Without its crawling part, BMA has become a Client API, which is to be removed and replaced by a new Client API named GVA (see GVA RFC).

    Finally, the synchronization process is more about the inter-node network than the interface with clients, so it makes sense to migrate this process into WS2P.

    Why remove shared peer records

    Historically, shared peer files had been set up to allow the BMA crawler to automatically detect the use of multiple member nodes with the same key and automatically assign a different prefix to different nodes of the same member so that they calculate different nonces spaces.

    Since Duniter-ts 1.6, the user can now manually assign a different prefix to each of his machines.

    In addition, WS2P v1 has provided a new way to uniquely identify nodes (the node id, we will discuss this later). This new way of uniquely identifying nodes opens the way to possible new automatic prefix allocation mechanisms that will not require a shared peer record.

    Finally, shared peer cards have a big drawback: a peer cannot prune obsolete endpoints of other peer sharing the same key because it cannot easily evaluate if these other endpoints are obsolete. In absolute terms, it is possible to bypass problems but it is simpler to directly delete shared peer records, and as we no longer need them: Occam's razor.

    What is the binary format used for serialization

    To allow easy interfacing between different implementations, WS2P v2 defines network features (detailed later in the RFC) allowing each WS2P Public to declare in its endpoint the list of features it supports. The default format is CBOR, a widespread binarization format based on JSON. But for byte-efficiency reasons, Dunitrust defined an other serialization format based on the Rust crate bincode.

    Conventions

    Raw format

    All documents in raw format are explained via a PEG grammar.
    The raw format is a plain text format used to sign a document or verify its signature.
    Only documents whose signature must be verifiable by a client software are signed in this way. This applies in particular to peer cards and HEADs.
    WS2pV2 messages are signed in binary format.

    The PEG grammars of raw formats are presented according to the pest syntax (a particular implementation of PEG).

    Common structures

    A certain number of structure like hash appear repeatedly in the messages. Here we describe it once and refer to it later.

    32 byte number

    A 32-bytes number is represented in JSON by an array of 32 8-bits numbers.

    [8, 56, 211, ..., 12] // array of 32 8-bits numbers

    64 byte number

    A 64-bytes number is represented in JSON by an array of 64 8-bits numbers.

    [8, 56, 211, ..., 12] // array of 64 8-bits numbers

    Hash, Pubkey, Signature

    Hashes or pubkeys or signatures are just numbers, most often 32B or 64B one.

    Message encapsulation

    All WS2P v2 message are encapsuled under PKSTL v1 protocol.

    Each WS2Pv2 message is binarized and then placed in the CUSTOM_DATAS field of the PKSTL message that encapsulates it.

    Several binary serialization formats are available: CBOR and Bincode. The message defined below are written in JSON, allowing straightforward serialization to CBOR. For the Bincode serialization, see Bincode format reference.

    All WS2Pv2 messages are structured as follows (CBOR format) :

    ```json
    {
        "ws2p_version": 2,
        "currency_name": "g1",
        "issuer_node_id": 395,
        "issuer_pubkey": {
            "algo": "Ed25519",
            "content": [52, 147, .., 93], // 32 bytes pubkey
        },
        "payload": {
            "type": "MESSAGE_TYPE_NAME",
            "content": {} // depends on payload.type
        },
    }
    ```

    ws2p_version := This field is placed first so that future versions of WS2P are not constrained on the other fields, the only constraint will be to start the message with the version number stored in u32.

    currency_name := Empty string is allowed, it allow the user to synchronize his node without having to manually enter the currency on which he synchronizes. The node will then adopt the currency that is specified in the CONNECT message it will receive from the selected reference node.

    * The type of payload is determined by the content of message_type.

    payload.type

    Exhaustive list of possible payload.type values:

    message type payload.type value Hashed Signed
    CONNECT "Connect" no yes
    ACK "Ack" no yes
    SECRET_FLAGS "SecretFlags" no yes
    OK "Ok" no yes
    KO "Ko" no yes
    REQUEST "Request" yes no
    REQUEST_RESPONSE "ReqRes" yes no
    PEERS "Peers" yes no
    HEADS_V2 "HeadsV2" yes no
    HEADS_V3 "HeadsV3" yes no
    BLOCKS "Blocks" yes no
    PENDING_IDENTITIES "PendingIdentities" yes no
    PENDING_MEMBERSHIPS "PendingMemberships" yes no
    PENDING_CERTS "PendingCerts" yes no
    PENDING_REVOCATIONS "PendingRevocations" yes no
    PENDING_TXS "PendingTxs" yes no

    Endpoints v2

    ```json
    {
        "api": "WS2P",
        "api_version": 2,
        "network_features": [], // array of bytes, defined below
        "api_features": [], // array of bytes, defined below
        "host": "subdomain.domaine.tld",
        "ip_v4": "5.135.188.170",
        "ip_v6": "2001:41d0:8:c5aa::1",
        "port": 443,
        "path": "ws2p",
    }
    ```

    utf8 format

    The utf8 format is used to display the endpoint in a human-readable format. It is also in this format that the user can manually enter an endpoint.

    General utf8 endpoint format :

    API_NAME VERSION NF1 .. NFn AF PORT HOST IP4 [IP6] PATH

    VERSION := version number prefixed by V (example : V1) NF := NETWORK_FEATURE AF := API_FEATURES in hexadecimal prefixed by 0x (example : 0x01)

    Example:

    WS2P V2 TLS 0x7 443 g1.durs.info ws2p

    network_features

    The 8 bits represent booleans to define the presence or absence of 8 network features. WS2Pv2 defines only 4 features, the remaining 4 are undefined and are in anticipation of future Ğfeatures.

    Network features :

    bit feature
    0000_0001 HTTP
    0000_0010 WS
    0000_0100 S
    0000_1000 TOR

    HTTP := This feature indicates that the endpoint should be contacted with http protocol.

    WS := This feature indicates that the endpoint should be contacted with websocket protocol.

    S := This feature indicates that the endpoint should be contacted with an SSL/TLS overlay (HTTPS or WSS or any other protocol that support TLS).

    TOR := This feature indicates that the endpoint must be contacted via the tor network (hidden service).

    Note about the network protocol to choose: Each API must define a default network protocol. If no network protocol is specified in the network features, the default API network protocol is chosen.

    For example, the default network protocol for WS2P is ws://, it's the only one possible for this API, so there is no need to indicate it in the network features.

    api_features

    The interpretation of this field depends on the API because it represents API-specific features. Here is the interpretation for the WS2P API :

    WS2PFeatures type definition :

    bit feature
    0000_0100 RBC
    0000_0010 LOW

    RBC := Support Rust BinCode format

    LOW := Accept low speed connection requests

    WS2P v2 uses only 2 of 8 features. The 6 free bits can be used for future versions of WS2P.

    Peer card

    Signed document declaring all the endpoints of a peer.

    Peer card v11 is signed in raw format (without SIGNATURE itself of course).

    A node's peer record can also contain signatures of the node's network key. This allows, for example, to sign mirror nodes with known and trusted keys. The signed key must be signed with the blockstamp corresponding to the date of signature. More precisely, it is the text PUBKEY:BLOCKSTAMP that must be signed.

    CBOR format

    ```json
    {
        "version": "v11",
        "currency_name": "g1",
        "issuer": {
            "algo": "Ed25519",
            "content": [122, 11, .., 95] // 32 bytes pubkey
        },
        "node_id": 10623,
        "created_on": 542,
        "endpoints": [], // Array of endpoitns v2
        "endpoints_str": [], // Array of Strings (endpoitn v2 in utf8 format)
        "sig": {
            "algo": "Ed25519",
            "content": [187, 7, .., 23] // 64 bytes signature
        },
        "certifiers": [
            {
                "certifier": {
                    "algo": "Ed25519",
                    "content": [122, 11, .., 95] // 32 bytes pubkey
                },
                "blockstamp": {
                    "id": 327,
                    "hash": [85, 132, .., 27] // 32 bytes hash
                },
                "sig": {
                    "algo": "Ed25519",
                    "content": [187, 7, .., 23] // 64 bytes signature
                },
            }
        ]
    }
    ```

    Peer card Raw format

    Approximate description :

    VERSION:CURRENCY:NODE_ID:PUBLIC_KEY:CREATED_ON
    ENDPOINT 1
    ...
    ENDPOINT n
    SIGNATURE

    /!\ This description is approximate, it allows you to quickly understand the format but is not authoritative for developing a code to parse the format exactly. If you want to develop a code using this format, please refer to its PEG grammar.

    Example :

    11:g1:0:7iMV3b6j2hSj5WtrfchfvxivS9swN3opDgxudeHq64fb:50
    WS2P V2 S 7 g1.durs.ifee.fr 443 ws2p
    WS2P V2 S 7 84.16.72.210 443 ws2p
    EQ2D5almq2RNUi3XZNtTpjo9nWtJF0PzsCW7ROAzCQKiEtpI7/fW8Z23GJ2a/SIxfYSzlq/cZqksE4EoVe1rAw==

    Peer card v11 PEG grammar :

    ```pest
    // Single character rules
    nl = _{ "\n" }
    no_zero_hexa_lower = @{ '1'..'9' | 'a'..'f' }
    hexa_lower = @{ ASCII_DIGIT | 'a'..'f' }
    hexa_upper = @{ ASCII_DIGIT | 'A'..'F' }
    base58 = { !("O" | "I" | "l") ~ ASCII_ALPHANUMERIC }
    base64 = { ASCII_ALPHANUMERIC | "+" | "/" }
    
    // Numbers rules
    tens = @{ '1'..'9' ~ ASCII_DIGIT }
    u8_hundreds = @{ ("2" ~ ('0'..'4' ~ ASCII_DIGIT | ('0'..'5'){2})) | ("1" ~ ASCII_DIGIT{2}) }
    u8 = @{ u8_hundreds | tens | ASCII_DIGIT }
    no_zero_u_int = @{ '1'..'9' ~ ASCII_DIGIT* }
    u_int = @{ "0" | no_zero_u_int }
    
    // Usefull types rules
    currency = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "-" | "_"){,255} }
    block_id = @{ u_int }
    hash = @{ hexa_upper{64} }
    pubkey = @{ base58{43,44} }
    ed25519_sig = @{ base64{88} | (base64{87} ~ "=") | (base64{86} ~ "==") }
    
    // IP v6 rules
    ip6_seg  = _{ hexa_lower{1,4} }
    ip6_full = _{ (ip6_seg ~ ":"){7} ~ ip6_seg }
    ip6_no_full = @{
        ip6_seg? ~
        (":" ~ ip6_seg){0,6} ~
        "::" ~
        (ip6_seg ~ ":"){0,6} ~
        ip6_seg?
    }
    ip6_inner = @{ ip6_full | ip6_no_full }
    ip6 = _{ "[" ~ ip6_inner ~ "] " }
    
    // Endpoint v2 rules
    api_version_inner = @{ no_zero_u_int }
    api_version = _{ "V" ~ api_version_inner ~ " " }
    http = @{ "HTTP " }
    ws = @{ "WS " }
    tls = @{ "S " }
    tor = @{ "TOR " }
    network_features = _{ http? ~ ws? ~ tls? ~ tor? }
    api_features_inner = @{ (hexa_lower{2})+ | no_zero_hexa_lower }
    api_features = _{ "0x" ~ api_features_inner ~ " " }
    domain_name_part = @{ ASCII_ALPHA_LOWER ~ (alphanum_lower | "-" | "_")* }
    domain_name_ext = @{ alphanum_lower+ }
    domain_name_parts = @{ (domain_name_part ~ ".")+ ~ domain_name_ext }
    domain_name_onion = @{ alphanum_lower{16} ~ ".onion" }
    domain_name_inner = @{ domain_name_parts | domain_name_onion | domain_name_part }
    domain_name = _{ domain_name_inner ~ " " }
    ip4_inner = { u8 ~ "." ~ u8 ~ "." ~ u8 ~ "." ~ u8 }
    ip4 = _{ ip4_inner ~ " " }
    path = _{ " " ~ path_inner }
    endpoint_v2 = ${ api_name ~ " " ~ (api_version)? ~ (network_features)? ~ (api_features)? ~ ip4? ~ ip6? ~ domain_name? ~ port ~ path? }
    
    // Peer v11 rules
    peer_v11 = ${ "11:" ~ currency ~ ":" ~ node_id ~ ":" ~ pubkey ~ ":" ~ block_id ~ nl ~ (endpoint_v2 ~ nl)+ ~ ed25519_sig? }
    ```

    Peer card JSON stringified format

    This human-readable stringified format will be used by all APIs that wish to provide peer cards in a human-readable format. For example, Client APIs.

    ```json
    {
        "version": 11,
        "currency": 1,
        "node_id": "15af24db",
        "algorithm": "Ed25519",
        "pubkey": "D9D2zaJoWYWveii1JRYLVK3J4Z7ZH3QczoKrnQeiM6mx"
        "blockstamp": "128310-000002A569DCEED62227CAC0ABDFE6B2647B21B3193A40398CD12BC2A95C24D9",
        "endpoinds": [
            "WS2P 2 1 TLS 3 DEF LOW ABF 443 g1.domaine.tld ws2p"
        ],
        "signature": "e2BtWtLs3Uqw80bvOx8YGfmDLfA44/apzpZA8YfH+WtnmPY5r4XUlgvctPsq2bHVw3iPWxuxx5oJh0JHITrECw==",
        "certifiers": [
            "Ez4huJahi6qfG8eVjyc84CwVtpHagmfUTtRFazXY4G2h:81234-0000047F1D3BB30C24CBD9D60977FC9474D3A985DF88BB9C9CAFE9160D94AC17:oP5iHvXRx1mI+Nic621/tU42kk8Mr+m5wgNZiBPJf4vGSlGSPkTSbzMJgHehnCcEF/UB+xPoLZ2WK1AMODlFDg=="
        ]
    }
    ```

    Getting peer cards from other nodes

    Initially, nodes are totally endpoint agnostic and don't know any node. The ws2p module requires that at least one or more peer cards are already stored in the database when launching the node, it's up to the sync command to handle this (see "WS2P synchronisation" part).

    When node starts up, the WS2P module accesses the peer cards stored in a local database following synchronization.

    Priority of WS2P outcoming connections

    The WS2P module attempts to establish outcoming connections at the kick-start of the node and then every 10 minutes.
    At each attempt, all known endpoints are contacted until the outcoming quota is reached.
    If the outcoming quota is already reached at the beginning of an attempt, the least priority connection is removed in order to keep the network dynamic and scalable.

    The WS2P module does not attempt to connect randomly to any node among those of which it knows a WS2P endpoint, it classifies them according to the following criteria :

    1st criterion: network features

    Some network features, imply that the node prioritizes endpoints having such or such network feature. For example, if the node has the TOR feature, it will prioritize nodes that also have the TOR feature.

    Note: the first criterion takes precedence over the second one.

    2nd criterion: priority score

    The priority score of a node depends on its public key and is calculated as follows :
    by default the priority score is zero
    If the key is a member: +1
    If the key is preferred/privileged: +2
    If the key is the same as yourself: +4 (for multi-node)

    In case of equality, the order in which the endpoints will be contacted will depend on the sorting algorithm used according to the implementations, there are no rules, we can consider that it is pseudo-random.

    Establishing a WS2P connection

    After sorting all WS2P endpoints according to the criteria described in the previous section, Duniter tries to connect to the other nodes.

    normal connection

    1. Opening a websocket pointing to the remote endpoint. If the websocket has been successfully opened before the timeout*, send a CONNECT message. Otherwise, log the error and close the websocket.
    2. Wait to receive a CONNECT message from the remote node, checking the validity. If the CONNECT message is valid, then sending an ACK message. Otherwise, log the error and close the websocket.
    3. Wait to receive an ACK message from the remote node, checking the validity. If the ACK message is valid, then sending a SECRET_FLAGS/OK message. Otherwise, log the error and close the websocket.
    4. Wait to receive a SECRET_FLAGS/OK message from the remote node, checking the validity. If the message is valid, then sending a SECRET_FLAGS/OK message. Otherwise, log the error and close the websocket. If the receiving message is OK and sending OK message too, skip step 4 and consider the ws2p connection completely established.
    5. Wait to receive an OK message from the remote node, checking the validity. If the OK message is valid, then sending an OK message and consider the ws2p connection completely established. Otherwise, log the error and close the websocket.

    *timeout : 15 seconds for connection to a conventional endpoint. 30 seconds for connection to a hidden tor service.

    For the remote node, only the first step change :

    1. Sudden receipt of a CONNECT message. If the CONNECT message is valid, then sending my own CONNECT message and an ACK message at the same time. Otherwise, log the error and close the websocket.

    sync connection

    The connection steps are the same for a sync connection excepts :

    1. The CONNECT message contains a chunkstamp telling from which chunk the asking node wants to synchronize from
    2. After having sent it's ACK message, the responding node also sends a SYNC_INFO message described below

    WARNING: independently of all these rules, each implementation must integrate its own anti-spam protections. Any connection can be rejected if the situation requires it (attack by denial of service for example).

    CONNECT message

    ```json
    {
      "challenge": [...], // 32 bytes hash
      "api_features": [...], // list of 4 integers corresponding to api_features (see in endpoint definition)
      "connect_type": {...}, // see below
      "peer_card": {...} // look at the peercard definition above
    }
    ```

    challenge := random hash generated by the sending node of the CONNECT message, the receiving node will then have to sign this challenge and then send this signature in its ACK message to prove that it has the corresponding private key to the public key it indicates.

    api_features := This is exactly the same type as the field of the same name in the endpoints. But private WS2P nodes do not declare endpoints, so they must be able to indicate in the CONNECT message which features they support. Public WS2P nodes also fill this field, so any changes in the configuration of a public node will be applied on the 1st new connection. (If this was not the case, we would have to wait for the update of the peer record).

    connect_type := See ConnectType below

    peer_card := See PeerCard above

    ConnectType type definition

    ConnectType represent the connection type. It can be:

    • Incoming: connect message sent when receiving an incoming websocket connection
    • Client: connect message sent by a client
    • Server: connect message sent between nodes for a normal connection
    • Sync: connect message sent for a synchronization request
    • SyncAskChunk: connect message sent for a chunk request

    Incoming, Client and Server are represented by a string.

    Sync is represented by an object

    {
        "Sync": {
          "from_blockstamp": "128310-000002A569DCEED62227CAC0ABDFE6B2647B21B3193A40398CD12BC2A95C24D9"
        }
    },

    SyncAskChunk is reprensented by an object

    {
        "SyncAskChunk": "128310-000002A569DCEED62227CAC0ABDFE6B2647B21B3193A40398CD12BC2A95C24D9",
    }

    ACK message

    ```json
    {
    	"hash": [...] // 32 bytes hash
    }
    ```

    Each node must sign the challenge of the other to prove that it's in possession of the private key corresponding to the public key under which it identifies.
    challenge := The challenge given by the other node in their CONNECT message. The message is already signed at the container level, so there is no need to repeat the signature in the ACK message payload. On the other hand, the challenge to be signed must be in the payload.

    SECRET FLAGS message

    TODO

    If it is valued, the member_proof field must contain a signature of the challenge* send by other node in their CONNECT message.

    *It's the remote challenge for the signatory, and the local challenge for the verifier.

    WS2PSecretFlags type definition

    bit flag
    0000_0001 LOW_FLOW_DEMAND

    LOW_FLOW_DEMAND := The sender node of this message indicates that it's behind a low speed connection and therefore care must be taken to transmit low volumes of data to it. All the code will do is to be careful to send only essential information and not already sent, which requires caching and pre-processing, that's why we don't do it for normal connections.

    OK message

    In most cases, the OK message is empty, it simply indicates that the remote node accepts the connection establishment. If the local node has also sent an OK message, then it considers the connection as fully established.
    But sometimes the OK message can also transmit additional informations :

    ```json
    {
      "prefix": ... // null or non-zero integer 
    }
    ```

    prefix := In ws2p v2, member nodes have the option of not publicly communicating their prefix. It will only reveal their prefix to the node of their choice among those with which they establish a connection. For example, a member node may decide to share its prefix only with other member nodes of the same key. If this field is zero, it means that the remote node does not want to reveal its prefix (the prefix being necessarily greater than or equal to 1).

    SYNC_INFO message

    ```json
    {
      "chunk_size": 500,
      "target_blockstamp": {
        "id": 0,
        "hash": [...] // 32 bytes hash
      },
      "milestones": [...], // list of 32 bytes hashes
      "peer_cards": [...] // list of peercards
    }
    ```

    chunk_size := defines the size of the block used in milestones. A 1000-block chunk is a good compromise between efficiency of compression and duration of transfer

    target_blockstamp := Indicates the current blockstamp of the message sender node. This blockstamp will be the target to reach for the node being synchronized.

    milestones := Hash table of the last block of each chunk. We do not need the block numbers, we know them. Here the remote node sends the hashs of all these chunk, which correspond to the current hashs of all the blocks having a number in 250 module 249, in ascending order.

    peer_cards := see definition of binary peer card

    KO message

    ```json
    0 // number representing reason
    ```

    reason

    value significance
    0 TIMEOUT
    1 FULL
    2 LOW_NO_SYNC
    3 DIFFERENT_BRANCH

    WS2Pv2 Messages

    REQUESTS

    data name size in bytes data type
    request_id 4 u32
    request_type 4 u32
    param_1 ? ?
    param_2 ? ?

    request_type interpretation

    value significance param_1 (type) param_2 (type)
    0 EMPTY_REQUEST - -
    1 CURRENT_BLOCKSTAMP - -
    2 BLOCKS_HASHS begin_block_id (u32) blocks_count (u16)
    3 CHUNK begin_block_id (u32) blocks_count (u16)
    4 CHUNK_BY_HASH chunkstamp (Blockstamp) -
    5 WOT_POOL folders_count (u16) min_cert (u8)

    BLOCKS_HASHS := In case of fork, to quickly find the fork point, the node will request the hashes of the ForkWindowsSize of the local blockchains of the other nodes. It would be counterproductive to ask directly for the entire blocks, when you will only need them if you actually decide to stack the corresponding branch.

    CHUNK_BY_HASH : During synchronization, chunk is requested by Chunkstamp (= Blockstamp of the last block of the chunk).

    WOT_POOL := For network performance reasons, a Duniter-Rust* node never shares its entire wot pool at once. It randomly selects folders_count folders among those having received at least min_cert certifications. It's the requesting node that sets the values of min_cert and folders_count according to its connection rate, its configuration and the rate of new folders it has obtained in these previous requests.

    *This is part of the specificities of WS2P requests in binary format, which will be introduced in all implementations in the future.

    REQUESTS RESPONSES

    data name size in bytes data type
    request_id 4 u32
    response_type 4 u32
    response_content ? ?

    response_type interpretation

    value significance
    0 EMPTY_RESPONSE
    1 BAD_REQUEST
    2 CURRENT_BLOCKSTAMP
    3 BLOCKS_HASHS
    4 CHUNK
    5 WOT_POOL

    EMPTY_RESPONSE := response_content field is absent.

    BAD_REQUEST := Used only in debug mode. In production, bad requests are simply ignored (The server does not respond).

    BAD_REQUEST response

    data name size in bytes data type
    reason ? String

    *The total payload size of any WS2P message is indicated in the metadata, field reason being the only one of unknown size, its size is deduced by calculation.

    CURRENT_BLOCKSTAMP response

    data name size in bytes data type
    blockstamp 36 Blockstamp

    BLOCKS_HASHS response

    data name size in bytes data type
    hashs_count 8 u64
    hashs 32*hashs_count [[u8; 32]; ?]

    CHUNK response

    data name size in bytes data type
    blocks_count 8 u64
    blocks ? [BlockDocument; blocks_count]

    WOT_POOL response

    data name size in bytes data type
    certs_count 8 u64
    certs ? CompactCertificationDocument
    folders_count 8 u64
    folders ? WotPoolFolder

    CompactCertificationDocument type description :

    data name size in bytes data type
    issuer* ? PubKeyBox
    target* ? PubKeyBox
    block_id 4 u32
    signature ? SigBox

    WotPoolFolder type description :

    data name size in bytes data type
    identity ? CompactIdentityDocument
    membership ? CompactPoolMembershipDoc
    certs_count 8 u64
    certs ? CompactCertificationDocument

    CompactIdentityDocument type description :

    data name size in bytes data type
    uid ? String
    blockstamp 36 Blockstamp
    pubkey ? PubkeyBox
    signature ? SigBox

    CompactPoolMembershipDoc type description :

    data name size in bytes data type
    blockstamp 36 Blockstamp
    signature ? SigBox

    HEADS v2

    data name size in bytes data type
    head_content ? String

    HEADs v3

    data name size in bytes data type
    api_outgoing_conf 1 u8
    api_incoming_conf 1 u8
    free_mirror_rooms 1 u8
    low_priority_rooms 1 u8
    node_id 4 u32
    pubkey ? PubkeyBox
    blockstamp 36 Blockstamp
    software_name ? String
    soft_version ? String
    signature ? SigBox
    step 1 u8

    api_outgoing_conf := See "api_outgoing_conf interpretation".

    api_incoming_conf := See "api_incoming_conf interpretation".

    free_member_rooms := Number of rooms available for incoming ws2p connections.

    low_priority_rooms := Number of rooms occupied by nodes with lower priority than a member. The number of rooms available for a member is equal to free_member_rooms + low_priority_rooms.

    node_id := Field randomly drawn by the node during its 1st configuration.

    pubkey := Public key of the network keychain of the node.

    blockstamp := Current blockstamp of the node.

    signature := Direct signature of the byte vector from software_size to soft_version included.

    step := Number of head rebounds (unsigned field).

    api_outgoing_conf interpretation

    bit signification
    0000_0001 WS2P_PRIVATE_CLEAR
    0000_0010 WS2P_PRIVATE_TLS
    0000_0100 WS2P_PRIVATE_TOR_TO_CLEAR
    0000_1000 WS2P_PRIVATE_TOR_TO_TOR

    There are still 4 flags for future network layers that could be implemented in the future.

    api_incoming_conf interpretation

    bit signification
    0000_0001 WS2P_PUBLIC_CLEAR
    0000_0010 WS2P_PUBLIC_TLS
    0000_0100 WS2P_PUBLIC_TOR

    There are still 5 flags for future network layers that could be implemented in the future.

    HEAD v3 Raw format

    Approximate description :

    VERSION:CURRENCY:OUTGOING_CONF:INCOMING_CONF:FREE_MIRROR_ROOMS:FREE_MEMBER_ROOMS:NODE_ID:PUBLIC_KEY:BLOCKSTAMP:SOFTWARE:SOFT_VERSION
    SIGNATURE
    STEP

    /!\ This description is approximate, it allows you to quickly understand the format but is not authoritative for developing a code to parse the format exactly. If you want to develop a code using this format, please refer to its PEG grammar.

    Example :

    3:g1:0:0:0:0:0:7iMV3b6j2hSj5WtrfchfvxivS9swN3opDgxudeHq64fb:50-000005B1CEB4EC5245EF7E33101A330A1C9A358EC45A25FC13F78BB58C9E7370:durs:0.2.0-a
    vwlxpkCbv83qYSiClYA/GD35hs0AsZBnqv7uoE8hqlarT2c6jVRKhjp8JBqmRI7Se4IDwC2owk0mF4CglvyACQ==
    2

    HEAD v3 PEG grammar :

    ```
    // Single character rules
    nl = _{ "\n" }
    no_zero_hexa_lower = @{ '1'..'9' | 'a'..'f' }
    hexa_lower = @{ ASCII_DIGIT | 'a'..'f' }
    hexa_upper = @{ ASCII_DIGIT | 'A'..'F' }
    base58 = { !("O" | "I" | "l") ~ ASCII_ALPHANUMERIC }
    base64 = { ASCII_ALPHANUMERIC | "+" | "/" }
    
    // Numbers rules
    tens = @{ '1'..'9' ~ ASCII_DIGIT }
    u8_hundreds = @{ ("2" ~ ('0'..'4' ~ ASCII_DIGIT | ('0'..'5'){2})) | ("1" ~ ASCII_DIGIT{2}) }
    u8 = @{ u8_hundreds | tens | ASCII_DIGIT }
    no_zero_u_int = @{ '1'..'9' ~ ASCII_DIGIT* }
    u_int = @{ "0" | no_zero_u_int }
    
    // Usefull types rules
    currency = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "-" | "_"){,255} }
    block_id = @{ u_int }
    hash = @{ hexa_upper{64} }
    pubkey = @{ base58{43,44} }
    blockstamp =  ${ block_id ~ "-" ~ hash }
    ed25519_sig = @{ base64{88} | (base64{87} ~ "=") | (base64{86} ~ "==") }
    
    // HEADv3 rules
    api_outgoing_conf = @{ u8 }
    api_incoming_conf = @{ u8 }
    free_member_rooms = @{ u8 }
    free_mirror_rooms = @{ u8 }
    software = @{ ASCII_ALPHA ~ ASCII_ALPHANUMERIC* }
    soft_version = @{ ASCII_DIGIT ~ "." ~ ASCII_DIGIT ~ "." ~ ASCII_DIGIT ~ ("-" ~ ("a" | "b" | "rc") ~ ASCII_DIGIT ~ ("." ~ ASCII_DIGIT){0,2})? }
    step = @{ u_int }
    
    head_v3 = ${
        "3:" ~ currency ~ ":" ~
        api_outgoing_conf ~ ":" ~
        api_incoming_conf ~ ":" ~
        free_member_rooms ~ ":" ~
        free_mirror_rooms ~ ":" ~
        node_id ~ ":" ~
        pubkey ~ ":" ~
        blockstamp ~ ":" ~
        software ~ ":" ~
        soft_version ~ nl ~
        ed25519_sig? ~
        (nl ~ step)?
    }
    ```

    HEADv3 JSON format

    This human-readable format will be used by all APIs that wish to provide heads in a human-readable format. For example, Client APIs.

    ```json
    {
        "content": "VERSION:CURRENCY:OUTGOING_CONF:INCOMING_CONF:FREE_MIRROR_ROOMS:FREE_MEMBER_ROOMS:NODE_ID:PUBLIC_KEY:BLOCKSTAMP:SOFTWARE:SOFT_VERSION",
        "signature": "SIGNATURE", // Signature in base64
        "step": 3
    }
    ```

    Example :

    ```json
    {
        "content": "3:g1:0:0:0:0:0:7iMV3b6j2hSj5WtrfchfvxivS9swN3opDgxudeHq64fb:50-000005B1CEB4EC5245EF7E33101A330A1C9A358EC45A25FC13F78BB58C9E7370:durs:0.2.0-a",
        "signature": "vwlxpkCbv83qYSiClYA/GD35hs0AsZBnqv7uoE8hqlarT2c6jVRKhjp8JBqmRI7Se4IDwC2owk0mF4CglvyACQ==",
        "step": 2,
    }
    ```

    Documents messages

    All WS2Pv2 documents are necessarily in Binary Format. When document binarization is coded in Duniter-ts, then it will be necessary to define a common binary format for each document. For performance reasons, we will probably opt for the format in which each document is signed.

    We must therefore wait for the binarization of the blockchain protocol.

    In the meantime, Durs nodes exchange documents directly as portable binary objects (this is a Rust feature).