diff --git a/rfc/0006_annex/bincode.md b/rfc/0006_annex/bincode.md
new file mode 100644
index 0000000000000000000000000000000000000000..588a0ba22ec111492712630402d28eb1d02655f5
--- /dev/null
+++ b/rfc/0006_annex/bincode.md
@@ -0,0 +1,236 @@
+# Bincode serialization format for ws2pv2
+
+Bincode is an alternative serialization format supported by nodes having "RBC" in their api features. The tables given here correspond to the successive fields in the bincode version of the messages.
+
+## Primitive types
+
+u8 : Unsigned 8-bit integer.
+u16 : Unsigned 16-bit integer.
+u32 : Unsigned 32-bit integer.
+u64 : Unsigned 64-bit integer.
+i8 : Signed 8-bit integer.
+i16 : Signed 16-bit integer.
+i32 : Signed 32-bit integer.
+i64 : Signed 64-bit integer.
+bool : boolean stored on 8 bits (0x00 = false, 0x01 = true, any other value must generate an error).
+0 : Corresponds to data that must be filled with bits to zero (for example padding).
+
+## Useful non-primitive types
+
+### (T, U)
+
+(T, U) = Tuples of 2 elements. the 1st type T and the second type U.
+A tuple is simply a concatenation of types written one after the other.
+A tuple can be of any size (no maximum limit), but its size is necessarily fixed.
+The different elements may be of different types, but the order cannot change.
+
+### [T; n]
+
+Array of n elements of type T.
+If the size of the array is variable, its size (in number of cells) is stored on 8 bytes.
+
+| data | size in bytes | data type |
+| :---------: | ------------: | ------------: |
+| cells_count | 8 | u64 |
+| content* | ?*cells_count | (T1, T2, ...) |
+
+All cells in the table are written one after the other without a separator, _like_ a tuple of `cells_count` elements of the same type.
+
+**/*\ CAUTION: If the size of the array is static (=a literal constant) then the array is serialized without a size field. **
+For example, the content of a public key ed25519 is a 32-byte array, (a [u8; 32] therefore), it is serialized directly without a size field.
+
+### Opt(T)
+
+Indicates that a field is optional. The optional fields are preceded by a 1-byte boolean that indicates whether or not the field is present.
+
+| data | size in bytes | data type |
+| :-----: | ------------: | --------: |
+| is_some | 1 | u8 |
+| content | ? | T |
+
+If `is_some` is equal to `1`, field `content` is present.
+If `is_some` is equal to `0`, field `content` is missing.
+Any other value of `is_some` is prohibited and will invalidate the entire message.
+
+### String
+
+String formatted in utf8 and [NFKC normalized](https://fr.wikipedia.org/wiki/Normalisation_Unicode#NFKC).
+
+**/!\ WARNING: All strings are systematically prefixed with their size, including for an empty string. The size is stored on 8 bytes as shown in the table below :**
+
+| data | size in bytes | data type |
+| :------: | ------------: | -------------: |
+| str_size | 8 | u64 |
+| content | str_size | [u8; str_size] |
+
+content := String in binary utf8 with [NFKC normalization](https://fr.wikipedia.org/wiki/Normalisation_Unicode#NFKC).
+
+If the string is empty, field `content` is missing and `str_size` is equal to zero.
+
+### Blockstamp
+
+| data | size in bytes | data type |
+| :--------: | ------------: | --------: |
+| block_id | 4 | u32 |
+| block_hash | 32 | [u8; 32] |
+
+### KeysAlgorithm
+
+| algorithm | u32 |
+| :-------: | ---: |
+| Ed25519 | 0 |
+| Schnorr | 1 |
+
+### PubkeyBox
+
+Contains the signatory's public key.
+
+| data | size in bytes | data type |
+| :-------: | ------------: | ------------: |
+| algorithm | 4 | KeysAlgorithm |
+| content | ? | ?* |
+
+_*The type of field `content` depends on the algorithm. In the case of Ed25519, `content` is a 32-byte array containing the public key._
+
+### SigBox
+
+Contains the cryptographic signature of a document.
+
+| data name | size in bytes | data type |
+| :-------: | ------------: | ------------: |
+| algorithm | 4 | KeysAlgorithm |
+| content | ? | ?* |
+
+_*The type of field `content` depends on the algorithm. In the case of Ed25519, `content` is a 64-byte array containing the signature._
+
+## Endianness
+
+All numbers (integers and floats) are encoded in little endian.
+
+## Endpoint v2
+
+| data name | size in bytes | data type |
+| :--------------: | ------------: | ------------: |
+| api_name | ? | String |
+| api_version | 2 | u16 |
+| network_features | 1 | u8 |
+| api_features | 1 | u8 |
+| endpoint_version | 4 | u32 |
+| host | ? | Opt(String) |
+| ip_v4 | 4 | Opt([u8; 4]) |
+| ip_v6 | 16 | Opt([u16; 8]) |
+| port | 2 | u16 |
+| path | ? | Opt(String) |
+
+## Peer card
+
+
+Signed document declaring all the endpoints of a peer.
+
+| data name | size in bytes | data type |
+| :---------------: | ------------- | --------------------------- |
+| version | 4 | u32 |
+| currency_code | 2 | u16 |
+| issuer_public_key | ? | PubkeyBox |
+| node_id | 4 | u32 |
+| blockstamp | 36 | Blockstamp |
+| endpoints_count | 8 | u64 |
+| endpoints_datas | ? | [Endpoint; endpoints_count] |
+| signature | ? | SigBox |
+
+## Messages
+
+All messages are encapsulated in the following format :
+
+| data name | size in bytes | data type |
+| :------------: | ------------: | --------: |
+| ws2p_version | 4 | u32 |
+| currency_name | ? | String |
+| issuer_node_id | 4 | u32 |
+| issuer_pubkey | ? | PubkeyBox |
+| message_type | 4 | u32 |
+| payload | ? | ?* |
+| signature | ? | SigBox |
+
+### message_type
+
+| `message_type` | code | Hashed | Signed |
+|:--------------------:|--------|--------|--------|
+| CONNECT | 0 | no | yes |
+| ACK | 1 | no | yes |
+| SECRET_FLAGS | 2 | no | yes |
+| OK | 3 | no | yes |
+| KO | 4 | no | yes |
+| REQUEST | 5 | yes | no |
+| REQUEST_RESPONSE | 6 | yes | no |
+| PEERS | 7 | yes | no |
+| HEADS_V2 | 8 | yes | no |
+| HEADS_V3 | 9 | yes | no |
+| BLOCKS | 10 | yes | no |
+| PENDING_IDENTITIES | 11 | yes | no |
+| PENDING_MEMBERSHIPS | 12 | yes | no |
+| PENDING_CERTS | 13 | yes | no |
+| PENDING_REVOCATIONS | 14 | yes | no |
+| PENDING_TXS | 15 | yes | no |
+
+### CONNECT message
+
+| data name | size in bytes | data type |
+| :-----------: | ------------- | --------------- |
+| challenge | 32 | [u8; 32] |
+| af_size | 1 | u8 |
+| api_features | af_size | WS2PFeatures |
+| flags_size | 1 | u8 |
+| flags_queries | flags_size | WS2PFlags |
+| peer_card | ? | Peer card |
+| chunkstamp | 37 | Opt(Blockstamp) |
+
+### ACK message
+
+| data name | size in bytes | data type |
+| :-------: | ------------- | --------- |
+| challenge | 32 | [u8;32] |
+
+### SECRET FLAGS message
+
+| data name | size in bytes | data type |
+| :----------: | ------------- | ---------------- |
+| flags_size | 1 | u8 |
+| flags | flags_size | WS2PSecretFlags |
+| member_proof | ? | Opt(MembreProof) |
+
+MembreProof type :
+
+| data name | size in bytes | data type |
+| :-------: | ------------- | --------- |
+| pubkey | ? | PubkeyBox |
+| sig | ? | SigBox |
+
+#### WS2PSecretFlags type definition
+
+| bit | flag |
+| :-------: | --------------- |
+| 0000_0001 | LOW_FLOW_DEMAND |
+
+### OK message
+
+| data name | size in bytes | data type |
+| :-------: | ------------- | --------- |
+| prefix/- | 2 | u16/0 |
+
+#### SYNC_INFO message
+
+| data name | size in bytes | data type |
+| :---------------: | --------------------- | ----------------------------- |
+| chunk_size | 32 | u32 |
+| target_blockstamp | 36 | Blockstamp |
+| milestones_count | 8 | u64 |
+| milestones | 32 × milestones_count | [Hash; milestones_count ] |
+| peer_cards_count | 8 | u64 |
+| peer_cards | ? × peer_cards_count | [PeerCard; peer_cards_count ] |
+
+### KO message
+
+| data name | size in bytes | data type |
+| :-------: | ------------- | --------- |
+| reason | 2 | u16 |
\ No newline at end of file
diff --git a/rfc/0006_ws2p_v2.md b/rfc/0006_ws2p_v2.md
index 90da17dd6e96b0fd00f083d0e2b661529881ca47..32ec732baa0475e7784572bd6ba9422cc03325b2 100644
--- a/rfc/0006_ws2p_v2.md
+++ b/rfc/0006_ws2p_v2.md
@@ -1,45 +1,29 @@
# Duniter WS2P API v2
-This document details the specifications of WS2P v2 which will be implemented in Duniter-ts 1.7 and Duniter-Rust 0.1.
-
-## 3 Main objectives
-
-1. Creating a synchronization process (upload phase only)
-2. Removal of shared peer cards to allow pruning of obsolete endpoints
-3. Partial binarization in order to prepare the future binarisaion of the Duniter protocol
-
-We are also taking advantage of these changes to address minor issues :
-
-* nodes/typescript/duniter#1226 Create an up/down peers report for the ws2p network layer
-* nodes/typescript/duniter#1264 Explicit SSL/TLS endpoints
-* nodes/typescript/duniter#1285 Sync : check integrity of downloaded chunks
+This document details the specifications of WS2P v2.
## Contents
-* [3 Main objectives](#3-main-objectives)
* [Contents](#contents)
* [FAQ](#faq)
* [What is WS2P ?](#what-is-ws2p)
* [Why create a synchronization process ?](#why-create-a-synchronization-process)
* [Why remove shared peer records ?](#why-remove-shared-peer-records)
* [Why not a total binarization?](#why-not-a-total-binarization)
- * [What consequences for clients softwares ?](#what-consequences-for-clients-softwares)
+ * [What consequences for clients software ?](#what-consequences-for-clients-software)
* [Conventions](#conventions)
* [Raw format](#raw-format)
- * [JSON format](#json-format)
* [Binary format](#binary-format)
- * [Primitive types](#primitive-types)
- * [Useful non-primitive types](#useful-non-primitive-types)
-
-* [Binary messages format](#binary-messages-format)
+* [Message encapsulation](#message-encapsulation)
+ * [Payload type](#payload.type)
* [Endpoints v2](#endpoints-v2)
* [Endpoint binary format](#endpoint-binary-format)
* [Endpoint utf8 format](#endpoint-utf8-format)
* [Peer card](#peer-card)
* [Peer card Raw format](#peer-card-raw-format)
- * [Peer card JSON format](#peer-card-json-format)
+ * [Peer card JSON stringified format](#peer-card-json-format)
* [Getting peer cards from other Duniter nodes](#getting-peer-cards-from-other-duniter-nodes)
-* [Priority of WS2P outcoming connections](#priority-of-ws2p-outcoming-connections)
+* [Priority of WS2P out-coming connections](#priority-of-ws2p-outcoming-connections)
* [Establishing a WS2P connection](#establishing-a-ws2p-connection)
* [List of accepted v1 messages](#list-of-accepted-v1-messages)
* [CONNECT Message](#connect-message)
@@ -49,7 +33,7 @@ We are also taking advantage of these changes to address minor issues :
* [Rules for accepting an incoming connection](#rules-for-accepting-an-incoming-connection)
* [WS2Pv2 Messages](#ws2pv2-messages)
* [List of accepted v1 messages](#list-of-accepted-v1-messages)
- * [Requets](#requests)
+ * [Requests](#requests)
* [Requests responses](#requests-responses)
* [HEADs v2](#heads-v2)
* [HEADs v3](#heads-v3)
@@ -63,7 +47,7 @@ WS2P means "**W**eb **S**ocket **To** **P**eer".
WS2P is the inter-node network layer, it is the only guarantor of synchronization between the different nodes of the network, so its role is critical.
-WS2P is the network part of the Duniter protocol, so any Duniter implementation must integrate the WS2P network layer in order to work.
+WS2P is the network part of the Duniter protocol
WS2P is exclusively based on websocket technology.
@@ -87,22 +71,9 @@ In addition, WS2P v1 has provided a new way to uniquely identify nodes (the node
Finally, shared peer cards have a big drawback: a peer cannot prune obsolete endpoints of other peer sharing the same key because it cannot easily evaluate if these other endpoints are obsolete. In absolute terms, it is possible to bypass problems but it is simpler to directly delete shared peer records, and as we no longer need them: Occam's razor.
-### Why not a total binarization
-
-I want to limit this RFC to changes that I am able to implement in duniter-ts, and those within a reasonable time, because I want ws2p v2 to be integrated into Duniter-ts 1.7.
-
-That's why I limit the Duniter-ts changes as much as possible: documents and requests will always be sent and received in the same format. Only peer files and heads change.
-
-On the other hand, on the Duniter-Rust side, the binarization will be total right away, because duniter-rust already stores all documents in binary format.
-To allow compatibility between different implementations, I introduced in WS2P v2 a powerful concept of features (this concept is detailed later in the RFC).
-To summarize, each WS2P Public node will declare in its endpoint the list of features it supports, and the nodes that contact them can adapt their communication format accordingly.
-
-### What consequences for clients softwares
+### What is the binary format used for serialization
-Developers of client software need to study the new format of peer and HEAD records.
-They must be able to parse this new format such that BMA 1.7 will provide them.
-
-Finally, users associating external endpoints to their peer cards will have to redeclare their external endpoints in the new generic enpoint format.
+To allow easy interfacing between different implementations, WS2P v2 defines network features (detailed later in the RFC) allowing each WS2P Public to declare in its endpoint the list of features it supports. The default format is [CBOR](https://en.wikipedia.org/wiki/CBOR), a widespread binarization format based on JSON. But for byte-efficiency reasons, Dunitrust defined an other serialization format based on the Rust crate *bincode*.
## Conventions
@@ -118,146 +89,33 @@ The PEG grammars of raw formats are presented according to the [pest syntax] (a
[PEG grammar]: https://en.wikipedia.org/wiki/Parsing_expression_grammar
[pest syntax]: https://pest.rs/book/grammars/syntax.html
-### JSON format
-
-All data in JSON format is presented in this RFC with explicit typing.
-
-Types list :
-
- Number(field description),
- String(field description),
- Bool(field description),
- Object(field description)
-
-Example :
-
- {
- "pubkey": String(identity pubkey in base58),
- "expires_on": Number(timestamp),
- "wasMember": Bool(true if the identity has already been a member at least once),
- "pendingMembership" : Object(pending membership document)
- ]
-
-### Binary format
-
-#### Primitive types
-
-u8 : Unsigned 8-bit integer.
-u16 : Unsigned 16-bit integer.
-u32 : Unsigned 32-bit integer.
-u64 : Unsigned 64-bit integer.
-i8 : Signed 8-bit integer.
-i16 : Signed 16-bit integer.
-i32 : Signed 32-bit integer.
-i64 : Signed 64-bit integer.
-bool : boolean stored on 8 bits (0x00 = false, 0x01 = true, any other value must generate an error).
-0 : Corresponds to data that must be filled with bits to zero (for example padding).
-
-#### Useful non-primitive types
-
-##### (T, U)
-
-(T, U) = Tuples of 2 elements. the 1st type T and the second type U.
-A tuple is simply a concatenation of types written one after the other.
-A tuple can be of any size (no maximum limit), but its size is necessarily fixed.
-The different elements may be of different types, but the order cannot change.
-
-##### [T; n]
-
-Array of n elements of type T.
-If the size of the array is variable, its size (in number of cells) is stored on 8 bytes.
-
-| data | size in bytes | data type |
-|:-----------:|--------------:|----------------:|
-| cells_count | 8 | u64 |
-| content* | ?*cells_count | (T1, T2, ...) |
-
-All cells in the table are written one after the other without a separator, _like_ a tuple of `cells_count` elements of the same type.
-
-**/*\ CAUTION: If the size of the array is static (=a literal constant) then the array is serialized without a size field. **
-For example, the content of a public key ed25519 is a 32-byte array, (a [u8; 32] therefore), it is serialized directly without a size field.
-
-##### Opt(T)
-
-Indicates that a field is optional. The optional fields are preceded by a 1-byte boolean that indicates whether or not the field is present.
-
-| data | size in bytes | data type |
-|:---------:|--------------:|----------------:|
-| is_some | 1 | u8 |
-| content | ? | T |
-
-If `is_some` is equal to `1`, field `content` is present.
-If `is_some` is equal to `0`, field `content` is missing.
-Any other value of `is_some` is prohibited and will invalidate the entire message.
-
-##### String
-String formatted in utf8 and [NFKC normalized](https://fr.wikipedia.org/wiki/Normalisation_Unicode#NFKC).
-**/!\ WARNING: All strings are systematically prefixed with their size, including for an empty string. The size is stored on 8 bytes as shown in the table below :**
+## Message encapsulation
-| data | size in bytes | data type |
-|:---------:|--------------:|----------------:|
-| str_size | 8 | u64 |
-| content | str_size | [u8; str_size] |
+All WS2P v2 message are encapsuled under [PKSTL v1 protocol](./0011_pkstl_v1.md).
-content := String in binary utf8 with [NFKC normalization](https://fr.wikipedia.org/wiki/Normalisation_Unicode#NFKC).
+Each WS2Pv2 message is binarized and then placed in the CUSTOM_DATAS field of the PKSTL message that encapsulates it.
-If the string is empty, field `content` is missing and `str_size` is equal to zero.
+Several binary serialization formats are available: CBOR and Bincode. The message defined below are written in JSON, allowing straightforward serialization to CBOR. For the Bincode serialization, see [Bincode format reference](./0006_annex/bincode.md).
-##### Blockstamp
+All WS2Pv2 messages are structured as follows (CBOR format) :
-| data | size in bytes | data type |
-|:----------:|--------------:|----------:|
-| block_id | 4 | u32 |
-| block_hash | 32 | [u8; 32] |
-
-##### KeysAlgorithm
-
-| algorithm | u32 |
-|:----------:|----:|
-| Ed25519 | 0 |
-| Schnorr | 1 |
-
-##### PubkeyBox
-
-Contains the signatory's public key.
-
-| data | size in bytes | data type |
-|:---------:|--------------:|-------------------:|
-| algorithm | 4 | KeysAlgorithm |
-| content | ? | ?* |
-
-_*The type of field `content` depends on the algorithm. In the case of Ed25519, `content` is a 32-byte array containing the public key._
-
-##### SigBox
-
-Contains the cryptographic signature of a document.
-
-| data name | size in bytes | data type |
-|:---------:|--------------:|----------------:|
-| algorithm | 4 | KeysAlgorithm |
-| content | ? | ?* |
-
-_*The type of field `content` depends on the algorithm. In the case of Ed25519, `content` is a 64-byte array containing the signature._
-
-#### Endianness
-
-All numbers (integers and floats) are encoded in little endian.
-
-## Binary messages format
-
-All binary WS2P v2 messages are encapsulated in the following format :
-
-| data name | size in bytes | data type |
-| :------------: | ------------: | ----------: |
-| ws2p_version | 4 | u32 |
-| currency_name | ? | String |
-| issuer_node_id | 4 | u32 |
-| issuer_pubkey | ? | PubkeyBox |
-| message_type | 4 | u32 |
-| payload | ? | ?* |
-| signature | ? | SigBox |
+ ```json
+ {
+ "ws2p_version": 2,
+ "currency_name": "g1",
+ "issuer_node_id": 395,
+ "issuer_pubkey": {
+ "algo": "Ed25519",
+ "content": [52, 147, .., 93], // Array of 32 bytes
+ },
+ "payload": {
+ "type": "MESSAGE_TYPE_NAME",
+ "content": {} // depends on payload.type
+ },
+ }
+ ```
`ws2p_version` := This field is placed first so that future versions of WS2P are not constrained on the other fields,
the only constraint will be to start the message with the version number stored in u32.
@@ -266,56 +124,64 @@ the only constraint will be to start the message with the version number stored
_* The type of `payload` is determined by the content of `message_type`._
-The signature is not always present, it's only present for connection negotiation messages.
-The signature is generated from the byte vector of the entire message (signature field excluded).
-
-### message_type interpretation
-
-| message type | code | Hashed | Signed |
-|:--------------------:|--------|--------|--------|
-| CONNECT | 0 | no | yes |
-| ACK | 1 | no | yes |
-| SECRET_FLAGS | 2 | no | yes |
-| OK | 3 | no | yes |
-| KO | 4 | no | yes |
-| REQUEST | 5 | yes | no |
-| REQUEST_RESPONSE | 6 | yes | no |
-| PEERS | 7 | yes | no |
-| HEADS_V2 | 8 | yes | no |
-| HEADS_V3 | 9 | yes | no |
-| BLOCKS | 10 | yes | no |
-| PENDING_IDENTITIES | 11 | yes | no |
-| PENDING_MEMBERSHIPS | 12 | yes | no |
-| PENDING_CERTS | 13 | yes | no |
-| PENDING_REVOCATIONS | 14 | yes | no |
-| PENDING_TXS | 15 | yes | no |
+### payload.type
+
+Exhaustive list of possible `payload.type` values:
+
+| message type | `payload.type` value | Hashed | Signed |
+|:--------------------:|----------------------|--------|--------|
+| CONNECT | "Connect" | no | yes |
+| ACK | "Ack" | no | yes |
+| SECRET_FLAGS | "SecretFlags" | no | yes |
+| OK | "Ok" | no | yes |
+| KO | "Ko" | no | yes |
+| REQUEST | "Request" | yes | no |
+| REQUEST_RESPONSE | "ReqRes" | yes | no |
+| PEERS | "Peers" | yes | no |
+| HEADS_V2 | "HeadsV2" | yes | no |
+| HEADS_V3 | "HeadsV3" | yes | no |
+| BLOCKS | "Blocks" | yes | no |
+| PENDING_IDENTITIES | "PendingIdentities" | yes | no |
+| PENDING_MEMBERSHIPS | "PendingMemberships" | yes | no |
+| PENDING_CERTS | "PendingCerts" | yes | no |
+| PENDING_REVOCATIONS | "PendingRevocations" | yes | no |
+| PENDING_TXS | "PendingTxs" | yes | no |
## Endpoints v2
-WS2Pv1 used endpoints version v1. So, in WS2Pv2 we improve to endpoints version v2.
+ ```json
+ {
+ "api": "WS2P",
+ "api_version": 2,
+ "network_features": [], // array of bytes, defined below
+ "api_features": [], // array of bytes, defined below
+ "host": "subdomain.domaine.tld",
+ "ip_v4": "5.135.188.170",
+ "ip_v6": "2001:41d0:8:c5aa::1",
+ "port": 443,
+ "path": "ws2p",
+ }
+ ```
+
+### utf8 format
+
+The utf8 format is used to display the endpoint in a human-readable format. It is also in this format that the user can manually enter an endpoint.
+
+General utf8 endpoint format :
+
+ API_NAME VERSION NF1 .. NFn AF PORT HOST IP4 [IP6] PATH
-### Endpoint binary format
+VERSION := version number prefixed by `V` (example : `V1`)
+NF := NETWORK_FEATURE
+AF := API_FEATURES in hexadecimal prefixed by `0x` (example : `0x01`)
-| data name | size in bytes | data type |
-| :--------------: | ------------: | ------------: |
-| api_name | ? | String |
-| api_version | 2 | u16 |
-| nf_size | 8 | u64 |
-| network_features | nf_size | specific |
-| af_size | 8 | u64 |
-| api_features | af_size | specific |
-| endpoint_version | 4 | u32 |
-| host | ? | Opt(String) |
-| ip_v4 | 4 | Opt([u8; 4]) |
-| ip_v6 | 16 | Opt([u16; 8]) |
-| port | 2 | u16 |
-| path | ? | Opt(String) |
+Example:
-endpoint_version := value is `1`.
+ WS2P V2 TLS 0x7 443 g1.durs.info ws2p
-#### network_features
+### network_features
-The 16 bits represent booleans to define the presence or absence of 16 network features. WS2Pv2 defines only 2 features, the remaining 14 are undefined and are in anticipation of future Ğfeatures.
+The 8 bits represent booleans to define the presence or absence of 8 network features. WS2Pv2 defines only 4 features, the remaining 4 are undefined and are in anticipation of future Ğfeatures.
Network features :
@@ -323,14 +189,14 @@ Network features :
|:---------:|----------|
| 0000_0001 | HTTP |
| 0000_0010 | WS |
-| 0000_0100 | TLS |
+| 0000_0100 | S |
| 0000_1000 | TOR |
HTTP := This feature indicates that the endpoint should be contacted with http protocol.
WS := This feature indicates that the endpoint should be contacted with websocket protocol.
-TLS := This feature indicates that the endpoint should be contacted with an SSL/TLS overlay (HTTPS or WSS or any other protocol that support TLS).
+S := This feature indicates that the endpoint should be contacted with an SSL/TLS overlay (HTTPS or WSS or any other protocol that support TLS).
TOR := This feature indicates that the endpoint must be contacted via the tor network (hidden service).
@@ -338,89 +204,74 @@ Note about the network protocol to choose: Each API must define a default networ
For example, the default network protocol for WS2P is `ws://`, it's the only one possible for this API, so there is no need to indicate it in the network features.
-#### api_features
+### api_features
The interpretation of this field depends on the API because it represents API-specific features. Here is the interpretation for the WS2P API :
WS2PFeatures type definition :
-| bit | feature |
-|:---------:|-----------|
-| 0000_0001 | DEF |
-| 0000_0010 | LOW |
-| 0000_0100 | ABF |
+| bit | feature |
+| :-------: | ------- |
+| 0000_0100 | RBC |
+| 0000_0010 | LOW |
-DEF := Supports permessage-deflate extension
+RBC := Support Rust BinCode format
LOW := Accept low speed connection requests
-ABF := Support all binary formats.
-
-WS2P v2 uses only 3 of 8 features. The 5 free bits can be used for future versions of WS2P.
-
-### Endpoint utf8 format
-
-The utf8 format is used to display the endpoint in a human-readable format. It is also in this format that the user can manually enter an endpoint.
-
-General utf8 endpoint format :
-
- API_NAME VERSION NF1 .. NFn AF PORT HOST IP4 [IP6] PATH
-
-VERSION := version number prefixed by `V` (example : `V1`)
-NF := NETWORK_FEATURE
-AF := API_FEATURES in hexadecimal prefixed by `0x` (example : `0x01`)
-
-Example:
-
- WS2P V2 TLS 0x7 443 g1.durs.info ws2p
-
-Same endpoint in binary format :
-
-| data name | size in bytes | data type | value |
-| :--------------: | ------------: | ------------: | --------------------------------------------- |
-| api_name | ? | String | 0x0000_0004 ++ "ws2p" encoded in utf8 |
-| api_version | 2 | u16 | 0x0002 |
-| nf_size | 8 | u64 | 0x0000_0001 |
-| network_features | nf_size | specific | 0x04 |
-| af_size | 8 | u64 | 0x0000_0001 |
-| api_features | af_size | specific | 0x07 |
-| endpoint_version | 4 | u32 | 0x0002 |
-| host | ? | Opt(String) | 0x0000_0004 ++ "g1.durs.info" encoded in utf8 |
-| ip_v4 | 4 | Opt([u8; 4]) | 0x00 |
-| ip_v6 | 16 | Opt([u16; 8]) | 0x00 |
-| port | 2 | u16 | 0x01BB |
-| path | ? | Opt(String) | 0x01 ++ 0x0000_0004 ++ "ws2p" encoded in utf8 |
+WS2P v2 uses only 2 of 8 features. The 6 free bits can be used for future versions of WS2P.
## Peer card
-
Signed document declaring all the endpoints of a peer.
-### Peer card binary format
-
-| data name | size in bytes | data type |
-|:-----------------:|---------------|------------------------------------|
-| version | 4 | u32 |
-| currency_code | 2 | u16 |
-| issuer_public_key | ? | PubkeyBox |
-| node_id | 4 | u32 |
-| blockstamp | 36 | Blockstamp |
-| endpoints_count | 8 | u64 |
-| endpoints_datas | ? | [Endpoint; endpoints_count] |
-| signature | ? | SigBox |
+Peer card v11 is signed in raw format (without SIGNATURE itself of course).
-`endpoint_size` type : u16.
+A node's peer record can also contain signatures of the node's network key. This allows, for example, to sign mirror nodes with known and trusted keys.
+The signed key must be signed with the blockstamp corresponding to the date of signature. More precisely, it is the text `PUBKEY:BLOCKSTAMP` that must be signed.
-`endpoint` type is defined above (See "Endpoint format").
+### CBOR format
-The document is represented by an array of bytes containing (peer_card_size + 2) bytes.
-to sign the document it is necessary to calculate the hash sha256 of this bytes array then to sign the hash obtained with the issuer's private key.
+ ```json
+ {
+ "version": "v11",
+ "currency_name": "g1",
+ "issuer": {
+ "algo": "Ed25519",
+ "content": [122, 11, .., 95] // Array of 32 bytes numbers
+ },
+ "node_id": 10623,
+ "created_on": 542,
+ "endpoints": [], // Array of endpoitns v2
+ "endpoints_str": [], // Array of Strings (endpoitn v2 in utf8 format)
+ "sig": {
+ "algo": "Ed25519",
+ "content": [187, 7, .., 23] // Array of 64 bytes numbers
+ },
+ "certifiers": [
+ {
+ "certifier": {
+ "algo": "Ed25519",
+ "content": [122, 11, .., 95] // Array of 32 bytes numbers
+ },
+ "blockstamp": {
+ "id": 327,
+ "hash": [85, 132, .., 27] // Array of 32 bytes numbers
+ },
+ "sig": {
+ "algo": "Ed25519",
+ "content": [187, 7, .., 23] // Array of 64 bytes numbers
+ },
+ }
+ ]
+ }
+ ```
### Peer card Raw format
Approximate description :
- VERSION:CURRENCY:NODE_ID:PUBLIC_KEY:BLOCKSTAMP
+ VERSION:CURRENCY:NODE_ID:PUBLIC_KEY:CREATED_ON
ENDPOINT 1
...
ENDPOINT n
@@ -430,92 +281,79 @@ _/!\ This description is approximate, it allows you to quickly understand the fo
Example :
- 11:g1:0:7iMV3b6j2hSj5WtrfchfvxivS9swN3opDgxudeHq64fb:50-000005B1CEB4EC5245EF7E33101A330A1C9A358EC45A25FC13F78BB58C9E7370
+ 11:g1:0:7iMV3b6j2hSj5WtrfchfvxivS9swN3opDgxudeHq64fb:50
WS2P V2 S 7 g1.durs.ifee.fr 443 ws2p
WS2P V2 S 7 84.16.72.210 443 ws2p
EQ2D5almq2RNUi3XZNtTpjo9nWtJF0PzsCW7ROAzCQKiEtpI7/fW8Z23GJ2a/SIxfYSzlq/cZqksE4EoVe1rAw==
Peer card v11 PEG grammar :
-```
-// Single character rules
-nl = _{ "\n" }
-no_zero_hexa_lower = @{ '1'..'9' | 'a'..'f' }
-hexa_lower = @{ ASCII_DIGIT | 'a'..'f' }
-hexa_upper = @{ ASCII_DIGIT | 'A'..'F' }
-base58 = { !("O" | "I" | "l") ~ ASCII_ALPHANUMERIC }
-base64 = { ASCII_ALPHANUMERIC | "+" | "/" }
-
-// Numbers rules
-tens = @{ '1'..'9' ~ ASCII_DIGIT }
-u8_hundreds = @{ ("2" ~ ('0'..'4' ~ ASCII_DIGIT | ('0'..'5'){2})) | ("1" ~ ASCII_DIGIT{2}) }
-u8 = @{ u8_hundreds | tens | ASCII_DIGIT }
-no_zero_u_int = @{ '1'..'9' ~ ASCII_DIGIT* }
-u_int = @{ "0" | no_zero_u_int }
-
-// Usefull types rules
-currency = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "-" | "_"){,255} }
-block_id = @{ u_int }
-hash = @{ hexa_upper{64} }
-pubkey = @{ base58{43,44} }
-blockstamp = ${ block_id ~ "-" ~ hash }
-ed25519_sig = @{ base64{88} | (base64{87} ~ "=") | (base64{86} ~ "==") }
-
-// IP v6 rules
-ip6_seg = _{ hexa_lower{1,4} }
-ip6_full = _{ (ip6_seg ~ ":"){7} ~ ip6_seg }
-ip6_no_full = @{
- ip6_seg? ~
- (":" ~ ip6_seg){0,6} ~
- "::" ~
- (ip6_seg ~ ":"){0,6} ~
- ip6_seg?
-}
-ip6_inner = @{ ip6_full | ip6_no_full }
-ip6 = _{ "[" ~ ip6_inner ~ "] " }
-
-// Endpoint v2 rules
-api_version_inner = @{ no_zero_u_int }
-api_version = _{ "V" ~ api_version_inner ~ " " }
-http = @{ "HTTP " }
-ws = @{ "WS " }
-tls = @{ "S " }
-tor = @{ "TOR " }
-network_features = _{ http? ~ ws? ~ tls? ~ tor? }
-api_features_inner = @{ (hexa_lower{2})+ | no_zero_hexa_lower }
-api_features = _{ api_features_inner ~ " " }
-host_v2_inner = @{ ASCII_ALPHA_LOWER ~ (ASCII_ALPHA_LOWER | ASCII_DIGIT | "-" | "_" | ".")* }
-host_v2 = _{ host_v2_inner ~ " " }
-ip4_inner = { u8 ~ "." ~ u8 ~ "." ~ u8 ~ "." ~ u8 }
-ip4 = _{ ip4_inner ~ " " }
-path = _{ " " ~ path_inner }
-endpoint_v2 = ${ api_name ~ " " ~ (api_version)? ~ (network_features)? ~ (api_features)? ~ host_v2? ~ ip4? ~ ip6? ~ port ~ path? }
-
-// Peer v11 rules
-peer_v11 = ${ "11:" ~ currency ~ ":" ~ node_id ~ ":" ~ pubkey ~ ":" ~ blockstamp ~ nl ~ (endpoint_v2 ~ nl)+ ~ ed25519_sig? }
-```
-
-### Peer card JSON format
-
-This human-readable format will be used by all APIs that wish to provide peer cards in a human-readable format. For example, Client APIs.
+ ```pest
+ // Single character rules
+ nl = _{ "\n" }
+ no_zero_hexa_lower = @{ '1'..'9' | 'a'..'f' }
+ hexa_lower = @{ ASCII_DIGIT | 'a'..'f' }
+ hexa_upper = @{ ASCII_DIGIT | 'A'..'F' }
+ base58 = { !("O" | "I" | "l") ~ ASCII_ALPHANUMERIC }
+ base64 = { ASCII_ALPHANUMERIC | "+" | "/" }
- {
- "version": Number(VERSION),
- "currency": Number(CURRENCY_CODE),
- "node_id": String(NODE_ID in base16),
- "algorithm": String(name of pubkey/sig algorithm)
- "pubkey": String(ISSUER_PUBLIC_KEY in base58),
- "blockstamp": String(BLOCK_NUMBER ++ "-" ++ BLOCK_HASH in base16),
- "endpoinds": Array([
- String(ENDPOINT_1 in endpoint utf8 format),
- String(ENDPOINT_2 in endpoint utf8 format),
- ...
- ]),
- "signature": String(SIGNATURE in base64),
+ // Numbers rules
+ tens = @{ '1'..'9' ~ ASCII_DIGIT }
+ u8_hundreds = @{ ("2" ~ ('0'..'4' ~ ASCII_DIGIT | ('0'..'5'){2})) | ("1" ~ ASCII_DIGIT{2}) }
+ u8 = @{ u8_hundreds | tens | ASCII_DIGIT }
+ no_zero_u_int = @{ '1'..'9' ~ ASCII_DIGIT* }
+ u_int = @{ "0" | no_zero_u_int }
+
+ // Usefull types rules
+ currency = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "-" | "_"){,255} }
+ block_id = @{ u_int }
+ hash = @{ hexa_upper{64} }
+ pubkey = @{ base58{43,44} }
+ ed25519_sig = @{ base64{88} | (base64{87} ~ "=") | (base64{86} ~ "==") }
+
+ // IP v6 rules
+ ip6_seg = _{ hexa_lower{1,4} }
+ ip6_full = _{ (ip6_seg ~ ":"){7} ~ ip6_seg }
+ ip6_no_full = @{
+ ip6_seg? ~
+ (":" ~ ip6_seg){0,6} ~
+ "::" ~
+ (ip6_seg ~ ":"){0,6} ~
+ ip6_seg?
}
+ ip6_inner = @{ ip6_full | ip6_no_full }
+ ip6 = _{ "[" ~ ip6_inner ~ "] " }
+
+ // Endpoint v2 rules
+ api_version_inner = @{ no_zero_u_int }
+ api_version = _{ "V" ~ api_version_inner ~ " " }
+ http = @{ "HTTP " }
+ ws = @{ "WS " }
+ tls = @{ "S " }
+ tor = @{ "TOR " }
+ network_features = _{ http? ~ ws? ~ tls? ~ tor? }
+ api_features_inner = @{ (hexa_lower{2})+ | no_zero_hexa_lower }
+ api_features = _{ "0x" ~ api_features_inner ~ " " }
+ domain_name_part = @{ ASCII_ALPHA_LOWER ~ (alphanum_lower | "-" | "_")* }
+ domain_name_ext = @{ alphanum_lower+ }
+ domain_name_parts = @{ (domain_name_part ~ ".")+ ~ domain_name_ext }
+ domain_name_onion = @{ alphanum_lower{16} ~ ".onion" }
+ domain_name_inner = @{ domain_name_parts | domain_name_onion | domain_name_part }
+ domain_name = _{ domain_name_inner ~ " " }
+ ip4_inner = { u8 ~ "." ~ u8 ~ "." ~ u8 ~ "." ~ u8 }
+ ip4 = _{ ip4_inner ~ " " }
+ path = _{ " " ~ path_inner }
+ endpoint_v2 = ${ api_name ~ " " ~ (api_version)? ~ (network_features)? ~ (api_features)? ~ ip4? ~ ip6? ~ domain_name? ~ port ~ path? }
+
+ // Peer v11 rules
+ peer_v11 = ${ "11:" ~ currency ~ ":" ~ node_id ~ ":" ~ pubkey ~ ":" ~ block_id ~ nl ~ (endpoint_v2 ~ nl)+ ~ ed25519_sig? }
+ ```
-Example :
+### Peer card JSON stringified format
+
+This human-readable stringified format will be used by all APIs that wish to provide peer cards in a human-readable format. For example, Client APIs.
+ ```json
{
"version": 11,
"currency": 1,
@@ -524,18 +362,22 @@ Example :
"pubkey": "D9D2zaJoWYWveii1JRYLVK3J4Z7ZH3QczoKrnQeiM6mx"
"blockstamp": "128310-000002A569DCEED62227CAC0ABDFE6B2647B21B3193A40398CD12BC2A95C24D9",
"endpoinds": [
- "WS2P 2 1 TLS 3 DEF LOW ABF 443 g1.durs.info ws2p"
+ "WS2P 2 1 TLS 3 DEF LOW ABF 443 g1.domaine.tld ws2p"
],
"signature": "e2BtWtLs3Uqw80bvOx8YGfmDLfA44/apzpZA8YfH+WtnmPY5r4XUlgvctPsq2bHVw3iPWxuxx5oJh0JHITrECw==",
+ "certifiers": [
+ "Ez4huJahi6qfG8eVjyc84CwVtpHagmfUTtRFazXY4G2h:81234-0000047F1D3BB30C24CBD9D60977FC9474D3A985DF88BB9C9CAFE9160D94AC17:oP5iHvXRx1mI+Nic621/tU42kk8Mr+m5wgNZiBPJf4vGSlGSPkTSbzMJgHehnCcEF/UB+xPoLZ2WK1AMODlFDg=="
+ ]
}
+ ```
-### Getting peer cards from other Duniter nodes
+### Getting peer cards from other nodes
-Initially, Duniter is totally endpoint agnostic and don't know any node.
+Initially, nodes are totally endpoint agnostic and don't know any node.
The ws2p module requires that at least one or more peer cards are already stored in the database when launching the node,
-it's up to the Duniter `sync` command to handle this (see "WS2P synchronisation" part).
+it's up to the `sync` command to handle this (see "WS2P synchronisation" part).
-When duniter starts up, the WS2P module accesses the peer cards stored in a local database following synchronization.
+When node starts up, the WS2P module accesses the peer cards stored in a local database following synchronization.
## Priority of WS2P outcoming connections
@@ -593,69 +435,35 @@ WARNING: independently of all these rules, each implementation must integrate it
### CONNECT message
-| data name | size in bytes | data type |
-|:-------------:|---------------|--------------|
-| challenge | 32 | [u8; 32] |
-| af_size | 1 | u8 |
-| api_features | af_size | WS2PFeatures |
-| flags_size | 1 | u8 |
-| flags_queries | flags_size | WS2PFlags |
-| peer_card | ? | Peer card |
-| chunkstamp | 37 | Opt(Blockstamp) |
+ ```json
+ {} // TODO
+ ```
challenge := random hash generated by the sending node of the CONNECT message, the receiving node will then have to sign this challenge and then send this signature in its ACK message to prove that it has the corresponding private key to the public key it indicates.
api_features := This is exactly the same type as the field of the same name in the endpoints. But private WS2P nodes do not declare endpoints, so they must be able to indicate in the CONNECT message which features they support. Public WS2P nodes also fill this field, so any changes in the configuration of a public node will be applied on the 1st new connection. (If this was not the case, we would have to wait for the update of the peer record).
-flags_queries := See "WS2PFlags type definition" below.
+connect_type := See ConnectType below
-peer_card := This field is optional, if it's not present it must be replaced by 2 bytes filled with zeros. Why two bytes ? Because the first field of any peer card, `peer_card_size`, is 2 bytes long.
+peer_card := See PeerCard above
-chunkstamp := Blockstamp of the last block of the chunk. This field is present only if flag `ASK_SYNC_CHUNK` is present in `flags_queries`.
+#### ConnectType type definition
-#### WS2PFlags type definition
-
-| bit | flag |
-|:---------:|-----------------|
-| 0000_0001 | SYNC |
-| 0000_0010 | ASK_SYNC_CHUNK |
-| 0000_0100 | RES_SYNC_CHUNK |
-| 0000_1000 | CLIENT |
-
-SYNC := Boolean indicating whether the connection corresponds to a synchronization request or not. In case of a synchronization request, the connection will necessarily be accepted if it's possible* but temporary. Whereas in the normal case the connection may or may not be accepted according to classic ws2p rules, but it will be permanent.
-
-_*The anti-spam procedures of the implementation may cause the node to still refuse the connection._
-
-ASK_SYNC_CHUNK := So that the synchronization is not too slow, the nodes to which chunk are requested are required to accept the connection and send at least 1 chunk before closing it. They can of course accept the connection for longer and send several chunks.
-
-RES_SYNC_CHUNK := A WS2P Public node that asks to synchronize sends its PeerCard to the network, which can then contact it spontaneously to send chunk, the node in synchronization will accept in priority this type of connections.
-
-CLIENT := A third-party program wants to connect in client mode.
+TODO
### ACK message
-| data name | size in bytes | data type |
-|:-------------:|---------------|-----------|
-| challenge | 32 | [u8;32] |
+ ```json
+ {} // TODO
+ ```
Each node must sign the challenge of the other to prove that it's in possession of the private key corresponding to the public key under which it identifies.
-challenge := The challenge given by the other node in their CONNECt message.
+challenge := The challenge given by the other node in their CONNECT message.
The message is already signed at the container level, so there is no need to repeat the signature in the ACK message payload. On the other hand, the challenge to be signed must be in the payload.
### SECRET FLAGS message
-| data name | size in bytes | data type |
-|:-------------:|---------------|------------------|
-| flags_size | 1 | u8 |
-| flags | flags_size | WS2PSecretFlags |
-| member_proof | ? | Opt(MembreProof) |
-
-MembreProof type :
-
-| data name | size in bytes | data type |
-|:-------------:|---------------|-----------------|
-| pubkey | ? | PubkeyBox |
-| sig | ? | SigBox |
+TODO
If its valued, the `member_proof` field must contain a signature of the challenge* send by other node in their CONNECT message.
@@ -674,23 +482,18 @@ LOW_FLOW_DEMAND := The sender node of this message indicates that it's behind a
In most cases, the OK message is empty (=3 zero-bytes), it simply indicates that the remote node accepts the connection establishment. If the local node has also sent an OK message, then it considers the connection as fully established.
But sometimes the OK message can also transmit additional informations :
-| data name | size in bytes | data type |
-|:--------------------:|---------------|--------------------------------|
-| prefix/- | 2 | u16/0 |
+ ```json
+ {} // TODO
+ ```
prefix := In ws2p v2, member nodes have the option of not publicly communicating their prefix. It will only reveal their prefix to the node of their choice among those with which they establish a connection. For example, a member node may decide to share its prefix only with other member nodes of the same key.
If this field is zero, it means that the remote node does not want to reveal its prefix (the prefix being necessarily greater than or equal to 1).
#### SYNC_INFO message
-| data name | size in bytes | data type |
-|:--------------------:|------------------------|--------------------------------|
-| chunk_size | 32 | u32 |
-| target_blockstamp | 36 | Blockstamp |
-| milestones_count | 8 | u64 |
-| milestones | 32 × milestones_count | [Hash; milestones_count ] |
-| peer_cards_count | 8 | u64 |
-| peer_cards | ? × peer_cards_count | [PeerCard; peer_cards_count ] |
+ ```json
+ {} // TODO
+ ```
chunk_size := defines the size of the block used in milestones. A 1000-block chunk is a good compromise between efficiency of compression and duration of transfer
@@ -702,11 +505,11 @@ peer_cards := see [definition of binary peer card](#peer-card-binary-format)
### KO message
-| data name | size in bytes | data type |
-|:---------:|---------------|-----------|
-| reason | 2 | u16 |
+ ```json
+ {} // TODO
+ ```
-#### reason interpretation
+#### reason
| value | significance |
|:-----:|------------------|
@@ -972,19 +775,23 @@ HEAD v3 PEG grammar :
This human-readable format will be used by all APIs that wish to provide heads in a human-readable format. For example, Client APIs.
+ ```json
{
- "content": String(VERSION:CURRENCY:OUTGOING_CONF:INCOMING_CONF:FREE_MIRROR_ROOMS:FREE_MEMBER_ROOMS:NODE_ID:PUBLIC_KEY:BLOCKSTAMP:SOFTWARE:SOFT_VERSION),
- "signature": String(SIGNATURE in base64),
- "step": Number
+ "content": "VERSION:CURRENCY:OUTGOING_CONF:INCOMING_CONF:FREE_MIRROR_ROOMS:FREE_MEMBER_ROOMS:NODE_ID:PUBLIC_KEY:BLOCKSTAMP:SOFTWARE:SOFT_VERSION",
+ "signature": "SIGNATURE", // Signature in base64
+ "step": 3
}
+ ```
Example :
+ ```json
{
"content": "3:g1:0:0:0:0:0:7iMV3b6j2hSj5WtrfchfvxivS9swN3opDgxudeHq64fb:50-000005B1CEB4EC5245EF7E33101A330A1C9A358EC45A25FC13F78BB58C9E7370:durs:0.2.0-a",
"signature": "vwlxpkCbv83qYSiClYA/GD35hs0AsZBnqv7uoE8hqlarT2c6jVRKhjp8JBqmRI7Se4IDwC2owk0mF4CglvyACQ==",
"step": 2,
}
+ ```
### Documents messages