Skip to content

GitLab

Open
Created by Cédric Moreau@c-geekOwner

Use YAML format for document parsing

Duniter documents' format is so much close to YAML that it would really be better to use diretly this format, allowing us to use YAML tools that exists in our prefered language.

Note: this does not require any change in the protocol, we can keep our current format (a.k.a. RAW) for signing documents. However, we can work with a YAML transformation for parsing documents coming from the network in RAW format:

RAW --> YAML --> JSON --> RAW
  • RAW --> YAML would be done by us, according the the very simple algorithms given below
  • YAML --> JSON would be done by js-yaml library (or equivalent on other platforms)
  • JSON --> RAW would be done by us, because we want very specifc layout for signing documents

That's cool because we won't need text parsers anymore, we can delegate this job to YAML parser. For example, this code would be removed from Duniter (notably the one in GenericParser.js).

I would then add this core stuff in duniter-common module, so anything concerning documents and their format would be available to any Node.js library which would need it.

Note how the YAML format for blocks makes it very easy to read every field:

Look:

Duniter block format

Version: VERSION
Type: Block
Currency: CURRENCY
Number: BLOCK_ID
PoWMin: NUMBER_OF_ZEROS
Time: GENERATED_ON
MedianTime: MEDIAN_DATE
UniversalDividend: DIVIDEND_AMOUNT
UnitBase: UNIT_BASE
Issuer: ISSUER_KEY
IssuersFrame: ISSUERS_FRAME
IssuersFrameVar: ISSUERS_FRAME_VAR
DifferentIssuersCount: ISSUER_KEY
PreviousHash: PREVIOUS_HASH
PreviousIssuer: PREVIOUS_ISSUER_KEY
Parameters: PARAMETERS
MembersCount: WOT_MEM_COUNT
Identities:
PUBLIC_KEY:SIGNATURE:I_BLOCK_UID:USER_ID
...
Joiners:
PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
...
Actives:
PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
...
Leavers:
PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
...
Revoked:
PUBLIC_KEY:SIGNATURE
...
Excluded:
PUBLIC_KEY
...
Certifications:
PUBKEY_FROM:PUBKEY_TO:BLOCK_ID:SIGNATURE
...
Transactions:
COMPACT_TRANSACTION
...
InnerHash: BLOCK_HASH
Nonce: NONCE
BOTTOM_SIGNATURE

What do we lack precisely to have a valid YAML document?

  • prefix each multiline field value with -
  • prefix BOTTOM_SIGNATURE with Sig:
  • change COMPACT_TRANSACTION format

Result:

Version: VERSION
Type: Block
Currency: CURRENCY
Number: BLOCK_ID
PoWMin: NUMBER_OF_ZEROS
Time: GENERATED_ON
MedianTime: MEDIAN_DATE
UniversalDividend: DIVIDEND_AMOUNT
UnitBase: UNIT_BASE
Issuer: ISSUER_KEY
IssuersFrame: ISSUERS_FRAME
IssuersFrameVar: ISSUERS_FRAME_VAR
DifferentIssuersCount: ISSUER_KEY
PreviousHash: PREVIOUS_HASH
PreviousIssuer: PREVIOUS_ISSUER_KEY
Parameters: PARAMETERS
MembersCount: WOT_MEM_COUNT
Identities:
- PUBLIC_KEY:SIGNATURE:I_BLOCK_UID:USER_ID
- ...
Joiners:
- PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
- ...
Actives:
- PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
- ...
Leavers:
- PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
- ...
Revoked:
- PUBLIC_KEY:SIGNATURE
- ...
Excluded:
- PUBLIC_KEY
- ...
Certifications:
- PUBKEY_FROM:PUBKEY_TO:BLOCK_ID:SIGNATURE
- ...
Transactions:
- Head: TX:VERSION:NB_ISSUERS:NB_INPUTS:NB_UNLOCKS:NB_OUTPUTS:HAS_COMMENT:LOCKTIME
  Blockstamp: BLOCKSTAMP
  Issuers:
  - PUBLIC_KEY
  - ...
  Inputs:
  - INPUT
  - ...
  Unlocks:
  - ...
  Outputs:
  - OUTPUT
  - ...
  Comment: COMMENT
  Sigs:
  - SIGNATURE
  - ...
- ...
InnerHash: BLOCK_HASH
Nonce: NONCE
Sig: BOTTOM_SIGNATURE

Duniter identity format

Almost YAML compliant, we need to add Sig: at the end:

Version: 10
Type: Identity
Currency: CURRENCY_NAME
Issuer: PUBLIC_KEY
UniqueID: USER_ID
Timestamp: BLOCK_UID
Sig: IDENTITY_SIGNATURE

Duniter revocation format

Almost YAML compliant, we need to add Sig: at the end:

Version: 10
Type: Certification
Currency: CURRENCY_NAME
Issuer: PUBLIC_KEY
IdtyIssuer: IDTY_ISSUER
IdtyUniqueID: USER_ID
IdtyTimestamp: BLOCK_UID
IdtySignature: IDTY_SIGNATURE
CertTimestamp: BLOCK_UID
Sig: CERTIFIER_SIGNATURE

Duniter membership format

Almost YAML compliant, we need to add Sig: at the end:

Version: VERSION
Type: Membership
Currency: CURRENCY_NAME
Issuer: ISSUER
Block: M_BLOCK_UID
Membership: MEMBERSHIP_TYPE
UserID: USER_ID
CertTS: BLOCK_UID
Sig: MEMBERSHIP_SIGNATURE

Duniter revocation format

Almost YAML compliant, we need to add Sig: at the end:

Version: 10
Type: Revocation
Currency: CURRENCY_NAME
Issuer: PUBLIC_KEY
IdtyUniqueID: USER_ID
IdtyTimestamp: BLOCK_UID
IdtySignature: IDTY_SIGNATURE
Sig: REVOCATION_SIGNATURE

Duniter transaction format

Almost YAML compliant, we need to add Sig: at the end + prefix every multiline field value with a - :

Version: VERSION
Type: Transaction
Currency: CURRENCY_NAME
Blockstamp: BLOCK_UID
Locktime: INTEGER
Issuers:
- PUBLIC_KEY
- ...
Inputs:
- INPUT
- ...
Unlocks:
- UNLOCK
- ...
Outputs:
- AMOUNT:BASE:CONDITIONS
- ...
Comment: COMMENT
Sigs:
- SIGNATURES
- ...
Edited by Éloïs
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information