Use YAML format for document parsing
Duniter documents' format is so much close to YAML that it would really be better to use diretly this format, allowing us to use YAML tools that exists in our prefered language.
Note: this does not require any change in the protocol, we can keep our current format (a.k.a. RAW) for signing documents. However, we can work with a YAML transformation for parsing documents coming from the network in RAW format:
RAW --> YAML --> JSON --> RAW
-
RAW --> YAML
would be done by us, according the the very simple algorithms given below -
YAML --> JSON
would be done byjs-yaml
library (or equivalent on other platforms) -
JSON --> RAW
would be done by us, because we want very specifc layout for signing documents
That's cool because we won't need text parsers anymore, we can delegate this job to YAML parser. For example, this code would be removed from Duniter (notably the one in GenericParser.js).
I would then add this core stuff in duniter-common
module, so anything concerning documents and their format would be available to any Node.js library which would need it.
Note how the YAML format for blocks makes it very easy to read every field:
Look:
Duniter block format
Version: VERSION
Type: Block
Currency: CURRENCY
Number: BLOCK_ID
PoWMin: NUMBER_OF_ZEROS
Time: GENERATED_ON
MedianTime: MEDIAN_DATE
UniversalDividend: DIVIDEND_AMOUNT
UnitBase: UNIT_BASE
Issuer: ISSUER_KEY
IssuersFrame: ISSUERS_FRAME
IssuersFrameVar: ISSUERS_FRAME_VAR
DifferentIssuersCount: ISSUER_KEY
PreviousHash: PREVIOUS_HASH
PreviousIssuer: PREVIOUS_ISSUER_KEY
Parameters: PARAMETERS
MembersCount: WOT_MEM_COUNT
Identities:
PUBLIC_KEY:SIGNATURE:I_BLOCK_UID:USER_ID
...
Joiners:
PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
...
Actives:
PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
...
Leavers:
PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
...
Revoked:
PUBLIC_KEY:SIGNATURE
...
Excluded:
PUBLIC_KEY
...
Certifications:
PUBKEY_FROM:PUBKEY_TO:BLOCK_ID:SIGNATURE
...
Transactions:
COMPACT_TRANSACTION
...
InnerHash: BLOCK_HASH
Nonce: NONCE
BOTTOM_SIGNATURE
What do we lack precisely to have a valid YAML document?
- prefix each multiline field value with
-
- prefix
BOTTOM_SIGNATURE
withSig:
- change COMPACT_TRANSACTION format
Result:
Version: VERSION
Type: Block
Currency: CURRENCY
Number: BLOCK_ID
PoWMin: NUMBER_OF_ZEROS
Time: GENERATED_ON
MedianTime: MEDIAN_DATE
UniversalDividend: DIVIDEND_AMOUNT
UnitBase: UNIT_BASE
Issuer: ISSUER_KEY
IssuersFrame: ISSUERS_FRAME
IssuersFrameVar: ISSUERS_FRAME_VAR
DifferentIssuersCount: ISSUER_KEY
PreviousHash: PREVIOUS_HASH
PreviousIssuer: PREVIOUS_ISSUER_KEY
Parameters: PARAMETERS
MembersCount: WOT_MEM_COUNT
Identities:
- PUBLIC_KEY:SIGNATURE:I_BLOCK_UID:USER_ID
- ...
Joiners:
- PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
- ...
Actives:
- PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
- ...
Leavers:
- PUBLIC_KEY:SIGNATURE:M_BLOCK_UID:I_BLOCK_UID:USER_ID
- ...
Revoked:
- PUBLIC_KEY:SIGNATURE
- ...
Excluded:
- PUBLIC_KEY
- ...
Certifications:
- PUBKEY_FROM:PUBKEY_TO:BLOCK_ID:SIGNATURE
- ...
Transactions:
- Head: TX:VERSION:NB_ISSUERS:NB_INPUTS:NB_UNLOCKS:NB_OUTPUTS:HAS_COMMENT:LOCKTIME
Blockstamp: BLOCKSTAMP
Issuers:
- PUBLIC_KEY
- ...
Inputs:
- INPUT
- ...
Unlocks:
- ...
Outputs:
- OUTPUT
- ...
Comment: COMMENT
Sigs:
- SIGNATURE
- ...
- ...
InnerHash: BLOCK_HASH
Nonce: NONCE
Sig: BOTTOM_SIGNATURE
Duniter identity format
Almost YAML compliant, we need to add Sig:
at the end:
Version: 10
Type: Identity
Currency: CURRENCY_NAME
Issuer: PUBLIC_KEY
UniqueID: USER_ID
Timestamp: BLOCK_UID
Sig: IDENTITY_SIGNATURE
Duniter revocation format
Almost YAML compliant, we need to add Sig:
at the end:
Version: 10
Type: Certification
Currency: CURRENCY_NAME
Issuer: PUBLIC_KEY
IdtyIssuer: IDTY_ISSUER
IdtyUniqueID: USER_ID
IdtyTimestamp: BLOCK_UID
IdtySignature: IDTY_SIGNATURE
CertTimestamp: BLOCK_UID
Sig: CERTIFIER_SIGNATURE
Duniter membership format
Almost YAML compliant, we need to add Sig:
at the end:
Version: VERSION
Type: Membership
Currency: CURRENCY_NAME
Issuer: ISSUER
Block: M_BLOCK_UID
Membership: MEMBERSHIP_TYPE
UserID: USER_ID
CertTS: BLOCK_UID
Sig: MEMBERSHIP_SIGNATURE
Duniter revocation format
Almost YAML compliant, we need to add Sig:
at the end:
Version: 10
Type: Revocation
Currency: CURRENCY_NAME
Issuer: PUBLIC_KEY
IdtyUniqueID: USER_ID
IdtyTimestamp: BLOCK_UID
IdtySignature: IDTY_SIGNATURE
Sig: REVOCATION_SIGNATURE
Duniter transaction format
Almost YAML compliant, we need to add Sig:
at the end + prefix every multiline field value with a -
:
Version: VERSION
Type: Transaction
Currency: CURRENCY_NAME
Blockstamp: BLOCK_UID
Locktime: INTEGER
Issuers:
- PUBLIC_KEY
- ...
Inputs:
- INPUT
- ...
Unlocks:
- UNLOCK
- ...
Outputs:
- AMOUNT:BASE:CONDITIONS
- ...
Comment: COMMENT
Sigs:
- SIGNATURES
- ...