jsonize_wot.rst

jsonize_wot.py

Sofar, this is only useful for a WoT snapshot done once. Re-running this to update an existing WoT.json is flawed at the very least because there is no tracking of when certificates/memberships expire and probably other reasons that I have yet to figure out.

Create a local WoT.json to inspect the Web of Trust.

The Web of Trust (WoT) is the directed graph created by members certifying others. Duniter nodes allow bma access to the WoT from the perspective of each Identity: to walk forwards through sent certifications (bma /wot/certified-by) as well as backwards through received certifications (bma /wot/certifiers-of).

While bma access via a duniter node is available, doing so for repetitive traversal from many member perspectives would be akin to DOS attacking the node. This tool serves to build a local WoT.json copy, and to update it over time, so that other tools can inspect your local WoT without making bma calls to a duniter node.

The WoT.json file loads as a python dictionary where each key is the pubkey of an Identity in the WoT, and each value is a dictionary with the following keys and values:

• uid: UniqueID of this WoT Identity,
• member: True if a member, False if no-longer,
• received: list -- pubkeys of confirmed certifiers-of,
• sent: list -- pubkeys of confirmed certified-by,
• uidblk: block num of their identity
• eldest: block num of eldest certificate sent or received
• latest: block num of latest certificate sent or received

Run it for the first time.

./jsonize_wot.py

I may be requesting *many* BMA results from a duniter node.

WoT.json not found.
WoT entries: 0, expected: 2862
bmawot_update...
...added 12Mxpv2MYYYFJrk4JeAC3erM7CmdzpfG7xeuVLihohVL adridu38
...added 13XfrqY92tTCDbtu2jFAHsgNbZ9Ne2r5Ts1VGhSCrvUb leomatteudi
...added 13fn6X3XWVgshHTgS8beZMo9XiyScx6MB6yPsBB5ZBia MelanieGraney

After minutes of continously requesting bma results from (arguably DOSing) a duniter node to build the local WoT, you may see failures; it will take a short break and continue on.

...added 3wV7CJTatK35GsvBWkWiTkv98GGe3ACgA7k65eznSTEM Bouli
...added 3wXjM3ApWwuMie3WMZEiu98oVWkaYGMBLJVHqEcDTMEn ElodieDoppagne
...added 3wdDzBz18mWupx1UChMnhky2Nut3XVnyn9U7Y662J7yE smyds
bma_request(http://localhost:20900/wot/certifiers-of/3weTTYy7H89MHNaxF9Fn7M295RavxNNJtqpfWC9L1fb7): 503
bma_request(http://localhost:20900/wot/certifiers-of/3weTTYy7H89MHNaxF9Fn7M295RavxNNJtqpfWC9L1fb7): 503
...added 3weTTYy7H89MHNaxF9Fn7M295RavxNNJtqpfWC9L1fb7 jaya
...added 3xCbMHV9btE6H1MXzqM65Drfxj5wqLQ4L61eEbJhrdgW Goldotek
...added 3xEKAfjdoGxzubNxuYp8W1mgzzisHYQ2dQSWSiVshHpZ VioletteBotter

You can interrupt it at any time, and it will save WoT.json with whatever work it has performed.

...added 42tqb6heiyft83xZKFAVghtC9oWC3jbGzjaSQmoNn3Tq NAGIOWOTALA
...added 43ASt2LkgazDH2mVAbVCtmbgxJ3FziQpHaa4nXEFYiHr EricPetit
...added 43LzkM8T2quLSBGatXKe3cvAzAfbkcvLfyZWKGqQ9oqC lilyber
^C
WoT.json saved.
Traceback (most recent call last):
  ...
KeyboardInterrupt

You can then restart it and it will pick-up where it left-off to finish its job.

$ ./jsonize_wot.py

I may be requesting *many* BMA results from a duniter node.

WoT.json loaded.
WoT entries: 353, expected: 3286
bmawot_update...
...added 43jKXm1ChtVKdENeYEzaWQ5TjNkJKa5UxVm3UqGMMSvx Lisa34
...added 43tUiA2iiRcu1X3jYaxFzcrUqGSsKyCmQtNHYrp2D13E FannyL
...added 44BCC1Gi4o65iJXm7cznFzeipx25Swwpwzkwyv49sxdy Vimale

After many minutes using your localhost duniter node, else hours, it will eventually complete -- having built a local WoT.json file for later inspection without requiring constant bma access.

...added E4X9jdURUkaiAzS1HMEyayXkRd9wBHqFac42fqqCkMeu fxbodin
...added 8vjFwPCwyZfWNJJk2KAeeCzCxmpVoBHjPFRhMEZRmbXB Lo2
...added GEMkZcDkHXxgktmgn3agPMiubd73r2EEhMguuSRAwbLd zuniga
...bmawot_update done.
bma Height: 383447, WoT entries: 3865, expected: 3865
WoT entries: 3865
WoT certs sent: 31430, received: 31430
WoT eldest: 181825, latest: 383449
WoT members: 2875
WoT sentinels: 1880
WoT.json saved.

As the real WoT changes with the ever-growing blockchain, your local WoT.json will become outdated. Running this tool again will update your local WoT.json copy with whatever changes have been confirmed since last time.

$ ./jsonize_wot.py

I may be requesting *many* BMA results from a duniter node.

WoT.json loaded.
wot_del_key_if_changes: 1...
...del wot[6ggWxVdRgxzSDvBBBTju7AjCvXBNzgKZjydt8cZe2Uad]
...del wot[AnZHUkqYF1Zenum7wXAqRh6xKGvBX9GT9b7kkyPDzTyY]
...wot_del_key_if_changes done.
WoT entries: 3863, expected: 3865
bmawot_update...
...added 6ggWxVdRgxzSDvBBBTju7AjCvXBNzgKZjydt8cZe2Uad EricGenin
...added AnZHUkqYF1Zenum7wXAqRh6xKGvBX9GT9b7kkyPDzTyY RoselyneBinesse
...bmawot_update done.
bma Height: 383455, WoT entries: 3865, expected: 3865
WoT entries: 3865
WoT certs sent: 31431, received: 31431
WoT eldest: 181825, latest: 383451
WoT members: 2875
WoT sentinels: 1881
WoT.json saved.

If your local WoT.json gets too out-of-date, or if it gets corrupted, remove WoT.json and rebuild it from scratch.

$ rm WoT.json
$ ./jsonize_wot.py