diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index edbb946d16607f17dc031e04acd121d9eb78daba..a9fb21eb7aa4089727b242dcbd456fdfe5c1e192 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -50,3 +50,7 @@ repos: args: - "--server" - "https://git.duniter.org" +- repo: https://github.com/executablebooks/mdformat + rev: 0.7.16 + hooks: + - id: mdformat diff --git a/CHANGELOG.md b/CHANGELOG.md index 1148c2879c51e32ea7b71a15589a0f566ea41f07..e933d36fdc15144fdd8ed9f62234ed54298e8704 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,15 +1,18 @@ ## [v1.1.0](https://git.duniter.org/clients/python/duniterpy/-/milestones/18) (14th February 2022) ### Project + - #179, !172: Drop Python v3.6 support - #188, !168: Introduce Python 3.10 support - #145, !171: Bump copyright to 2022 ### Code + - #187, !167: Implement `__eq__` and `__hash__` `Documents` methods - #146, !166: Use `f-string` ### Dev Env/CI + - #90, #103, !113: Pytest introduction, coverage report and badge generations - #165, !170: Stop using the Makefile in the CI and tools installed from Poetry dev dependencies in favor of pre-commit - #186, !164: Introduce `pyupgrade` pre-commit hook @@ -17,27 +20,32 @@ - !169: Bump pre-commit hooks version, Handle new Pylint rules - #182, !170: Bump Sphinx to v4 ---- +______________________________________________________________________ - Thanks @Moul, @vtexier ## [v1.0.0](https://git.duniter.org/clients/python/duniterpy/-/milestones/12) (21st August 2021) + **See below the main changes for the v1.0** No changes since the rc1. ## [v1.0.0rc1](https://git.duniter.org/clients/python/duniterpy/-/milestones/12) (25th July 2021) + **See below the main changes for the v1.0** ### Fixes + - #152, #170, !159: Fix `ESUserEndpoint` regex - Fix datapod domain name in `request_data_elasticsearch` example ### Development Environment + - #149, !161: Change workflow by changing from `master` and default `dev` branches to a single `main` branch - Add changelog link to PyPI ### Documentation + - !160: Fix autodoc generation by running it into Poetry virtualenv - !158: Allow to trigger the documentation job without having the release stage completed @@ -47,11 +55,14 @@ TL;DR: The key changes are the `Documents` refactoring, the removal of the async The inclusion of the examples into the package, and the introduction of `pre-commit` tool for development purposes. ### This release breaks backward compatibility! + From this version, only major releases might break backward compatibility. For 1.x minor versions, only `DeprecationWarning` will occur. ### Features + #### Documents refactoring + Major refactoring have been done on `Document` and its sub-classes. All documents are now internally using a `signature` attribute which is a string and no longer a list of strings, except for the `Transaction` document which can handle multi-signatures. It is no longer possible to pass the `signature` as argument at instantiation since it doesn’t make sense in the document life cycle. @@ -60,21 +71,31 @@ The document version and the currency arguments are now set optional and default The methods to check the signature·s have been moved as specific methods into the documents classes. - #95, !144: Remove `signatures` argument from all `__init__()`’s documents subclasses: + - `Document` constructor is `Document(version: int, currency: str)` + - All `Document` subclasses can be directly signed when the optional `signing_key` argument is passed + - #172 `version` argument is now optional and have a proper default value per document type + - `currency` argument is now optional and defaults to `G1_CURRENCY_CODENAME` - signing_key: SigningKey = None, - version: int = VERSION, - currency: str = G1_CURRENCY_CODENAME, + ``` + signing_key: SigningKey = None, + version: int = VERSION, + currency: str = G1_CURRENCY_CODENAME, + ``` - Introduce `Identity.from_bma_lookup_response()` to simplify examples + - Refactor `Document.signatures` `List` as `Document.signature` `str` + - `VerifyingKey.verify_document()` has been removed in favour of `Document.check_signature(pubkey)` + - Set `time` argument optional in `SimpleTransaction` - #173, !150, !156: Make Transaction easier with single and multi-sign: + - `Transaction(signing_keys: Optional[Union[SigningKey, List[SigningKey]]] = None)` allows single signature or multi-signatures when created - `Transaction.sign()` and `multi_sign()` now append signature to allow different key owner to sign additionally the Tx doc - Introduce `Transaction.check_signature()` and `check_signatures()` @@ -82,6 +103,7 @@ The methods to check the signature·s have been moved as specific methods into t - #178, !152: `Membership`: Make `membership_type` argument optional defaulting to `IN` - #175, !155: Rename `timestamp` and `blockstamp` attributes and arguments depending on their content: + - `timestamp` for time in seconds - `block_id` for `BlockID` type - `BlockUID` has been renamed to `BlockID` and `BlockID.block_uid()` to `get_block_id()` @@ -89,6 +111,7 @@ The methods to check the signature·s have been moved as specific methods into t - #163, !137: `Revocation.from_signed_raw()` now stores the block ID into a `BlockID` object #### Network + The asynchronous property has been dropped to improve DuniterPy accessibility to new comers. Improvements have been made on the endpoints handling. @@ -104,11 +127,13 @@ Improvements have been made on the endpoints handling. - !138: Add `flags` and `path` attributes into `GVAEndpoint.eq()` and `hash()` computations #### Key handling + - #150, !130: Introduce DUBP Mnemonic SigningKey feature with `SigningKey.from_dubp_mnemonic()` - #161, !142: Add DUBP Mnemonic example - #181, !154: `CRCPubkey` class moved from `documents` to `key` package #### Others + - #160, !134: Include `examples` into the Python package: - They can be run into Python shell - Read new examples usage documentation in the [README.md](README.md#Examples) @@ -116,6 +141,7 @@ Improvements have been made on the endpoints handling. - !147: Update `websocket-client` and `attrs` to major releases ### Development Environment + `pre-commit` tool has been introduce to run our development tools as pre-commit hooks to check when committing a change. `isort`, a tool that sorts the imports automatically, has been introduced. @@ -127,15 +153,17 @@ Improvements have been made on the endpoints handling. - !132: Add support `pylint` v2.8.0 #### CI/CD + - #166, !140: Fix MR based pipelines which do not trigger the tools jobs when there is a code change - #169, !146: Fix with a token for `gitlab-ci-lint` hook, which was no longer working since GitLab CI lint API requires authentication - #177: Run `pre-commit` into tools jobs ### Fixes + - #134, !141: Fix license header showed into the documentation by using `#` comments rather than docstrings - Introduce [insert-license hook](https://github.com/Lucas-C/pre-commit-hooks) ---- +______________________________________________________________________ - Thanks @vtexier, @Moul @@ -144,32 +172,38 @@ Improvements have been made on the endpoints handling. ### This release breaks backward compatibility ! ### BEFORE + - `bma.network.peers` was pointing to `/network/peering/peers` request ### AFTER + - `bma.network.peers` point to `/network/peers` request - `bma.network.peering_peers` point to `/network/peering/peers` request ### Features + - #141 Helper function to get best available nodes (for a real p2p client) - #130 Allow building Block instance from local Duniter json ### Fixes + - #143 Block : fix computed_inner_hash(), sign() and proof_of_work bug - Fixed documentation version on new release ### Development + - #147 Support pylint v1.7.2 rule R0801 - #118 Set up complete CI/CD pipeline +______________________________________________________________________ + - Thanks @vtexier, @Moul, @matograine, @HugoTrentesaux ## [v0.61.0](https://git.duniter.org/clients/python/duniterpy/-/milestones/14) (30th November 2020) -- #59 add GVA query support and example -- #137 add GVAEndpoint and GVASUBEndpoint classes -- fix bug in API.reverse_url - +- #59: add GVA query support and example +- #137: add `GVAEndpoint` and `GVASUBEndpoint` classes +- fix bug in `API.reverse_url` - Readme: Update dependencies list - Upgrade to Poetry v1.1.x - Fix regex to update the documentation version @@ -187,7 +221,7 @@ Improvements have been made on the endpoints handling. - #111, !111: Clearly define the copyright and license statements - #68, !106: Package in Debian Bullseye v11 ---- +______________________________________________________________________ - Thanks @vtexier, @Moul @@ -202,49 +236,62 @@ Improvements have been made on the endpoints handling. [As Python 3.5 security fixes have been dropped on September 13th of 2020](https://devguide.python.org/#status-of-python-branches). ### CI/CD + - #127: Change deprecated keyword in `.gitlab-ci.yml` - #124: Fix `publish_doc` job not working ### Dependencies + - !107: Fix pylint v2.6.0 new checks - Update black to v20.8b1, format code - !102: Update base58 to v2 - !102: Update Sphinx to v3 ### Examples + - !104: Rework send membership and identity documents examples ### Documentation + - !102: Add Repology’s packaging status to the Readme - !103: Rename file to `CONTRIBUTING.md` to be recognised by GitLab ---- +______________________________________________________________________ - Thanks @vtexier, @Moul ## [v0.57.0](https://git.duniter.org/clients/python/duniterpy/-/milestones/11) (2nd April 2020) + ### Code + #### Enhancements + - #123 Implement authentication from Scuttlebutt .ssb/secret file - #77 Implement authentication from credentials file - #122, !99 Implement `/wot/requirements-of-pending` command support in BMA api - #120, !98 `HeadV2`, `HeadV1`, `HeadV0` and `Head` classes now each inherit from previous class **BREAK BACKWARD COMPATIBILITY** + #### Fixes + - #119, !96 `software_version` field in WS2Pv1 messages now accept a string after patch number (`1.7.21-beta` accepted) - #114, !100 Fix `bma.blockhain.revoked` command class calling `/blockchain/with/excluded` url ### Documentation + - #104 add CONTRIBUTE.md file with release workflow ### CI/CD + - #66, !93 Migrate to [Poetry](https://python-poetry.org/) (build and development environment) ---- +______________________________________________________________________ - Thanks @vtexier, @Moul ## [v0.56.0](https://git.duniter.org/clients/python/duniterpy/-/milestones/10) (20th January 2020) + ### Code + - #58, !84: Introduce WS2P API support: **BREAK BACKWARD COMPATIBILITY**: - `bma.ws`: now websocket connections use the special `WSConnection` class and `async`, check examples. - Add two examples, a helper to retrieve the WS2P API from BMA. @@ -258,59 +305,74 @@ Improvements have been made on the endpoints handling. - Introduce `output_available()` helper to check if an output is available ### Dependencies + - !91: fix the `attrs`/`attr` dependency ### Build + - #107, !81: Enhance version definition in `setup.py` ### CI/CD + - !78: Release on PyPI only on tags ### Documentation + - #101, !80: Publish auto-documentation on GitLab Pages - #106, !83: Migrate the `README` back to markdown, reword and update it. - !77: Improve v0.55.0 and add v0.55.1 changelog - !92: Add v0.56.0 changelog ---- + +______________________________________________________________________ - Thanks @vtexier, @Moul ## v0.55.1 (19th July 2019) + - #102: wheel build does not longer include `duniterpy` sub-folders - Trigger release only on `master` not on tag as protected environment variables are not shared with pipelines started on a tag - Fix Makefile: rebuild not working ## [v0.55.0](https://git.duniter.org/clients/python/duniterpy/-/milestones/9) (18th July 2019) + ### Code + - Refactor request `api.ws2p.heads` in `api.bma.network.ws2p_heads` (BMA command to get ws2p heads) - Fix bug in PubSec v1 secret key length check while loading ### Checks + #### Pylint + - Add Pylint as a dev dependency to check code validity - Add `pylint` command to the `Makefile` - #91, !65: Apply Pylint on the code #### Black + - #54: Add Black as dev dependency to format the code (not PEP8 compliant) - Add `format` and `check-format` commands to the `MakeFile` - !63: Format the code with Black #### Others + - `Makefile`: `check` command for `mypy`, `pylint`, and `check-format` checks - #94: Apply Mypy, Pylint and Black on `tests` and `examples` folders ### CI + - Add MyPy, Black, and Pylint jobs - Rename `mypy` job to `check` which run `make check`: `mypy`, `pylint`, `check-format` - Add `check-format` job at the first stage ### Build + - Add `build` command to the Makefile - Install build dependencies separately via a `requirements_deploy.txt` file - #98, !72: Do not include `tests` folder in the wheel package for PyPi distribution ### CD + - Add `deploy` and `deploy_test` commands to the Makefile - #99, !74: Add job for PyPi test deployment - !73: Trigger PyPi release job only on tag @@ -319,31 +381,36 @@ Improvements have been made on the endpoints handling. - Also trigger the pipeline when the `Makefile` changes ### Project + - #96, !70: Add support for PEP 561 for DuniterPy type hints to be recognized by mypy when imported - Remove not used `coveralls` dependency ## v0.54.3 (29th May 2019) + - Upload again to PyPi as previous release haven’t been uploaded thanks to the tag - Transaction: fix `time` type ## v0.54.2 (27th May 2019) + - fix Transaction document generation - lock transaction document generation with a test ## v0.54.1 (9th May 2019) + - `Transaction`: add __eq__() and __hash__() methods - Transaction Unlock parameters: add __eq__() and __hash__() methods - Transaction: add 'time' variable for read and write but not for doc generation - output conditions: add __eq__() and __hash__() methods - test transaction equality at all levels ---- +______________________________________________________________________ - Thanks @Moul, @vtexier ## v0.54.0 (5th May 2019) ### Code/tests + - Fix OutputSource and InputSource from_inline() regex matching - Transaction document: tests and code: drop versions 2 and 3 management - Block document: code: drop vensions 2 and 3 management @@ -353,13 +420,14 @@ Improvements have been made on the endpoints handling. - output conditions: add token() and compose() tests ### Other + - CI: Do not trigger build, tests, type check on modification of non-relevant files - Makefile: use python3 module to run tests and type check - Add coveralls as dev dependency - setup.py: add classifiers: Python versions, Intended Audience - Add CHANGELOG.md from v0.53.1 ---- +______________________________________________________________________ - Thanks @Moul, @vtexier @@ -367,7 +435,8 @@ Improvements have been made on the endpoints handling. - Implement equality `__eq__()` and `__hash__()` methods for InputSource and OutputSource classes ---- +______________________________________________________________________ + Thanks @Moul, @vtexier ## v0.53.0 (30 March 2019) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4686e14ad2cd063afa49f5372fbb514db704970b..f3d4989ffea73b8831dd2bec8586d571271c8481 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,97 +1,120 @@ # Contribute guide ## Development workflow + Create and push a feature or bugfix release out of the `main` branch. Name this branch after the ticket number `###`. Open a merge request out of this branch to the `main` branch: + ```bash main <− ###_feature_branch ``` ## Pre-commit + We are using [`pre-commit`](https://pre-commit.com/) tool to perform checks on staged changes before committing. We are using it for `black` formatting, `isort` imports sorting, `pylint` code linting, `pyupgrade` syntax upgrader, `mypy` typing, and `gitlab-ci` linting. Install `pre-commit` from your distribution. In case it is an outdated version, install it from `pip`: + ```bash sudo apt install pre-commit pip install --user pre-commit ``` To install the `git-hooks`, run: + ```bash duniterpy> pre-commit install ``` + Then each time you commit changes, the hooks will perform checks. To manually run one of the tool above, run (eg for `isort`): + ```bash duniterpy> pre-commit run --all-files isort ``` + To run all checks on all files: + ```bash duniterpy> pre-commit run -a ``` ### Authorization for GitLab CI linter hook + `pre-commit run -a (gitlab-ci-linter)` is failing due to authorization required for CI lint API accesses. When running this command, just ignore this failed hook. In case you want to commit a `.gitlab-ci.yml` edition, this hook will prevent the commit creation. You can [skip the hooks](https://stackoverflow.com/a/7230886) with `git commit -m "msg" --no-verify`. This is fine for occasional `.gitlab-ci.yml` editions. In case you would like to edit this file more often and have it checked, ask a maintainer to provide you with `GITLAB_PRIVATE_TOKEN` environment variable that can be set into a shell configuration. With Bash, in `$HOME/.bashrc` add the following: + ```bash export GITLAB_PRIVATE_TOKEN="" ``` With Fish, in `$HOME/.config/fish/config.fish` add the following: + ```fish set -xg GITLAB_PRIVATE_TOKEN "" ``` + Check out #169 for more details. ### Black formatting + We are using [Black](https://github.com/psf/black) formatter tool. Run Black on a Python file to format it: + ```bash poetry run black duniterpy/file.py ``` + With `pre-commit`, Black is called on staged files, so the commit should fail in case black would make changes. You will have to add Black changes in order to commit your changes. ## Tests + We are using [`pytest` framework](https://docs.pytest.org/). - Run all tests with: + ```bash duniterpy> poetry run pytest ``` - Run specific tests by specifying the path to a file: + ```bash duniterpy> poetry run pytest tests/helpers/test_ws2p.py ``` - You can even specify a test from the selected file: + ```bash duniterpy> poetry run pytest tests/helpers/test_ws2p.py::test_generate_ws2p_endpoint +``` ### Update copyright year + Follow [this documentation](https://github.com/Lucas-C/pre-commit-hooks#removing-old-license-and-replacing-it-with-a-new-one) Only difference is to update the year in `license_header.txt` rather than `LICENSE.txt`. ## Release workflow + To handle a release, you have to follow this workflow: -* Verify all features and bug fixes are merged in the `main` branch. -* Checkout on the `main` branch -* Update the `CHANGELOG.md` file and commit -* Run the `release.sh` script with the version semantic number as argument: +- Verify all features and bug fixes are merged in the `main` branch. +- Checkout on the `main` branch +- Update the `CHANGELOG.md` file and commit +- Run the `release.sh` script with the version semantic number as argument: ```bash ./release.sh 0.50.0 ``` -* Create a MR containing the changelog and the commit updating the version -* Once the MR is merged to `main`, push the tag with `git push --tags` -* To release to PyPI, the pipeline based on the tag should be used by triggerring the manual job on the `main` branch +- Create a MR containing the changelog and the commit updating the version +- Once the MR is merged to `main`, push the tag with `git push --tags` +- To release to PyPI, the pipeline based on the tag should be used by triggerring the manual job on the `main` branch diff --git a/README.md b/README.md index 9f879824b9436da228588b501b22186008772cd9..d61e05acbcfa6e061d58371be128d40d98f0482e 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,19 @@ # DuniterPy + Most complete client oriented Python library for [Duniter](https://git.duniter.org/nodes/typescript/duniter)/Ğ1 ecosystem. This library was originally developed for [Sakia](https://git.duniter.org/clients/python/sakia) desktop client which is now discontinued. It is currently used by following programs: + - [Tikka](https://git.duniter.org/clients/python/tikka), the desktop client. - [Silkaj](https://silkaj.duniter.org/), command line client. - [Jaklis](https://git.p2p.legal/axiom-team/jaklis), command line client for Cs+/Gchange pods. - [Ğ1Dons](https://git.duniter.org/matograine/g1pourboire), Ğ1Dons, paper-wallet generator aimed at giving tips in Ğ1. ## Features + ### Network + - APIs support: BMA, GVA, WS2P, and CS+: - [Basic Merkle API](https://git.duniter.org/nodes/typescript/duniter/-/blob/dev/doc/HTTP_API.md), first Duniter API to be deprecated - GraphQL Verification API, Duniter API in developement meant to replace BMA. Based on GraphQL. @@ -20,6 +24,7 @@ It is currently used by following programs: - Endpoints management ### Blockchain + - Support [Duniter blockchain protocol](https://git.duniter.org/documents/rfcs#duniter-blockchain-protocol-dubp) - Duniter documents management: transaction, block and WoT documents - Multiple authentication methods @@ -27,6 +32,7 @@ It is currently used by following programs: - Sign/verify and encrypt/decrypt messages with Duniter credentials ## Requirements + - Python >= 3.8.0 - [websocket-client](https://pypi.org/project/websocket-client) - [jsonschema](https://pypi.org/project/jsonschema) @@ -37,32 +43,40 @@ It is currently used by following programs: - [pyaes](https://pypi.org/project/pyaes) ## Installation + You will require following dependencies: + ```bash sudo apt install python3-pip python3-dev python3-wheel libsodium23 ``` You can install DuniterPy and its dependencies with following command: + ```bash pip3 install duniterpy --user ``` ## Install the development environment + - [Install Poetry](https://python-poetry.org/docs/#installation) ## Documentation + [Online official automaticaly generated documentation](https://clients.duniter.io/python/duniterpy/index.html) ## Examples + The [examples folder](https://git.duniter.org/clients/python/duniterpy/tree/master/examples) contains scripts to help you! - Have a look at the `examples` folder - Run examples from parent folder directly + ```bash poetry run python examples/request_data.py ``` Or from Python interpreter: + ```bash poetry run python >>> import examples @@ -71,6 +85,7 @@ poetry run python ``` `request_data_async` example requires to be run with `asyncio`: + ```bash >>> import examples, asyncio >>> asyncio.get_event_loop().run_until_complete(examples.request_data_async()) @@ -79,62 +94,77 @@ poetry run python ### How to generate and read locally the autodoc - Install Sphinx, included into the development dependencies: + ```bash poetry install ``` - Generate HTML documentation in `public` directory: + ```bash make docs ``` ## Development -* When writing docstrings, use the reStructuredText format recommended by https://www.python.org/dev/peps/pep-0287/#docstring-significant-features -* Use `make check` commands to check the code and the format. -* Install runtime dependencies +- When writing docstrings, use the reStructuredText format recommended by https://www.python.org/dev/peps/pep-0287/#docstring-significant-features + +- Use `make check` commands to check the code and the format. + +- Install runtime dependencies + ```bash poetry install --no-dev ``` -* Before submitting a merge requests, please check the static typing and tests. +- Before submitting a merge requests, please check the static typing and tests. + +- Install dev dependencies -* Install dev dependencies ```bash poetry install ``` -* Check static typing with [mypy](http://mypy-lang.org/) +- Check static typing with [mypy](http://mypy-lang.org/) + ```bash make mypy ``` ## Packaging and deploy + ### PyPI + Change and commit and tag the new version number (semantic version number) + ```bash ./release.sh 0.42.3 ``` Build the PyPI package in the `dist` folder + ```bash make build ``` Deploy the package to PyPI test repository: + ```bash make deploy_test ``` Install the package from PyPI test repository + ```bash pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.python.org/simple/ duniterpy ``` Deploy the package on the PyPI repository: + ```bash make deploy ``` ## Packaging status + [](https://repology.org/project/python:duniterpy/versions)