diff --git a/docs/dev/img/pipeline_screenshot.png b/docs/dev/img/pipeline_screenshot.png new file mode 100644 index 0000000000000000000000000000000000000000..2c2870de4baa32f9f9f0a5a18bb280c7be926366 Binary files /dev/null and b/docs/dev/img/pipeline_screenshot.png differ diff --git a/docs/dev/launch-a-live-network.md b/docs/dev/launch-a-live-network.md index 833edeb3d59a2454703317f74f7dd6d4ecda862d..cc87050d84d0466a55a7d876f1188bfb3647cce3 100644 --- a/docs/dev/launch-a-live-network.md +++ b/docs/dev/launch-a-live-network.md @@ -1,22 +1,43 @@ # How to launch a live network -## 1. Choose the currency type +Launching a new live network is more difficult than spawning a local blockchain. Follow this process if you know what you are doing and you already experimented a bit with local blockchains. Part of this process is automated with Rust scripts, including interaction with GitLab's GraphQL API. Do not hesitate to improve and complete it (see TODOs inside `xtask/**/*.rs` files). -Ensure that the currency type you want has the requirements. +## Requirements + +In order to build in a standardized environment, you need Docker. + +- see docker docs to [install docker](https://docs.docker.com/engine/install/) +- make sure you can run docker as non-root user with `docker info` or so + +## Preparation + +When launching a new network, you're likely to use a new runtime. See how to [release a new runtime](./release-new-runtime.md). + +### Inject runtime in chainspec + +ÄžDev runtime is automatically embeded in the raw chainspec with the `include_bytes!` macro. An other way to inject the runtime is to use "inject-runtime-code" xtask: + +```bash +cargo xtask inject-runtime-code --runtime runtime/gdev/target/srtool/release/wbuild/gdev-runtime/gdev_runtime.compact.compressed.wasm --raw-spec resources/gdev-raw.json +``` + +## Bootstraping + +### Choose the currency type For now, only `gdev` is supported. In the commands that will be indicated afterwards, you will have to replace `CURRENCY` by the currency type you have chosen. -## 2. Choose the docker image +### Choose the docker image -Choose or build the docker image that contains the version of the code thut you want to use. +Choose the docker image that contains the version of the code that you want to use. In the commands that will be indicated afterwards, you will have to replace `TAG` by the tag of the -docker image that you have chosen. +docker image that you have chosen (example : runtime-400). -## 3. Generate the session keys of genesis authority +### Generate the session keys of genesis authority Generate a random secret phrase: @@ -39,11 +60,11 @@ $ docker run --rm duniter/duniter-v2s:TAG -- key generate-session-keys --chain C Session Keys: 0x87189d723e1b2826c243bc433c718ac26ba60526932216a09102a254d54462b890a0c2866034db9d05f8193a95fe5af8d5e12ab295a501c17c95cdbeaf226d6290a0c2866034db9d05f8193a95fe5af8d5e12ab295a501c17c95cdbeaf226d6290a0c2866034db9d05f8193a95fe5af8d5e12ab295a501c17c95cdbeaf226d62 ``` -## 4. Paste sessions keys in the genesis configuration file +### Paste sessions keys in the genesis configuration file -An example of genesis configuration file: `resources/gdev.json` +An example of genesis configuration file: `resources/gdev.json`. Paste your session keys in your `smith` identity with key `session_keys`. -## 5. Generate raw spec +### Generate raw spec ```docker docker run -v $HOME/dev/duniter-v2s/resources:/var/lib/duniter/resources -e DUNITER_GENESIS_CONFIG=/var/lib/duniter/resources/gdev.json --rm duniter/duniter-v2s:TAG -- build-spec -lerror --chain=gdev-gl --raw > name-raw.json @@ -53,24 +74,46 @@ docker run -v $HOME/dev/duniter-v2s/resources:/var/lib/duniter/resources -e DUNI ./scripts/gen-live-network-raw-spec.sh CURRENCY "<path/to/your/genesis/config/file>" ``` -## 6. Generate the docker compose and prepare nodes keys +This builds the specs using debug version of Duniter. + +### Generate the docker compose and prepare nodes keys ```bash ./scripts/create-live-network.sh "<your secret phrase>" CURRENCY "<path/to/dist/folder>" ``` -The distribution folder can then be copied to a server, then the compose must be launched from the -root of the distribution folder: +The new distribution folder can be copied to a server ```bash -scp -r -P SSH_PORT "<path/to/dist/folder>" user@ip:/remote/dist/path -cd "<path/to/dist/folder>" -docker compose up -d +scp -r "<path/to/dist/folder>" <server>:/remote/dist/path ``` -Then, on the server: +then on the server, launch the compose file from the the distribution folder's root: ```bash +ssh <server> cd "<path/to/dist/folder>" docker compose up -d ``` + +This is the first node of the new live network. + +## Finalization + +The following steps should be completed once you are satisfied with the new live network. + +### Rotate session keys + +You should rotate session keys for more secured keys produced on the server (the one you used before are still in your develop machine bash history and clipboard). + +### Publish image + +With these new session keys in the chainspec and the runtime build with srtool, you can release the new runtime again with: + +```bash +cargo xtask release-runtime 400 +``` + +### Tell the other smith + +Once you completed all these steps, the other smith can pull the docker image with a genesis containing your bootnode with the correct session keys. They can base their `docker-compose.yml` on the `duniter-validator` template. diff --git a/docs/dev/release-new-runtime.md b/docs/dev/release-new-runtime.md new file mode 100644 index 0000000000000000000000000000000000000000..43920e1c5486ba9fee8b8839ef5b05bf98683ece --- /dev/null +++ b/docs/dev/release-new-runtime.md @@ -0,0 +1,41 @@ +# Release a new runtime + +Here you will learn how to release a new runtime using `gitlab ci` and `cargo xtask`. + +## Runtime tag and spec version + +Our runtime tags use `xxyy` version numbers where `x` corresponds to major change and `y` hotfix. + +1. Make sure to move any issue or merge request assigned to the choosen milestone `runtime-xxyy` to the next one. This prevents from forgetting unfinished work. +1. Check that the [CI on release/runtime-XX00](https://git.duniter.org/nodes/rust/duniter-v2s/-/pipelines?scope=all&page=1&ref=runtime-400) (runtime major release branch) is passing. This is necessary to build the docker images. +1. Increment the `spec_version` in the code. Example `spec_version: 300` -> `spec_version: 400`. + +Publish the `runtime-400` branch. Go to the pipelines and run the manual tasks as shown on the screenshot below. + +- `build_release_manual` +- `deploy_docker_release_sha` + + + +## Release runtime + +We choose [`xtask`](https://github.com/matklad/cargo-xtask/) to run Rust scripts using `cargo`. To build these scripts, just run: + +```bash +cargo xtask -h # this will build the scripts and show the available commands +``` + +To interact with GitLab, the `release_runtime` script uses the environment variable `GITLAB_TOKEN` which must contain a token with write permission on the repository. Go to your [personal access token preferences](https://git.duniter.org/-/profile/personal_access_tokens) to generate one and `export` it (you might want to add it to your `.bashrc`). You can then run this command: + +```bash +cargo xtask release-runtime 400 # requires to run docker as non-root user +``` + +This command does the following things (use your system monitor to track the progress as it produces not log): + +1. Download the [`paritytech/srtool`](https://docs.substrate.io/reference/command-line-tools/srtool/) (Substrate Runtime TOOLbox) docker image (about 2 minutes with a good connection). It is a docker container enabling to build the Substrate WebAssembly runtime in a deterministic way (machine-independant). +1. Build gdev runtime in release mode using `srtool` (about 5 minutes depending on your computing power). The target is put in the `runtime/gdev/target` folder (about 2GB). +1. Generate release notes using the template in `xtask/res` +1. Publish the release with the notes on GitLab using your `GITLAB_TOKEN` + +This release will trigger a GitLab CI publishing a docker image of the Substrate client with embeded runtime. \ No newline at end of file