Skip to content
Snippets Groups Projects
Commit d3cd7d52 authored by Éloïs's avatar Éloïs
Browse files

feat: add storage benchmarks & base block benchmarks (!118)

* add base block weights

* db weights licence

* remove unused import

* fix base weights computation

* force skip some warnings in weights module

* force skip fmt on weights module

* add storage benchmarks
parent 99d2b419
No related branches found
No related tags found
1 merge request!118add storage benchmarks & base block benchmarks
Pipeline #17779 passed
# How to benchmarks weights of a Call/Hook/Pallet
# Weights benchmarking
## What is the reference machine?
For now (09/2022), it's a `Raspberry Pi 4 Model B - 4GB` with an SSD connected via USB3.
To cross-compile the benchmarks binary for armv7:
```
./scripts/cross-build-arm.sh --features runtime-benchmarks
```
The cross compiled binary is generated here: `target/armv7-unknown-linux-gnueabihf/release/duniter`
## How to benchmarks weights of a Call/Hook/Pallet
1. Create the benchmarking tests, see commit 31057e37be471e3f27d18c63818d57cc907b4b4f for a
complete real example.
......@@ -19,15 +33,27 @@ Note 1: You *must* replace `CURRENCY` by the currency type, or for ĞDev use dir
Note 2: If the reference machine does not support wasmtime, you should replace `--wasm-execution=compiled`
by `--wasm-execution=interpreted-i-know-what-i-do`.
## What is the reference machine?
For now (09/2022), it's a `Raspberry Pi 4 Model B - 4GB` with an SSD connected via USB3.
## Generate base block benchmarking
To cross-compile the benchmarks binary for armv7:
1. Build binary for reference machine and copy it on reference machine.
2. Run base block benchmarks command:
```
./scripts/cross-build-arm.sh --features runtime-benchmarks
./duniter benchmark overhead --chain=gdev --execution=wasm --wasm-execution=interpreted-i-know-what-i-do --weight-path=. --warmup=10 --repeat=100
```
3. Copy the generated file `block_weights.rs` in the codebase in folder `runtime/common/src/weights/`.
4. Commit changes and open an MR.
The cross compiled binary is generated here: `target/armv7-unknown-linux-gnueabihf/release/duniter`
## Generate storage benchmarking
1. Build binary for reference machine and copy it on reference machine.
2. Copy a DB on reference machine (on ssd), example: `scp -r -P 37015 tmp/t1 pi@192.168.1.188:/mnt/ssd1/duniter-v2s/`
3. Run storage benchmarks command, example:
```
./duniter benchmark storage -d=/mnt/ssd1/duniter-v2s/t1 --chain=gdev --mul=2 --weight-path=. --state-version=1
```
4. Copy the generated file `paritydb_weights.rs` in the codebase in folder `runtime/common/src/weights/`.
5. Commit changes and open an MR.
......@@ -15,10 +15,11 @@
// along with Duniter-v2S. If not, see <https://www.gnu.org/licenses/>.
use crate::{Balance, BlockNumber};
use frame_support::weights::constants::WEIGHT_PER_MICROS;
use frame_support::weights::Weight;
use sp_runtime::Perbill;
pub use crate::weights::paritydb_weights::constants::ParityDbWeight as DbWeight;
/// This determines the average expected block time that we are targeting.
/// Blocks will be produced at a minimum duration defined by `SLOT_DURATION`.
/// `SLOT_DURATION` is picked up by `pallet_timestamp` which is in turn picked
......@@ -63,39 +64,23 @@ pub const NORMAL_DISPATCH_RATIO: Perbill = Perbill::from_percent(75);
// WEIGHTS CONSTANTS //
// Read DB weights
pub const READ_WEIGHTS: Weight = 250 * WEIGHT_PER_MICROS; // ~250 µs
// Write DB weights
pub const WRITE_WEIGHTS: Weight = 1_000 * WEIGHT_PER_MICROS; // ~1000 µs
// Execution cost of everything outside of the call itself:
// signature verification, pre_dispatch and post_dispatch
pub const EXTRINSIC_BASE_WEIGHTS: Weight = READ_WEIGHTS + WRITE_WEIGHTS;
// DB weights
frame_support::parameter_types! {
pub const DbWeight: frame_support::weights::RuntimeDbWeight = frame_support::weights::RuntimeDbWeight {
read: READ_WEIGHTS,
write: WRITE_WEIGHTS,
};
}
// Block weights limits
pub fn block_weights(
expected_block_weight: Weight,
normal_ratio: sp_arithmetic::Perbill,
) -> frame_system::limits::BlockWeights {
let base_weight = DbWeight::get().reads(1) + DbWeight::get().writes(1);
let normal_weight = normal_ratio * expected_block_weight;
frame_system::limits::BlockWeights::builder()
.base_block(crate::weights::block_weights::BlockExecutionWeight::get())
.for_class(frame_support::weights::DispatchClass::Normal, |weights| {
weights.base_extrinsic = EXTRINSIC_BASE_WEIGHTS;
weights.base_extrinsic = base_weight;
weights.max_total = normal_weight.into();
})
.for_class(
frame_support::weights::DispatchClass::Operational,
|weights| {
weights.base_extrinsic = EXTRINSIC_BASE_WEIGHTS;
weights.base_extrinsic = base_weight;
weights.max_total = expected_block_weight.into();
weights.reserved = (expected_block_weight - normal_weight).into();
},
......
......@@ -14,6 +14,12 @@
// You should have received a copy of the GNU Affero General Public License
// along with Duniter-v2S. If not, see <https://www.gnu.org/licenses/>.
#![cfg_attr(rustfmt, rustfmt_skip)]
#![allow(unused_parens)]
#![allow(unused_imports)]
#![allow(clippy::unnecessary_cast)]
pub mod block_weights;
pub mod frame_system;
pub mod pallet_babe;
pub mod pallet_balances;
......@@ -24,3 +30,4 @@ pub mod pallet_scheduler;
pub mod pallet_timestamp;
pub mod pallet_universal_dividend;
pub mod pallet_upgrade_origin;
pub mod paritydb_weights;
// Copyright 2021 Axiom-Team
//
// This file is part of Duniter-v2S.
//
// Duniter-v2S is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, version 3 of the License.
//
// Duniter-v2S is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with Duniter-v2S. If not, see <https://www.gnu.org/licenses/>.
//! THIS FILE WAS AUTO-GENERATED USING THE SUBSTRATE BENCHMARK CLI VERSION 4.0.0-dev
//! DATE: 2022-11-18 (Y/M/D)
//! HOSTNAME: `raspberrypi`, CPU: `ARMv7 Processor rev 3 (v7l)`
//!
//! SHORT-NAME: `block`, LONG-NAME: `BlockExecution`, RUNTIME: `Ğdev`
//! WARMUPS: `10`, REPEAT: `100`
//! WEIGHT-PATH: `.`
//! WEIGHT-METRIC: `Average`, WEIGHT-MUL: `1`, WEIGHT-ADD: `0`
// Executed Command:
// ./duniter
// benchmark
// overhead
// --chain=gdev
// --execution=wasm
// --wasm-execution=interpreted-i-know-what-i-do
// --weight-path=.
// --warmup=10
// --repeat=100
use frame_support::{
parameter_types,
weights::{constants::WEIGHT_PER_NANOS, Weight},
};
parameter_types! {
/// Time to execute an empty block.
/// Calculated by multiplying the *Average* with `1` and adding `0`.
///
/// Stats nanoseconds:
/// Min, Max: 23_866_638, 90_077_105
/// Average: 24_871_527
/// Median: 23_915_377
/// Std-Dev: 6645558.32
///
/// Percentiles nanoseconds:
/// 99th: 30_529_787
/// 95th: 27_134_555
/// 75th: 23_951_395
pub const BlockExecutionWeight: Weight = 24_871_527 * WEIGHT_PER_NANOS;
}
#[cfg(test)]
mod test_weights {
use frame_support::weights::constants;
/// Checks that the weight exists and is sane.
// NOTE: If this test fails but you are sure that the generated values are fine,
// you can delete it.
#[test]
fn sane() {
let w = super::BlockExecutionWeight::get();
// At least 100 µs.
assert!(w >= 100 * constants::WEIGHT_PER_MICROS, "Weight should be at least 100 µs.");
// At most 50 ms.
assert!(w <= 50 * constants::WEIGHT_PER_MILLIS, "Weight should be at most 50 ms.");
}
}
// Copyright 2021 Axiom-Team
//
// This file is part of Duniter-v2S.
//
// Duniter-v2S is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, version 3 of the License.
//
// Duniter-v2S is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with Duniter-v2S. If not, see <https://www.gnu.org/licenses/>.
//! THIS FILE WAS AUTO-GENERATED USING THE SUBSTRATE BENCHMARK CLI VERSION 4.0.0-dev
//! DATE: 2022-11-18 (Y/M/D)
//! HOSTNAME: `raspberrypi`, CPU: `ARMv7 Processor rev 3 (v7l)`
//!
//! DATABASE: `ParityDb`, RUNTIME: `Ğdev`
//! BLOCK-NUM: `BlockId::Number(85630)`
//! SKIP-WRITE: `false`, SKIP-READ: `false`, WARMUPS: `1`
//! STATE-VERSION: `V1`, STATE-CACHE-SIZE: `0`
//! WEIGHT-PATH: `.`
//! METRIC: `Average`, WEIGHT-MUL: `2`, WEIGHT-ADD: `0`
// Executed Command:
// ./duniter
// benchmark
// storage
// -d=/mnt/ssd1/duniter-v2s/t1
// --chain=gdev
// --mul=2
// --weight-path=.
// --state-version=1
/// Storage DB weights for the `Ğdev` runtime and `ParityDb`.
pub mod constants {
use frame_support::{
parameter_types,
weights::{constants, RuntimeDbWeight},
};
parameter_types! {
/// `ParityDB` can be enabled with a feature flag, but is still experimental. These weights
/// are available for brave runtime engineers who may want to try this out as default.
pub const ParityDbWeight: RuntimeDbWeight = RuntimeDbWeight {
/// Time to read one storage item.
/// Calculated by multiplying the *Average* of all values with `2` and adding `0`.
///
/// Stats nanoseconds:
/// Min, Max: 62_017, 5_238_182
/// Average: 125_949
/// Median: 124_943
/// Std-Dev: 19659.02
///
/// Percentiles nanoseconds:
/// 99th: 151_424
/// 95th: 143_017
/// 75th: 133_498
read: 250_000 * constants::WEIGHT_PER_NANOS,
/// Time to write one storage item.
/// Calculated by multiplying the *Average* of all values with `2` and adding `0`.
///
/// Stats nanoseconds:
/// Min, Max: 88_054, 107_065_367
/// Average: 419_064
/// Median: 424_994
/// Std-Dev: 423253.1
///
/// Percentiles nanoseconds:
/// 99th: 611_825
/// 95th: 512_789
/// 75th: 457_938
write: 840_000 * constants::WEIGHT_PER_NANOS,
};
}
#[cfg(test)]
mod test_db_weights {
use super::constants::ParityDbWeight as W;
use frame_support::weights::constants;
/// Checks that all weights exist and have sane values.
// NOTE: If this test fails but you are sure that the generated values are fine,
// you can delete it.
#[test]
fn bound() {
// At least 1 µs.
assert!(
W::get().reads(1) >= constants::WEIGHT_PER_MICROS,
"Read weight should be at least 1 µs."
);
assert!(
W::get().writes(1) >= constants::WEIGHT_PER_MICROS,
"Write weight should be at least 1 µs."
);
// At most 1 ms.
assert!(
W::get().reads(1) <= constants::WEIGHT_PER_MILLIS,
"Read weight should be at most 1 ms."
);
assert!(
W::get().writes(1) <= constants::WEIGHT_PER_MILLIS,
"Write weight should be at most 1 ms."
);
}
}
}
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment