Performance and benchmarks

Measure the boundary your application depends on. Executor throughput helps identify execution bottlenecks; it does not answer how many successful transactions a live fleet can finalize and preserve per second. RPC requests per second, GPU operations per second and successful settled transfers per second are also different quantities.

This page explains what each harness measures, how its accounting changes the conclusion, and why historical numbers need context. For the procedure to select and run correctness checks before measuring a change, use validation strategy.

Choose the right measurement

A one-million-transaction executor invocation also bypasses ordinary block admission/packing. It is not evidence that a normal block can carry one million of that transaction under its byte and requested-compute budgets. Fees and limits owns those limits.

Historical numbers need their original context

docs/PERFORMANCE.md records approximately 232,898 TPS for one million fully Ed25519-verified transactions in a single-node executor benchmark, explicitly excluding consensus and networking. It identifies an AMD Ryzen 9 5900X and RTX 4080 SUPER reference machine.

However, docs/benchmarks.md attributes the same figures to a 128-core/H100 environment and contains older commit/merkle descriptions. The source documents disagree on provenance. This run did not locate a complete raw result pack tying that historical table to an exact commit, machine, state format and current harness behavior, and did not reproduce the measurement. Treat it as a historical documented result, not a verified current capacity promise. See the performance table and the conflicting benchmark context.

Other figures require their own labels:

• Delta/wash aggregation measures many identical deltas collapsed into fewer state transitions. It is not per-transaction network throughput.
• The documented GPU-EVM warm-storage benchmarks exclude signatures, networking and consensus, with specific scratch-transfer choices. They are not a CuEVM-enabled node's end-to-end capacity.
• CPU-only figures with unstated verification mode cannot be compared as security-equivalent verified throughput.
• Zone-count multiplication is not a measured multi-zone benchmark.
• A latest-block finality gauge is an observation, not a tail-latency distribution or a service-level guarantee.

The runtime architecture includes Bombil at VM level, superseding old “five VMs” positioning in performance notes. Neither that architecture nor its name supplies a Bombil throughput result.

Inspect what the harness actually does

Verified transfer executor

The verified section constructs funded benchmark state and signed transactions before the execution timer. It configures verification on, requests the GPU verifier and prints the actual verifier name and whether GPU use was achieved. A request for GPU acceleration is not proof that the run used it.

The harness calls execute_block_verified_ultrafast and explicitly finalizes the returned commit/merkle handle. It reports successful, failed and verification-failed counts alongside stage timings and the root. Inspect those counts: a timer divided by all submitted items can look better when more transactions fail.

BENCH_SECTIONS=verified selects 1,000, 100,000, 500,000 and 1,000,000 transaction cases. It avoids unrelated sections, but still represents substantial work rather than a quick smoke test. BENCH_STATE_DIR chooses its state-directory parent. Source: verified section, section selection.

“Diverse” System workload

Despite its filename and --token-pct, --stake-pct and --vote-pct knobs, bench_diverse generates System transfers and CreateAccount. The token/stake/vote-like generators change amounts, not the target program. It therefore cannot establish actual token, staking and voting throughput or representative cross-program contention.

The multi-block loop calls execute_block_preclassified(..., true, ...), declaring the transactions preverified. It overlaps the previous block's deferred-merkle wait with current work, then accounts for those waits. Report that mode separately from inline verification. The commit profile distinguishes executor-thread consolidation/commit work from drain-thread serialization, WAL sync, key persistence and related storage phases. See generator scope and multi-block timing.

GPU SBF trace replay

svm-trace-replay synth defaults to sixteen programs, twenty blocks, 1,024 transactions per block, 50,000 iterations in the generated program and two warmup blocks. It installs the canonical GPU artifact executor. Captured replay takes a finalized trace and an associated state snapshot; without matching pre-state, failures and timings need not represent the captured chain.

The execution loop uses preverified transactions. It invokes a commit finalizer but drops its deferred handle without calling its explicit finalization wait. Its measured wall therefore must not be described as waiting for the full deferred state-root/durability boundary. The report divides total submitted transactions by measured wall and separately prints successes/failures and block p50/p95/p99. These are block latencies, not client transaction-finality percentiles. See defaults, timer and reporting.

Regression and RPC suites

The Turbo Criterion suite includes fresh-state, 1,024-transfer cases with signature verification and fees disabled. That is useful for measuring changes to a component while controlling setup; it is not equivalent to the verified section above.

The regular benchmark workflow is advisory and permits command failure. CodSpeed uses CPU simulation for relative comparisons; simulation measurements are not native wall-clock CUDA throughput. Also distinguish a job's allowed failure from an independently reported service check when deciding what CI actually gates. Source: Criterion configuration, benchmark workflow, CodSpeed workflow.

rpc_load_test defaults to local RPC, thirty seconds and half the detected CPU count, with at least one worker. It tracks request successes and latency samples per method. A responsive method can be cheap metadata rather than application execution. Run it only against a selected test environment; it is a load generator, not a read-only diagnostic probe with negligible effects.

A reproduction recipe must name its regime

For an intended GPU benchmark host with the required toolchain and CuEVM/GPU libraries installed, this source-derived recipe selects the verified section:

cd runtime
../scripts/eto-build cargo build --release --features cuda --bin turbo_bench
BENCH_STATE_DIR=/path/to/dedicated/benchmark-storage \
  BENCH_SECTIONS=verified ./target/release/turbo_bench

Replace the storage path with a dedicated existing location appropriate to the machine. Preserve stdout and stderr, source revision, build flags and all benchmark output. This command was not executed during documentation generation. It can consume substantial CPU/GPU time, memory and disk.

For a CPU control, use a separate explicitly identified build with --no-default-features --features mimalloc. That changes the experiment; it does not reproduce the GPU regime. Do not enable bench-skip-verify or bench-preload in a production node to obtain a faster result.

A captured GPU replay has a different contract:

svm-trace-replay replay --trace /path/to/finalized.etotrace   --snapshot /path/to/matching-snapshot --warmup 2 --json replay-result.json

The trace and snapshot identifiers must travel with the result. A screenshot of a TPS line loses the information needed to assess it.

Make comparisons reviewable

Record the commit and dirty state; compiler/profile/features; CPU model, cores and affinity; GPU/driver/CUDA; memory; storage/filesystem; state size; transaction generator and account-conflict distribution; warmup; measured duration; and log/profiler settings.

Then report offered, accepted and successful counts, rejected/failed counts, backlog growth and the correct timing denominator. Compare equal semantics: same verification, fee/account effects and relevant state/root checks. A faster result caused by dropped work or an omitted persistence wait is not an optimization of the original workload.

Use stage timers for diagnosis, not as a substitute for a wall timer. Concurrent verify, execute and drain work need not add linearly. Long-enough runs expose periodic eviction, log rotation, storage flush and epoch work that a single tiny batch misses.

For a network claim, add the actual node topology and network, client-observed confirmation distribution, common-height state agreement, and durable progress. For HILSA settlement, include source observation/finality and destination verification/execution latency; local executor TPS cannot predict those costs.

No new throughput benchmark was run for this wiki. The executed checks were bounded SDK/CLI/configuration validations. Use validation strategy to establish correctness before measuring a change.