| Benchmark | CV% | 📊 ./main-65abe37.so | …branch-a84af50.so |
|---|---|---|---|
| cm-online-stats/cm-online-stats | 2.26% | 882.1M cycles (p25) | No significant difference |
| dart-deltablue/dart-deltablue | 1.26% | 75.6M cycles (p25) | No significant difference |
| dotnet-raytracer/dotnet-raytracer | 1.84% | 45.3M cycles (p25) | No significant difference |
| gcc-loops | 0.15% | 6.2M cycles (p25) | 2.11% faster |
| hashset | 1.08% | 10.3M cycles (p25) | 0.38% slower |
| hex-simd | 1.01% | 7.9M cycles (p25) | No significant difference |
| hoot-earley-boyer/hoot-earley-boyer | 6.35% | 205.4M cycles (p25) | No significant difference |
| intgemm-simd | 1.84% | 7.3M cycles (p25) | 0.78% faster |
| java-navier-stokes/java-navier-stokes | 0.11% | 16.9M cycles (p25) | No significant difference |
| kotlin-richards/kotlin-richards | 1.41% | 38.0M cycles (p25) | 0.73% slower |
| libsodium/libsodium-core5 | 0.08% | 26.0M cycles (p25) | No significant difference |
| libsodium/libsodium-hash3 | 0.57% | 5.7M cycles (p25) | 0.82% faster |
| libsodium/libsodium-keygen | 0.19% | 22.6M cycles (p25) | No significant difference |
| libsodium/libsodium-siphashx24 | 3.67% | 5.8M cycles (p25) | 1.14% faster |
| ocaml-raytrace/ocaml-raytrace | 1.74% | 52.0M cycles (p25) | No significant difference |
| richards | 1.47% | 19.7M cycles (p25) | No significant difference |
| rust-json | 10.57% | 12.0M cycles (p25) | No significant difference |
| rust-protobuf | 2.42% | 25.6M cycles (p25) | 0.69% faster |
| rust-wasmprinter/rust-wasmprinter | 2.05% | 22.1M cycles (p25) | No significant difference |
| shootout/shootout-heapsort | 0.74% | 25.1M cycles (p25) | No significant difference |
| shootout/shootout-keccak | 9.18% | 18.7M cycles (p25) | No significant difference |
| shootout/shootout-memmove | 1.55% | 338.2M cycles (p25) | No significant difference |
| shootout/shootout-sieve | 4.93% | 7.3M cycles (p25) | No significant difference |
| shootout/shootout-switch | 1.18% | 1.0M cycles (p25) | No significant difference |
| shootout/shootout-xchacha20 | 0.37% | 5.9M cycles (p25) | 0.19% slower |
| splay/splay | 0.71% | 4.7B cycles (p25) | No significant difference |
| starlingmonkey/starlingmonkey-markdown | 1.72% | 7.1M cycles (p25) | 1.08% slower |
| tinygo/tinygo-regex | 1.01% | 13.4M cycles (p25) | 0.62% faster |
| wat-bytecode-interp/wat-bytecode-interp | 0.64% | 6.6M cycles (p25) | No significant difference |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 882.1M cycles (p25) | — | — | 2.26% |
| …branch-a84af50.so | 879.7M cycles | No significant difference | ±5.9M cycles | 2.41% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 75.6M cycles (p25) | — | — | 1.26% |
| …branch-a84af50.so | 75.8M cycles | No significant difference | ±282.3K cycles | 1.37% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 45.3M cycles (p25) | — | — | 1.84% |
| …branch-a84af50.so | 45.6M cycles | No significant difference | ±202.1K cycles | 1.25% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 6.2M cycles (p25) | — | — | 0.15% |
| …branch-a84af50.so | 6.1M cycles | 2.11% faster | ±2.6K cycles | 0.15% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 10.3M cycles (p25) | — | — | 1.08% |
| …branch-a84af50.so | 10.4M cycles | 0.38% slower | ±27.4K cycles | 0.77% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 7.9M cycles (p25) | — | — | 1.01% |
| …branch-a84af50.so | 7.9M cycles | No significant difference | ±20.0K cycles | 0.77% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 205.4M cycles (p25) | — | — | 6.35% |
| …branch-a84af50.so | 205.7M cycles | No significant difference | ±3.8M cycles | 6.39% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 7.3M cycles (p25) | — | — | 1.84% |
| …branch-a84af50.so | 7.3M cycles | 0.78% faster | ±36.6K cycles | 1.66% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 16.9M cycles (p25) | — | — | 0.11% |
| …branch-a84af50.so | 16.9M cycles | No significant difference | ±5.4K cycles | 0.12% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 38.0M cycles (p25) | — | — | 1.41% |
| …branch-a84af50.so | 38.1M cycles | 0.73% slower | ±193.2K cycles | 2.10% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 26.0M cycles (p25) | — | — | 0.08% |
| …branch-a84af50.so | 26.0M cycles | No significant difference | ±5.2K cycles | 0.06% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 5.7M cycles (p25) | — | — | 0.57% |
| …branch-a84af50.so | 5.7M cycles | 0.82% faster | ±9.3K cycles | 0.59% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 22.6M cycles (p25) | — | — | 0.19% |
| …branch-a84af50.so | 22.6M cycles | No significant difference | ±10.0K cycles | 0.11% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 5.8M cycles (p25) | — | — | 3.67% |
| …branch-a84af50.so | 5.8M cycles | 1.14% faster | ±43.2K cycles | 0.55% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 52.0M cycles (p25) | — | — | 1.74% |
| …branch-a84af50.so | 52.2M cycles | No significant difference | ±533.8K cycles | 4.75% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 19.7M cycles (p25) | — | — | 1.47% |
| …branch-a84af50.so | 19.7M cycles | No significant difference | ±97.9K cycles | 2.00% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 12.0M cycles (p25) | — | — | 10.57% |
| …branch-a84af50.so | 12.1M cycles | No significant difference | ±274.6K cycles | 2.77% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 25.6M cycles (p25) | — | — | 2.42% |
| …branch-a84af50.so | 25.4M cycles | 0.69% faster | ±155.7K cycles | 1.86% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 22.1M cycles (p25) | — | — | 2.05% |
| …branch-a84af50.so | 22.1M cycles | No significant difference | ±119.9K cycles | 1.76% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 25.1M cycles (p25) | — | — | 0.74% |
| …branch-a84af50.so | 25.1M cycles | No significant difference | ±45.8K cycles | 0.55% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 18.7M cycles (p25) | — | — | 9.18% |
| …branch-a84af50.so | 18.7M cycles | No significant difference | ±584.7K cycles | 12.20% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 338.2M cycles (p25) | — | — | 1.55% |
| …branch-a84af50.so | 337.9M cycles | No significant difference | ±1.4M cycles | 1.29% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 7.3M cycles (p25) | — | — | 4.93% |
| …branch-a84af50.so | 7.3M cycles | No significant difference | ±108.7K cycles | 5.32% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 1.0M cycles (p25) | — | — | 1.18% |
| …branch-a84af50.so | 1.0M cycles | No significant difference | ±3.5K cycles | 1.23% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 5.9M cycles (p25) | — | — | 0.37% |
| …branch-a84af50.so | 5.9M cycles | 0.19% slower | ±7.9K cycles | 0.56% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 4.7B cycles (p25) | — | — | 0.71% |
| …branch-a84af50.so | 4.7B cycles | No significant difference | ±11.8M cycles | 1.05% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 7.1M cycles (p25) | — | — | 1.72% |
| …branch-a84af50.so | 7.1M cycles | 1.08% slower | ±47.5K cycles | 2.84% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 13.4M cycles (p25) | — | — | 1.01% |
| …branch-a84af50.so | 13.3M cycles | 0.62% faster | ±70.5K cycles | 2.43% |
| Engine | Cycles (p25) | Performance | Confidence Interval | CV% |
|---|---|---|---|---|
| 📊 ./main-65abe37.so | 6.6M cycles (p25) | — | — | 0.64% |
| …branch-a84af50.so | 6.6M cycles | No significant difference | ±12.1K cycles | 0.65% |
Uses effect size analysis with confidence intervals based on Student's t-test. Statistical significance indicates the difference is likely real (not measurement noise). Tests with CV > 5% may have unreliable measurements.
Baseline Engine (📊): The baseline engine is the reference point for all comparisons. Performance differences are calculated relative to the baseline. The baseline engine appears with a chart icon (📊) and is highlighted with a light blue background in detailed tables.
P25 Metric: Cycle counts show the 25th percentile (p25) rather than mean or median. This metric is more robust to outliers, as noise in benchmarking typically appears as spikes on the higher end (system interrupts, background processes, etc.). P25 represents a "good run" that is consistently achievable—25% of runs are this fast or faster.
Color Coding: Green indicates significantly faster performance, red indicates significantly slower, and gray indicates high measurement noise (CV > 5%). No highlighting means no statistically significant difference from baseline.
References: Benchmark Suite RFC | Sightglass Documentation