Quickstart

This guide walks through the three common workflows:

1. Single-size benchmark (default mode)
2. Multi-size series sweep
3. Crossover analysis (finding size thresholds where structures change relative performance)
4. Parsing JSON output with a small Python helper

All commands assume you have run a build already (e.g. scripts/build.sh -t Release --test).

Tip: Prefer --profile ci, --profile series, --profile crossover, or --profile deep for repeatable named workflows, then override only the few knobs you intentionally want to change.

1. Single-size benchmark

Run the executable directly specifying a size and number of runs, or use a canonical named profile:

./build/hashbrowns --profile ci --out-format json --output benchmark_single.json

You’ll see a colorful banner followed by benchmark progress:

🥔 hashbrowns - C++ Data Structure Benchmarking Suite
======================================================

=== Benchmark Results (avg ms over 5 runs, size=10000) ===
- array: insert=712.99, search=16337.3, remove=199231, mem=655416 bytes
- slist: insert=336.687, search=57940.9, remove=79.0626, mem=480104 bytes
- hashmap: insert=1099.77, search=198.998, remove=142.816, mem=786664 bytes

Saved JSON to: benchmark_single.json

Key points: - --structures picks the set; omit it to use all defaults (array, slist, dlist, hashmap). - --out-format csv would emit a CSV instead. - --memory-tracking includes allocation deltas (the mem= values shown). - --bootstrap 200 computes 95% confidence intervals for statistical rigor.

Minimal CSV example:

./build/hashbrowns --size 10000 --runs 8 --output results/csvs/benchmark_results.csv

2. Multi-size series

Use a linear series of sizes up to a max (given by --size) with --series-count:

./build/hashbrowns --profile series

This produces 6 evenly spaced sizes between a small floor and 60,000. For explicit sizes:

./build/hashbrowns --series-sizes 1024,4096,8192,16384,32768,65536 --series-runs 2 --out-format csv \
  --series-out results/csvs/series_results.csv

JSON series output contains an array of per-size measurements plus a meta block including runs_per_size, seed, and the selected profile.

Wizard alternative (interactive prompts):

./build/hashbrowns --wizard

3. Crossover analysis

Estimate sizes where one structure overtakes another for each operation:

./build/hashbrowns --profile crossover

Optional time budget:

./build/hashbrowns --crossover-analysis --max-size 200000 --max-seconds 15 --runs 2 \
  --structures array,hashmap --out-format csv --output results/csvs/crossover_results.csv

Each crossover row captures an operation (insert|search|remove), structure pair (a,b), and estimated size at crossover. Use JSON for richer meta when publishing.

4. Parsing JSON output (Python example)

We provide an example parser at scripts/example_parse.py:

python3 scripts/example_parse.py results/csvs/benchmark_results.json

Sample output:

Structure  Insert(ms)  Search(ms)  Remove(ms)  Memory(bytes)
array      12.34       4.11        8.22        65536
hashmap    7.91        5.02        6.10        98304

Add --summary to reduce output to a one-line aggregate or --csv to emit a condensed table for piping.

Schema awareness

All JSON blobs contain schema_version. The parser checks this first. If you upgrade the schema, adjust the script or add a validation step (scripts/validate_json_schema.py, planned).

Reproducibility checklist

For comparing benchmark runs over time: - Pin the CPU or disable turbo on Linux (--pin-cpu --no-turbo). - Fix random seed: --seed 12345. - Record branch / commit (--version flag is script friendly). - Include bootstrap CIs for statistical confidence. - Document environment meta (already embedded in JSON).

Troubleshooting

Next steps

Explore deeper memory & probe interpretation in memory_and_probes.md (to be added), or build plots with:

scripts/run_benchmarks.sh --runs 8 --size 50000 --max-size 65536 --plots --yscale auto

Check your version:

./build/hashbrowns --version
# Output: hashbrowns 1.0.0 (git b792a4354ada)

Last updated: February 2026