Skip to Content

Benchmarking RISCV SBCs with SPEC 2017 (Single/Multi Core)

Benchmarking RISCV SBCs with SPEC 2017 (Single/Multi Core)

By Akif Ejaz

In this article, we’ll be benchmarking some of the available RISC‑V single-board computers (SBCs) using SPEC 2017. We'll be diving into both single-core and multi-core CPU performance to see how these boards handle real-world compute workloads.

We’ll also walk through the complete setup process for running SPEC 2017 on RISC‑V hardware — so you can easily replicate the benchmarks on your own system.

1. System & Environment Details

All system, hardware and environment specifications used in this benchmark — including vendor, board name, OS, compiler version, SoC, CPU, cores, RAM, and RISC‑V extensions — are listed in the table below.

Vendor Board Name OS Compiler (GCC) SOC CPU Cores RAM GB RISCV Extensions
StarFive VisionFive V1 Ubuntu 24.04.1 LTS gcc/g++ 13.2.0 JH7100 U74 @ 1.2GHz + E24 2+1 8 RV64GC
StarFive VisionFive 2 Ubuntu 24.04.1 LTS gcc/g++ 13.2.0 JH7110 U74 @ 1.5GHz 4 8 RV64GC
Milk-V Milk-V Jupiter Bianbu 2.2 gcc/g++ 13.2.0 SpacemiT K1/M1 X60 @ 1.6GHz 8 16 RV64GCVB, RVA22, RVV1.0
Milk-V Milkv Pioneer Box Debian GNU/Linux 13 gcc/g++ 14.2.0 SG2042 C920 @ 2.0GHz 64 128 RVV 0.7.1
Banana Pi Banana Pi BPI-F3 Bianbu 2.2 gcc/g++ 13.2.0 SpacemiT K1 X60 @ 2.0GHz 8 16 RV64GCVB, RVA22, RVV1.0

2. Setup and Installation

SPEC CPU 2017 is a licensed benchmark suite, so you must obtain it from SPEC’s website. In this article, I'll go with "cpu2017-1.1.9.iso" version. Once get the .iso file follow these below steps:

export ISO_FILE="cpu2017-1.1.9.iso"
export MOUNT_DIR="$PWD/spec-mount"
mkdir $MOUNT_DIR && sudo mount "$ISO_FILE" "$MOUNT_DIR"
cd "$MOUNT_DIR" && ./install.sh 

This will prompt you to confirm the path of installation, provide the absolution path where you want to install spec. In my case I installed it in "INSTALL_DIR="$PWD/spec-17"

To confirm the installation you can run these below commands:

cd $INSTALL_DIR && ./bin/runcpu -h

This should list all the available options for SPEC.

3. Running CPU benchmark

There are mainly 4 major benchmark types that SPEC provides as mentioned in 1.3 Benchmarks and Suites. 

Short Tag Suite Contents Metrics How many copies?
What do Higher Scores Mean?
intspeed SPECspeed® 2017 Integer 10 integer benchmarks SPECspeed®2017_int_base
SPECspeed®2017_int_peak
SPECspeed®2017_int_energy_base
SPECspeed®2017_int_energy_peak 		SPECspeed suites always run one copy of each benchmark.
Higher scores indicate that less time is needed.
fpspeed SPECspeed® 2017 Floating Point 10 floating point benchmarks SPECspeed®2017_fp_base
SPECspeed®2017_fp_peak
SPECspeed®2017_fp_energy_base
SPECspeed®2017_fp_energy_peak
intrate SPECrate® 2017 Integer 10 integer benchmarks SPECrate®2017_int_base
SPECrate®2017_int_peak
SPECrate®2017_int_energy_base
SPECrate®2017_int_energy_peak 		SPECrate suites run multiple concurrent copies of each benchmark.
The tester selects how many.
Higher scores indicate more throughput (work per unit of time).
fprate SPECrate® 2017 Floating Point 13 floating point benchmarks SPECrate®2017_fp_base
SPECrate®2017_fp_peak
SPECrate®2017_fp_energy_base
SPECrate®2017_fp_energy_peak
The "Short Tag" is the canonical abbreviation for use with runcpu, where context is defined by the tools. In a published document, context may not be clear.
To avoid ambiguity in published documents, the Suite Name or the Metrics should be spelled as shown above.

In this article we'll be going forward for only 'intrate' other benchmarks seem to have some problem running on RISCV as of today.

In order to run benchmark, follow these commands below.

cd "$INSTALL_DIR"
./bin/runcpu \
        --config=gcc-linux-riscv.cfg \
        --action=run \
        --tune=all \
        --iterations=1 \
        --copies=$(nproc) \
        --output_format=text,html,mail \
        --reportable \
        intrate
Note: Download the "gcc-linux-riscv.cfg" from here. To also run "fprate, intspeed, fpspeed" add these at the end of above command.

In this .cfg file search for 'EDIT' and it should point you, which variables/setting you need to change, mainly we should change the label, num cores, gcc and g++ path and version etc. For more details see all options here.

4. Auto-Setup CoreMark Script for RISCV

You can use this below script which automates setup and running the SPEC 2017 benchmark.

#!/bin/bash
# ==============================================================================
# SPEC CPU 2017 Benchmark - Automation Script (Tuned for Risc-V systems)
#
# Description:
# This script automates the setup and execution of Spec CPU benchmark.
# It runs intrate, fprate only for now.
#
# It's designed for auto-run scenarios — — all user prompts are
# pre-handled within the script, making it ideal for remote execution and
# continuous integration (CI) workflows.
#
# Usage:
# sudo bash <script_name>.sh
#
# Notes:
# - Feel free to customize or Tweak it as needed :)
#
# Author:
# Akif Ejaz | github.com/akifejaz
#
# ==============================================================================
set -euo pipefail
#==============================================================================
# Helper Functions
#==============================================================================
LOG_FILE="/tmp/spec_$(date +%Y%m%d).log"
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; BLUE='\033[0;34m'; NC='\033[0m'
log() {
echo "$(date '+%Y-%m-%d %H:%M:%S') [$1] ${*:2}" | tee -a "$LOG_FILE"
case $1 in
ERROR) echo -e "${RED}[ERROR]${NC} ${*:2}" >&2 ;;
SUCCESS) echo -e "${GREEN}[SUCCESS]${NC} ${*:2}" ;;
WARNING) echo -e "${YELLOW}[WARNING]${NC} ${*:2}" ;;
*) echo -e "${BLUE}[INFO]${NC} ${*:2}" ;;
esac
}
die() { log ERROR "$@"; exit 1; }
check_sudo() {
log INFO "Checking sudo privileges..."
[[ $EUID -eq 0 ]] || die "Must run with sudo: sudo $0"
export USER="${SUDO_USER:-$(whoami)}"
}
# Change these if needed
ISO_FILE="cpu2017-1.1.9.iso"
MOUNT_DIR="$PWD/mnt-spec"
INSTALL_DIR="$PWD/spec-17"
setup_spec() {
log INFO "Installing SPEC ..."
if [[ -z "$ISO_FILE" ]]; then
log ERROR "No ISO file matching '$ISO_FILE' found."
exit 1
fi
# dirs create
mkdir -p "$MOUNT_DIR" "$INSTALL_DIR"
log INFO "Mounting ISO: $ISO_FILE"
sudo mount "$ISO_FILE" "$MOUNT_DIR"
log INFO "Starting SPEC installer..."
cd "$MOUNT_DIR"
INSTALL_SCRIPT="./install.sh"
if [[ ! -x "$INSTALL_SCRIPT" ]]; then
log ERROR "Installer script not found"
exit 1
fi
# Feed input non-interactively to the install script
"$INSTALL_SCRIPT" <<EOF
$INSTALL_DIR
yes
EOF
log SUCCESS "SPEC installed to: $INSTALL_DIR"
}
clean_spec(){
if [[ -d "$INSTALL_DIR" ]]; then
sudo umount "$MOUNT_DIR"
rm -rf "$MOUNT_DIR" "$INSTALL_DIR"
log SUCCESS "Removed $INSTALL_DIR and $MOUNT_DIR"
else
log WARNING "No SPEC installation found to clean."
fi
}
tweak_config() {
log INFO "Tweaking SPEC configuration..."
# Download the example config
curl -L -o $INSTALL_DIR/gcc-linux-riscv.cfg https://www.spec.org/cpu2017/Docs/Example-gcc-linux-riscv.cfg
# label of type <hostname>-<memory>-<ram>-<user>-test
HOSTNAME=$(hostname -s)
MEM=$(free -g | awk '/Mem:/ {print $2}')
RAM=$(df -h / | awk 'NR==2 {print $4}')
USER=$(whoami)
# Replace 'mytest' with dynamic label
sed -i "s|% define label \"mytest\"|% define label \"$HOSTNAME-$MEM\GB-$RAM-$USER-test\"|" $INSTALL_DIR/gcc-linux-riscv.cfg
echo "default:
mailto = akif.ejaz@10xengineers.ai, akifejaz40@gmail.com" >> $INSTALL_DIR/gcc-linux-riscv.cfg
log SUCCESS "Configuration tweaked successfully."
}
run_spec() {
log INFO "Running SPEC benchmark..."
cd "$INSTALL_DIR"
cores=$(nproc)
benchmarks="intrate "
# Run benchmark with multiple copies (multi-core rate mode)
./bin/runcpu \
--config=$INSTALL_DIR/gcc-linux-riscv.cfg \
--action=run \
--tune=all \
--iterations=1 \
--copies="$cores" \
--output_format=text,html,mail \
--reportable \
$benchmarks
if [[ $? -ne 0 ]]; then
log ERROR "SPEC benchmark failed."
exit 1
fi
}
#==============================================================================
# Main Execution
#==============================================================================
main() {
log INFO "Starting SPEC - Log: $LOG_FILE"
# clean_spec
check_sudo
setup_spec
tweak_config
run_spec
log SUCCESS "SPEC benchmark completed successfully. Check $LOG_FILE for details."
}
main "$@"
view raw run-spec.sh hosted with ❤ by GitHub

5. Benchmark Results & Performance Comparison

To cut the effort I've used the above script and that should give you following results.

5.1 Single Core

In order to run for single core make sure to update the `--copies=1 and --thread=1` in command mentioned in section 3. In this article we'll be only looking for multicore results. If you are running for single core or face any issue running SPEC for single core feel free to reach out to us at : cloud-v@10xengineers.io

5.2 Multi Core

Multi Core results for SPEC 2017 are attached below:

6. Overall Results and Conclusion

By running each test multiple times and averaging the results, we cut out random noise and get fair, consistent numbers. Looking at those numbers, the Milk-V Pioneer Box clearly leads the pack. The major reason being its multicore support having 64 Cores. 

7. Appendices & Artifacts

SPEC 2017 Official Site 
See Gist, Auto-Setup SPEC 2017 Script

← Back to RISC-V
Benchmarking Milk-V Megrez