DeterminismTest fails when we build on RISC-V with GCC. GCC's RVV vectorizer generates vfmacc.vv / vfmadd.vv / vfmsac.vv / vfmsub.vv / vfnmsac.vv / vfnmsub.vv, even though we build with -ffp-contract=off. FMA rounds a + b * c once instead of twice, so each fused operation differs from the unfused one up to 1 ULP. Verified on QEMU and real SpacemiT x60 hardware (Banana Pi BPI-F3).
Setup
- RISC-V GCC 16.1.0
- Clang 22.1.6 (for comparison)
- QEMU 11.0.1 riscv64
- box2d commit
56edae79f2949d86142b03450d5d60f63bcf5a6f
- CMake flag:
-DCMAKE_BUILD_TYPE=Release
- Compiler flag:
-march=rv64gcv
-ffp-contract=off stops GCC from fusing mul and add into a single FMA instruction. Scalar code generator obeys this rule, but RISC-V/GCC auto-vectorizer doesn't. It generates FMA vector instructions regardless of the fp-contract setting.
Some statistics
FMA instructions per object file:
contact_solver.c.o: 110 (20 vfmacc.vv, 32 vfmadd.vv, 11 vfmsac.vv, 5 vfmsub.vv, 18 vfnmsac.vv, 24 vfnmsub.vv)
joint.c.o: 2 (1 vfmacc.vv, 1 vfmadd.vv)
manifold.c.o: 1 (1 vfmacc.vv)
motor_joint.c.o: 2 (1 vfmadd.vv, 1 vfnmsub.vv)
recording_replay.c.o: 3 (3 vfmadd.vv)
shape.c.o: 1 (1 vfmadd.vv)
scalar fmadd.s: 0
FMA instructions per function:
b2SolveContactsTask: 58 (9 vfmacc.vv, 18 vfmadd.vv, 6 vfmsac.vv, 2 vfmsub.vv, 9 vfnmsac.vv, 14 vfnmsub.vv)
b2ApplyRestitutionTask: 30 (6 vfmacc.vv, 8 vfmadd.vv, 1 vfmsac.vv, 3 vfmsub.vv, 4 vfnmsac.vv, 8 vfnmsub.vv)
b2WarmStartContactsTask: 22 (5 vfmacc.vv, 6 vfmadd.vv, 4 vfmsac.vv, 5 vfnmsac.vv, 2 vfnmsub.vv)
b2CollidePolygons: 6 (4 vfmacc.vf, 2 vfmadd.vv)
b2RecPlayer_DrawFrameQueries: 3 (3 vfmadd.vv)
b2DrawJoint: 3 (1 vfmacc.vv, 1 vfmadd.vv, 1 vfmadd.vf)
CreateWasher: 2 (1 vfmacc.vf, 1 vfmadd.vf)
b2WarmStartWeldJoint: 2 (1 vfmadd.vv, 1 vfnmsub.vv)
b2WarmStartRevoluteJoint: 2 (1 vfmadd.vv, 1 vfnmsub.vv)
b2WarmStartMotorJoint: 2 (1 vfmadd.vv, 1 vfnmsub.vv)
Human_SetScale: 1 (1 vfmadd.vv)
b2GetShapeCentroid: 1 (1 vfmadd.vv)
Clang with identical flags has none of these.
Build & run commands
cmake -S . -B build-riscv -DCMAKE_TOOLCHAIN_FILE=riscv64-toolchain.cmake
cmake --build build-riscv
qemu-riscv64 -L $SYSROOT build-riscv/bin/test DeterminismTest
Expected: pass (sleepStep=294, hash=0x006F0F5E)
Actual: fail (sleepStep=261, hash=0x98B124F7)
This appears to be a GCC bug. The proposed quick workaround is in attach.
fix.patch
DeterminismTest fails when we build on RISC-V with GCC. GCC's RVV vectorizer generates
vfmacc.vv/vfmadd.vv/vfmsac.vv/vfmsub.vv/vfnmsac.vv/vfnmsub.vv, even though we build with-ffp-contract=off. FMA roundsa + b * conce instead of twice, so each fused operation differs from the unfused one up to 1 ULP. Verified on QEMU and real SpacemiT x60 hardware (Banana Pi BPI-F3).Setup
56edae79f2949d86142b03450d5d60f63bcf5a6f-DCMAKE_BUILD_TYPE=Release-march=rv64gcv-ffp-contract=off stops GCC from fusing mul and add into a single FMA instruction. Scalar code generator obeys this rule, but RISC-V/GCC auto-vectorizer doesn't. It generates FMA vector instructions regardless of the fp-contract setting.
Some statistics
FMA instructions per object file:
contact_solver.c.o: 110 (20 vfmacc.vv, 32 vfmadd.vv, 11 vfmsac.vv, 5 vfmsub.vv, 18 vfnmsac.vv, 24 vfnmsub.vv)
joint.c.o: 2 (1 vfmacc.vv, 1 vfmadd.vv)
manifold.c.o: 1 (1 vfmacc.vv)
motor_joint.c.o: 2 (1 vfmadd.vv, 1 vfnmsub.vv)
recording_replay.c.o: 3 (3 vfmadd.vv)
shape.c.o: 1 (1 vfmadd.vv)
scalar fmadd.s: 0
FMA instructions per function:
b2SolveContactsTask: 58 (9 vfmacc.vv, 18 vfmadd.vv, 6 vfmsac.vv, 2 vfmsub.vv, 9 vfnmsac.vv, 14 vfnmsub.vv)
b2ApplyRestitutionTask: 30 (6 vfmacc.vv, 8 vfmadd.vv, 1 vfmsac.vv, 3 vfmsub.vv, 4 vfnmsac.vv, 8 vfnmsub.vv)
b2WarmStartContactsTask: 22 (5 vfmacc.vv, 6 vfmadd.vv, 4 vfmsac.vv, 5 vfnmsac.vv, 2 vfnmsub.vv)
b2CollidePolygons: 6 (4 vfmacc.vf, 2 vfmadd.vv)
b2RecPlayer_DrawFrameQueries: 3 (3 vfmadd.vv)
b2DrawJoint: 3 (1 vfmacc.vv, 1 vfmadd.vv, 1 vfmadd.vf)
CreateWasher: 2 (1 vfmacc.vf, 1 vfmadd.vf)
b2WarmStartWeldJoint: 2 (1 vfmadd.vv, 1 vfnmsub.vv)
b2WarmStartRevoluteJoint: 2 (1 vfmadd.vv, 1 vfnmsub.vv)
b2WarmStartMotorJoint: 2 (1 vfmadd.vv, 1 vfnmsub.vv)
Human_SetScale: 1 (1 vfmadd.vv)
b2GetShapeCentroid: 1 (1 vfmadd.vv)
Clang with identical flags has none of these.
Build & run commands
Expected: pass (sleepStep=294, hash=0x006F0F5E)
Actual: fail (sleepStep=261, hash=0x98B124F7)
This appears to be a GCC bug. The proposed quick workaround is in attach.
fix.patch