UQO Working Papers · WP40 · v1.0
RAID: A Metal-Accelerated C++ Framework for Reproducible High-Frequency Financial Econometrics
Batched, Fused and Streaming Estimation on Apple Silicon without Sacrificing Inference
- Year
- 2026
- First release
- November 2026
- Latest revision
- November 2026
- Publication status
- Working paper
High-frequency econometrics on an FP32-only GPU: accurate and reproducible, but only sometimes faster
RAID carries every Metal accumulator as a compensated double-float pair and combines chunked partial sums in FP64, so GPU and CPU results agree to 10⁻⁶ and are bitwise reproducible; on one year of 1-minute bars for 1,000 US equities the fused realized kernel beats an 18-thread CPU beyond about 4 × 10⁴ elements, batched GARCH likelihood grids gain 5–8×, rolling sums and single-vector recursions lose, and conversion traffic, not kernel time, sets GPU wall-clock.
Abstract
We ask whether heterogeneous high-frequency econometric workloads—realized measures, rolling and batched regressions, HAR forecasting models, GARCH-family quasi-maximum likelihood, block bootstrap inference—can be reorganised into batched, fused, streaming computation on Apple Silicon GPUs without sacrificing estimator accuracy, statistical inference, or reproducibility. Apple GPUs expose no 64-bit floating point. RAID answers with a C++20 library whose FP64 CPU backend is the reference, whose Metal backend accumulates every statistic in compensated “double-float” arithmetic and combines chunked partial sums in FP64, and whose estimators report backend, precision, convergence and provenance in every result.
Tier A methods reproduce highfrequency, rugarch, statsmodels and arch to documented tolerances; Monte Carlo experiments confirm bias, RMSE and coverage; GPU and CPU results agree to 10⁻⁶ relative and are bitwise reproducible across runs and thread counts. Using one year of 1-minute bars for a seeded universe of 1,000 US equities served by a private FirstRate Data lakehouse, we measure break-even points on an M5 Max laptop and across eleven Apple Silicon generations (M2 to M6).
The GPU wins on fused realized measures beyond roughly 4 × 10⁴ elements and on batched GARCH likelihood evaluation when many parameter vectors are evaluated per series; the CPU wins on rolling sums, on single-vector GARCH recursions and on small panels. Conversion and buffer traffic, not kernel time, dominate GPU wall-clock: unified memory is not free computation. We publish the negative results, the tolerances, and a reproducible benchmark protocol.
Key findings
10⁻⁶
relative agreement between Metal (FP32 double-float) and FP64 realized measures and regression statistics (2 × 10⁻⁵ on batched GARCH likelihoods), bitwise reproducible across runs and 1, 4 or 18 threads
≈ 4 × 10⁴ elements
break-even of the fused realized kernel against the 18-thread CPU on the M5 Max; 2.0–3.4× at the kernel level, 1.07–1.41× once the estimator's FP64 post-processing is counted on 40 million real returns
5.1× to 7.9×
batched GARCH likelihood grids on Metal (256 parameter vectors × 888 series: 23 ms vs 117 ms); with a single parameter vector per series the CPU wins everywhere
0.4–0.9×
rolling sum of squares on Metal relative to the CPU's O(1) running sum at every size: a published negative result
20 M bars/s
RAID Parquet-to-estimator pipeline on the CPU vs 14 M for Polars and DuckDB and 0.9 M for pandas; column projection mattered more than any kernel and the GPU plays no role
4.0× → 1.1×
fused realized speedup across eleven Apple Silicon machines, from the M2 to the M5 Max; all Metal tests pass on every machine and the GPU helps most where the CPU is weakest
Scientific contributions
- A precision strategy for FP32-only GPUs
- Every Metal accumulator is a compensated double-float pair built with the TwoSum error-free transformation, chunked partial sums (4,096 elements) are combined in FP64 on the host in a fixed order, precision-critical constants, solves and inference stay in FP64, and passes are fused; realized measures and regression statistics stay within 10⁻⁶ of FP64 and GARCH likelihoods within 2 × 10⁻⁵, with bitwise reproducibility.
- A method registry with reference validation
- Each Tier A method (returns, calendar-aware aggregation, RV/BPV/RSV/RQ/jumps/realized covariance, OLS/WLS/ridge with HC/HAC/cluster inference, HAR family, GARCH/GJR/EGARCH with Gaussian and Student-t QML, block bootstrap) pins its convention and is compared with highfrequency, rugarch, statsmodels, arch and exchange_calendars under identical conventions, with tolerances recorded and re-run by the test suite.
- A benchmark protocol that counts everything
- Randomised case order, warm-ups, repeated measurements, 2,000-draw bootstrap confidence intervals of medians, explicit cold and cached shader compilation, FP64→FP32 conversion and buffer traffic inside wall-clock, and manifests recording hardware, compiler, load and thermal state; a backend wins only when intervals do not overlap.
- Measured break-even answers, including the cases where the CPU wins
- Fused realized measures beat the CPU beyond about 4 × 10⁴ elements, batched GARCH likelihood grids gain 5–8×, regression statistics are mostly ties, rolling sums and single-vector recursions lose, small panels tie, and device time is 5–20 times smaller than wall-clock: unified memory removes the copy but not the conversion.
- An open, provenance-carrying library
- C++20 under the MIT licence with an FP64 CPU reference backend, an optional Metal backend with CPU fallback, nanobind Python bindings that release the GIL, a command-line tool, an Arrow/Parquet reader with column projection, an exact NYSE calendar and a FinData OS client whose every result records dataset version, request identifiers and input hashes.
- A sweep across eleven Apple Silicon generations
- The same self-contained bundle runs on eleven cluster machines from the M2 to the M6; the speedup pattern is governed by the CPU's multicore throughput and the CPU-bound conversion overhead rather than by GPU core count.
Methodology
Methodology (5)
Double-float Metal execution
Seven batched primitives over CSR batches (sums of squares, absolute adjacent products, semivariances, fourth powers, rolling sums, regression sufficient statistics, batched GARCH negative log-likelihood) implemented once in FP64 on the CPU and once in Metal with TwoSum accumulation, 256-thread tree reductions, fixed-order FP64 combination and fastMathEnabled = NO.
Reference validation and Monte Carlo inference
Reference comparisons at 10⁻¹² to 10⁻⁶ on synthetic fixtures; experiment E5 with R = 1,000 replications gives GARCH(1,1) biases of at most 0.005 at n = 2,000 with 0.93–0.97 coverage, HAR HAC coverage of 0.90–0.94 at h = 5, and 55 test cases pass on every build.
Microbenchmarks of the primitives
Series counts from 1 to 10,000 and lengths from 390 to 390,000 on the M5 Max laptop (18 CPU cores, 40 GPU cores), medians of five repetitions with bootstrap intervals, GPU device time read separately from wall-clock, shader compilation measured cold, warm and prebuilt.
End-to-end experiments on the real panel
E1 batched realized measures on 926 symbols with day, week, month and year segments; E2 causal rolling HAR-RV, log-HAR and HARQ forecasts on 100 symbols; E3 GARCH-family QML on 888 series with a 256-point likelihood grid; E4 realized covariance and rolling Diebold–Yilmaz connectedness for 10 to 100 assets; E6 the Parquet-to-estimator pipeline against pandas, Polars and DuckDB.
Generation sweep (E7)
A self-contained bundle built with -mcpu=apple-m1 run on eleven idle Apple Silicon machines (M2, M2 Max, M2 Ultra, M3, M3 Ultra, M4, M4 Pro, M4 Max, M5 Pro, M5 Max, M6) with three repetitions, both with the prebuilt metallib and with runtime compilation.
Research data
- FinData OS lakehouse: FirstRate Data 1-minute bars, 2024
- A seeded random sample of 1,000 symbols among the 6,046 US equities of the market_bars.stock.1m.raw dataset active through all of 2024, served by the author's private FinData OS lakehouse under a private-use licence; twenty asynchronous Parquet exports (43.3 million bars, 0.63 GB, dataset version 17 375) cached with SHA-256 and request identifiers; 888 symbols have at least 200 regular sessions with positive realized variance and 926 at least 200 sessions of returns. Raw bars are not redistributed.
- Daily bars 2015–2024
- Ten years of daily bars (1.82 million rows) for the same symbols, used for the GARCH-family panel of 888 series (median 2,459 training observations, last 40 sessions of 2024 held out).
- Synthetic fixtures and reference values
- Committed synthetic bars and Student-t(5) return panels, with reference outputs produced by statsmodels 0.15.0, arch 8.0.0, highfrequency 1.0.3, rugarch 1.5.6 and exchange_calendars 4.13.2, so that the validation suite runs without network access or vendor data.
Reproducibility
- The repository contains the library, kernels, bindings, CLI, tests, benchmark drivers and every raw benchmark record with its manifest (host, OS, compiler, library versions, load, thermal state, seeds, input hashes).
- No number in the text was typed by hand: every table and figure is generated from the raw records by experiments/scripts/make_paper_assets.py, and the prose quotes experiments/results/key_numbers.json; SHA-256 of each input and output is written to experiments/manifests/paper_assets.json.
- A clean checkout builds with CMake presets (macOS with Metal and Arrow; CPU-only for Linux) and passes 55 test cases; synthetic fixtures and reference values are committed, so validation needs no network access or vendor data.
- Reproducing E1–E6 requires a FinData OS API key; the universe selection is seeded and the dataset version pinned, so the same bars are served to a reader with access; E7 requires only the bundle and SSH access to the machines.
- RAID is released under the MIT licence; FirstRate Data bars are used under a private licence and only aggregates appear in the paper.
Research artifacts
Version history
| Version | Date | Description | File |
|---|---|---|---|
| v1.0current | November 2026 | Current manuscript as published on www.spboucher.ai (revised edition after the 2026-10-17 scientific review for papers 1–17). | PDF · 36 p. |