Architecture & Measurement-Design Analysis

GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System Architecture

Source: Qureshi, Z.; Mailthody, V. S.; Gelado, I.; Min, S.; Masood, A.; Park, J.; Xiong, J.; Newburn, C. J.; Vainbrand, D.; Chung, I.-H.; Garland, M.; Dally, W.; Hwu, W. Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '23), Vol. 2, March 25-29, 2023, Vancouver, BC, Canada, pp. 325-339. DOI: 10.1145/3575693.3575748 Code: Open-source hardware + software (publicly accessible per §1; IMPACT/UIUC + NVIDIA). Affiliations: NVIDIA, UIUC, AMD, University at Buffalo, IBM Research, Stanford. Reader: Direct PDF read via Claude Read tool with pages parameter (gemini-reader quota exhausted; codex-reader not attempted — full 12-page PDF fit in one read window). Analyst: Vishwakarma Date: 2026-06-15

Table of Contents

1. System Architecture (the "specimen": GPU-initiated storage stack)
2. System-Under-Test Architecture (BaM prototype testbed)
3. Design-Space Diagram (axes swept, axes held fixed)
4. Algorithm / Control Flow Diagrams (queue insertion, cache probe, warp coalescing)
5. Quantitative Results — Empirical Findings by Regime
6. Configuration-Regime Trade-off Tables
7. Bottlenecks & Insights Surfaced by the Measurements
8. Limitations of the Methodology
9. Analogy

1. System Architecture (the "specimen": GPU-initiated storage stack)

BaM ("Big accelerator Memory") is an accelerator-centric system architecture. Its thesis is a single inversion of control: instead of the CPU initiating and orchestrating every storage access (the "CPU-centric" baseline), the GPU thread itself initiates on-demand, fine-grained access to data wherever it lives — host memory or NVMe storage — without any CPU involvement on the critical path. The authors name this new family GPU Kernel Initiated (KI) storage access: GPUDirect Async KI Storage.

The architecture has three load-bearing components, all resident in GPU memory: (1) a scalable, highly concurrent software cache that coalesces redundant on-demand accesses and exposes locality; (2) a set of high-throughput I/O submission/completion (SQ/CQ) queues mapped into GPU memory, with the NVMe doorbell registers mapped into the GPU address space so GPU threads can ring them directly; and (3) the bam::array<T> abstraction — an array-like, mmap-style API that hides all of the above behind an overloaded subscript operator.

+-------------------------------------------------------------------+
|                    BaM Logical Architecture                       |
|                   (everything below lives in GPU memory)          |
|                                                                   |
|  +-----------------------+        +---------------------------+   |
|  | bam::array<T>         |  --->  | Software Cache            |   |
|  | (user-facing API,     |        | - Cache Metadata          |   |
|  |  overloaded [] op)    |        | - Cache-line data buffers |   |
|  | (role: mmap'd handle) |        | - clock replacement       |   |
|  +-----------+-----------+        | - ref-count + lock per CL |   |
|              |                    +-------------+-------------+   |
|         offset calc                              | MISS           |
|         + coalesce                               v                |
|              |                    +---------------------------+   |
|              |                    | High-Throughput I/O Queues|   |
|              +------------------> | - Submission Queues (SQ)  |   |
|                                   | - Completion Queues (CQ)  |   |
|                                   | - ticket counter          |   |
|                                   | - turn_counter[] array    |   |
|                                   | - mark bit-vector + lock  |   |
|                                   +-------------+-------------+   |
|                                                 | ring doorbell   |
+-------------------------------------------------|-----------------+
                                                  v
              ============= PCIe Gen4 x16 Interconnect =============
                                                  |
+-------------------------------------------------|-----------------+
|                       Storage (NVMe SSDs)        v                |
|  +-------------+   +-------------+   ...   +-------------+         |
|  | DB Reg #N   |   | DB Reg #N   |         | DB Reg #N   |         |
|  | + DMA engine|   | + DMA engine|         | + DMA engine|         |
|  | + CTRL      |   | + CTRL      |         | + CTRL      |         |
|  | + Media     |   | + Media     |         | + Media     |         |
|  +-------------+   +-------------+         +-------------+         |
|   (role: storage controller per SSD, posts CQ entries)           |
+-------------------------------------------------------------------+
^ Fig 1: Logical view of BaM (paper Figure 1). The control path
  (doorbell ring) and the data path (DMA over PCIe) are decoupled:
  GPU threads ring SSD doorbells directly; SSD DMA engines move
  bytes peer-to-peer into GPU memory. No CPU on either path.

The architecturally decisive choice is that the doorbell registers of the SSD controllers are memory-mapped into the GPU's address space (via GPUDirect Async, with the SSD BAR space mapped through the cudaHostRegister API). This is what lets a GPU thread initiate an I/O without a host round trip. Everything else in BaM — the cache, the queue protocol, the coalescing — exists to make this raw capability survive the GPU's thousand-fold-higher thread-level parallelism, which would otherwise serialize catastrophically on the shared doorbell and queue state.

The contrast with the CPU-centric baseline is the entire point. In the CPU-centric model, the CPU page-fault handler (or an explicit io_uring/read call) transfers data to GPU memory before a compute kernel can run, then launches the kernel, then synchronizes, repeatedly. This incurs CPU-GPU synchronization overhead, I/O traffic amplification (the CPU cannot know which bytes the data-dependent kernel will actually touch, so it over-fetches), and long CPU software latencies. BaM removes the CPU from the storage control path entirely.

2. System-Under-Test Architecture (BaM prototype testbed)

The prototype is built from off-the-shelf hardware: a single NVIDIA A100-80GB PCIe GPU, a Supermicro AS-4124GS-TNR server with 2x AMD EPYC 7702 64-core CPUs and 1 TB DDR4-3200, and — critically — a PCIe expansion chassis (H3 Platform Falcon-4016) that supplies the extra PCIe Gen4 x16 slots needed to attach many NVMe SSDs at full bandwidth. Software is Ubuntu 20.04 LTS, NVIDIA driver 470.82, CUDA 11.4, plus a custom Linux character-device driver that creates one device node per NVMe SSD and uses GPUDirect RDMA to pin and map the NVMe queues and I/O buffers into GPU memory.

+--------------- BaM Prototype Testbed (paper Table 1) -------------+
|                                                                   |
|   Supermicro AS-4124GS-TNR (4U)                                   |
|   +-----------------------------------------------------------+   |
|   |  2x AMD EPYC 7702 (64-core)   |   1 TB Micron DDR4-3200   |   |
|   +--------------------------+--------------------------------+   |
|                              |                                    |
|                       PCIe Gen4 x16                               |
|                              |                                    |
|   +--------------------------+--------------------------------+   |
|   |        NVIDIA A100-80GB PCIe GPU (the initiator)          |   |
|   +--------------------------+--------------------------------+   |
|                              |                                    |
|              =====  PCIe expansion fabric  =====                  |
|              H3 Platform Falcon-4016 chassis                      |
|              (two independent drawers, x16 each,                  |
|               PCIe bifurcation -> >16 M.2 SSDs/drawer)            |
|                              |                                    |
|   +--------------------------+--------------------------------+   |
|   |  Up to 10 Intel Optane P5800X (DC SSDs) OR                |   |
|   |  Samsung 980pro (consumer) OR Samsung Z-NAND PM1735       |   |
|   |  (P2P DMA target; doorbells mapped into GPU addr space)   |   |
|   +-----------------------------------------------------------+   |
+-------------------------------------------------------------------+

  SSD technology trade-off (paper Table 2):
  +------------+------------------+-----------+------+--------+-------+
  | Technology | RD IOPs(512B,4KB)| Latency us| DWPD | $/GB   | Gain  |
  +------------+------------------+-----------+------+--------+-------+
  | DRAM DDR4  | >10M             | O(0.1)    |>1000 | 11.13  | 1.0x  |
  | Optane     | 5.1M / 1.5M      | O(10)     | 100  | 2.54   | 4.4x  |
  | Z-NAND     | 1.1M / 1.6M      | O(25)     | 3    | 2.64   | 4.3x  |
  | NAND Flash | 700-800K         | O(100)    | 0.3  | 0.51   | 21.8x |
  +------------+------------------+-----------+------+--------+-------+
^ Fig 2: System-under-test. The expansion chassis is not incidental —
  no single NVMe SSD can match a PCIe x16 Gen4 link, so BaM must
  *scale out* SSDs to saturate the GPU's PCIe bandwidth. The $/GB
  "Gain" column is the cost argument: 4.3x-21.8x cheaper per GB than
  a DRAM-only solution, even after paying for chassis + risers.

The testbed embodies the paper's economic thesis. The GPU's PCIe Gen4 x16 link delivers ~26 GB/s; the authors quote 26 GB/s / 512 B = 51 M accesses/sec and 26 GB/s / 4 KB = 6.35 M accesses/sec as the access- rate ceilings the storage tier must feed. A single Optane SSD tops out near 5.1 M read IOPs, so reaching the PCIe ceiling requires striping across ~10 SSDs — which is exactly why the expansion chassis is a first-class component, not an accessory.

Little's Law is the sizing tool the authors apply explicitly: T x L = Q_d, where T is target throughput, L is device latency, and Q_d is the minimal queue depth to sustain T. For Optane (L = 11 us), sustaining 51 M accesses of 512 B needs Q_d = 51M x 11us = 561 in-flight requests; for Samsung 980pro (L = 324 us), the same target needs Q_d = 16,524. That second number — tens of thousands of concurrent in-flight requests — is the structural justification for the entire high-throughput-queue design: only the GPU's massive thread-level parallelism can keep that many requests in flight.

  Software / hardware stack of the SUT:
  +------------------------------------------------+
  | Application GPU kernel (bam::array<T> access)  |  user code
  +------------------------------------------------+
  | bam::array<T>  high-level abstraction          |  API layer
  +------------------------------------------------+
  | BaM Software Cache (coalesce + locality)       |  caching layer
  +------------------------------------------------+
  | BaM I/O Stack (SQ/CQ queue protocol)           |  protocol layer
  +------------------------------------------------+
  | Custom Linux char driver (1 node / SSD)        |  setup-time only
  | + GPUDirect RDMA / GPUDirect Async             |  (not on hot path)
  +------------------------------------------------+
  | NVMe over PCIe Gen4 x16 (P2P DMA)              |  transport
  +------------------------------------------------+
  | Intel Optane / Samsung Z-NAND / NAND SSDs      |  media
  +------------------------------------------------+
^ Fig 3: SUT layered stack. The custom driver participates only at
  initialization (pin + map queues/doorbells); once mapped, the GPU
  bypasses it entirely. This is the "setup in kernel space, steady
  state in user space" split that gives BaM its throughput.

3. Design-Space Diagram (axes swept, axes held fixed)

BaM's evaluation is a multi-axis sweep over storage configuration, workload class, cache geometry, and baseline. Unlike a survey paper, the swept axes here are properties of one system (BaM) measured against fixed external baselines (GDS, ActivePointers, RAPIDS, host-memory Target).

                  DESIGN SPACE (swept + held-fixed)
  +---------------------------------------------------------------+
  |                                                               |
  |  Axis 1: NUMBER OF SSDs (raw-throughput + app scaling)         |
  |    [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]                    |
  |    (round-robin striping; replication for app workloads)      |
  |                                                               |
  |  Axis 2: I/O ACCESS GRANULARITY / cache-line size             |
  |    [512 B] [4 KB] [8 KB]  (raw); also vs GDS 4KB..1MB          |
  |                                                               |
  |  Axis 3: WORKLOAD CLASS                                        |
  |    [Raw IOPs microbench] (512B/4KB random read+write)         |
  |    [Graph analytics: BFS, CC]  (5 graphs, Table 3)            |
  |    [Data analytics: 6 NYC-Taxi queries Q0..Q5 vs RAPIDS]      |
  |    [vectorAdd: write-intensive, 4B-element x 4 billion]       |
  |                                                               |
  |  Axis 4: CACHE GEOMETRY                                        |
  |    cache size [1,2,4,8,16,32,64 GB]                           |
  |    cache mode [No cache | Naive cache | Optimized (coalesce + |
  |                reference reuse)]                              |
  |                                                               |
  |  Axis 5: NVMe QUEUE-PAIR COUNT (SQ/CQ depth = 1024)           |
  |    [128] [96] [80] [64] [48] [40] [32]  queue pairs           |
  |                                                               |
  |  Axis 6: BASELINE / COMPARATOR                                |
  |    [NVIDIA GDS] [ActivePointers+GPUfs] [host-memory Target T] |
  |    [RAPIDS (CPU-mem preloaded)] [DRAM-only solution]          |
  |                                                               |
  |  Held FIXED (no sweep, unless noted):                         |
  |    - GPU: single NVIDIA A100-80GB PCIe                        |
  |    - Interconnect: PCIe Gen4 x16 (~26 GB/s ceiling)          |
  |    - Default SSD: Intel Optane P5800X, 128 QP @ 1024 depth   |
  |    - Default graph cache: 8 GB, 4 KB cache-line              |
  |    - Bare-metal, direct NVMe access (no virtualization)      |
  |    - bam::array default template params (no specialization)  |
  +---------------------------------------------------------------+
^ Fig 4: 6-axis design space. The most important "held fixed" lines
  are the single A100 + single PCIe x16 link: BaM's whole story is
  saturating ONE PCIe link from MANY SSDs. The cache-mode axis
  (No / Naive / Optimized) is where BaM's software contribution is
  isolated from the raw hardware capability.

Two scoping choices define the measurement. First, the cache-mode axis (No cache / Naive cache / Optimized) isolates the software contribution from the raw hardware: "No cache" measures the bare GPUDirect-Async capability, "Naive cache" adds the coalescing cache, and "Optimized" adds warp coalescing + cache-line reference reuse. The gap between these three columns is BaM's software value, separated from the SSD's raw IOPs. Second, almost everything runs against a single A100 over a single PCIe x16 link — the SSDs scale out, but the GPU does not — so the experiment is deliberately bottlenecked at the PCIe ceiling, which makes "did we saturate PCIe?" the central success criterion.

4. Algorithm / Control Flow Diagrams

4.1 Life of a GPU thread accessing data (the master control flow)

This is the end-to-end path from data[tid] to bytes-in-GPU-memory, spanning the BaM lookup, the I/O stack, and the storage controller (paper Figure 2).

  START: GPU thread executes  val = data[tid]   (bam::array)
       |
       v
  (1) Offset calc: compute cache-line for this element
       |
       v
  (2) Coalescer: threads in a warp accessing the SAME cache line
       elect ONE leader to probe metadata (warp coalescing)
       |
       v
  (3) Cache lookup ----> read(off, tid)
       |
       +-- HIT  --> atomically inc cache-line ref-count -->
       |            directly read data in GPU memory --> [done]
       |
       +-- MISS --> lock cache line, find victim (clock replace),
                    enter the BaM I/O stack:
            |
            v
  (4) Prepare CMD            (build NVMe read command)
       |
       v
  (5) Submit CMD to SQ       (ticket counter -> entry; turn ordering;
       |                      copy command into physical SQ slot)
       v
  (6) Ring doorbell (sq_db)  (coalesced: move_tail does ONE doorbell
       |                      write for many threads' commands)
       v
  ===== Storage Controller side =====
  (a) Wait for doorbell
  (b) Read SQ entry from GPU
  (c) Process CMD
  (d) DMA write to GPU I/O buffer   (P2P, bypasses CPU)
  (e) Write CQ entry in GPU mem
  ===================================
       |
       v
  (7) Poll for completion    (no lock; scan CQ for this request)
       |
       v
  (8) Update cache state     (mark line valid, store data)
       |
       v
  (9) Update SQ/CQ head      (winner thread rings CQ head doorbell,
       |                      advances heads on behalf of coalesced peers)
       v
  Access fetched data in GPU memory --> [done]
^ Fig 5: Life of a GPU thread in BaM (paper Figure 2). Steps (4)-(9)
  only execute on a MISS. The whole design effort is to make steps
  (5), (6), and (9) survive thousands of concurrent threads with
  minimal critical sections.

The design intent is visible in which steps hold locks. The cache probe (3) and completion poll (7) are lock-free — only a miss that must insert/evict (4) or a doorbell ring (6) and head advance (9) touch the small critical sections. This is the same "minimize the critical section, maximize lock-free fast paths" discipline that lets the GPU's thread-level parallelism actually translate into IOPs rather than collapsing onto a serialized doorbell.

4.2 Lock-free queue insertion (the ticket / turn protocol)

The SQ insertion protocol is the cleverest piece of the I/O stack. It turns a naive "lock-enqueue-ring-doorbell" critical section (which would serialize thousands of threads) into a mostly-lock-free ticketed protocol with a single coalesced doorbell write.

  Per-SQ metadata in GPU memory:
    head, tail   |  ticket counter  |  turn_counter[]  |  mark[]  |  lock

  Thread wants to enqueue a command:
       |
       v
  (1) atomically  ticket += 2     -> returns a global index into a
       |                              VIRTUAL queue of 2^32 entries
       v
  (2) entry = ticket % phys_size  (which physical slot)
      turn  = ticket / phys_size  (which "pass" over that slot)
       |
       v
  (3) spin until turn_counter[entry] == my turn
       |   (turn_counter creates per-slot SUB-QUEUES ordered by turn;
       |    waiting threads on the same slot are FIFO-ordered)
       v
  (4) slot is now mine: copy I/O command into SQ[entry]
       set mark[entry] = 1
       |
       v
  (5) call move_tail:
        - ONE thread wins the lock
        - advances tail past consecutive marked entries
        - reset_marks for those entries
        - rings the SQ doorbell ONCE (coalesces many writes)
        - releases lock
      (losing threads return immediately if their mark was cleared;
       else they retry to take the lock and push the tail further)
^ Fig 6: Lock-free-ish SQ insertion. The doorbell — the expensive PCIe
  write — is amortized across all threads whose commands the winner's
  move_tail covered. CQ dequeue is symmetric (winner advances CQ head
  and signals forward progress via new-head in the CQ entry).

The turn_counter[] array is the subtle invention: it maps a 2^32-entry virtual queue onto a finite physical queue, with each physical slot owning a FIFO of waiting threads ordered by turn. This gives correctness (no two threads write the same slot concurrently) without a global lock on the common path — the lock is only taken in move_tail, and only by one winning thread per batch.

4.3 Cache probe + clock eviction (concurrency-safe)

  Thread probes cache line for an offset:
       |
       v
  state == VALID? --yes--> atomically inc ref_count --> use data
       |                   (when done: dec ref_count)
       no
       v
  lock cache line --> find victim via CLOCK replacement:
       |
       v
  global counter++ -> assigns a candidate cache SLOT
       |
       +-- slot maps to line with ref_count != 0 (pinned)?
       |      yes --> counter++ again, try next slot (skip pinned)
       |       no  --> mark old line INVALID, remap slot to new line
       v
  request new line from backing memory (SSD or host)
       |
       v
  on completion: unlock, set VALID, inc ref_count
^ Fig 7: Concurrency-safe clock replacement. The global atomic counter
  lets many concurrent evictions pick DIFFERENT slots in parallel;
  pinned (non-zero ref-count) lines are skipped, never evicted
  mid-use. This is what lets the cache "support many more concurrent
  accesses" than an OS-kernel allocator's coarse critical sections.

4.4 Warp coalescing (__match_any_sync)

The final mechanism collapses redundant cache probes within a warp. Prior work (GPUfs/ActivePointers) serialized unique probes per warp, costing many instructions; BaM uses the __match_any_sync warp primitive to partition the warp's 32 threads into groups by cache line, elects a leader per group via __shfl_sync, and lets each group's leader probe in parallel with no inter-group dependency.

  32 threads in a warp, each with a target offset:
       |
       v
  __match_any_sync(mask, cache_line_id)
       -> partitions warp into GROUPS sharing a cache line
       |
       v
  per group: elect leader (lowest lane)         [parallel across groups]
       |
       v
  leader probes cache for the group's line       [no inter-group dep]
       |
       v
  __shfl_sync broadcasts the GPU-memory address to group members
       |
       v
  all members read their element from the shared cache line
^ Fig 8: Warp coalescing. vs prior serialized-probe designs, BaM's
  groups probe concurrently. This is the micro-architectural reason
  the "Optimized" cache mode beats the "Naive" cache mode in Fig 8
  of the paper (6.07x BFS / 11.24x CC additional speedup).

5. Quantitative Results — Empirical Findings by Regime

5.1 Headline end-to-end results (abstract)

The framing is two-pronged: BaM is on-par-or-better in performance and dramatically cheaper, because it replaces expensive DRAM with cheap SSDs while matching throughput.

5.2 Raw I/O throughput (microbenchmark, §4.3, Figure 4)

BaM's I/O stack reaches 45.8 M read IOPs for 512 B accesses across 10 SSDs and scales linearly with additional SSDs. 22.9 GBps is 90% of the measured Gen4 x16 peak — i.e., BaM's software is not the bottleneck; PCIe is. Write bandwidth has further headroom (5.3 GBps is not yet PCIe-limited; more SSDs would help).

5.3 vs NVIDIA GDS — the small-granularity cliff (§5.1, Figure 5)

GDS cannot saturate PCIe below 32 KB because of the traditional CPU software stack overhead; GDS only reaches 23.6% of bandwidth at 4 KB. BaM saturates the interface at even 4 KB (~25 GBps). This is the cleanest demonstration that the CPU software path — not the hardware — is what kills fine-grained access.

5.4 vs ActivePointers (§5.1, Figure 6)

BaM beats ActivePointers by >1 order of magnitude in both cache-miss handling and hit-data delivery. ActivePointers is hindered by the GPUfs mechanism (GPU threads must request transfers from the CPU); BaM's GPU-initiated path removes that handshake.

5.5 Graph analytics (§5.2, Figures 7-9)

Scaling 1 -> 4 SSDs improves BaM's aggregate bandwidth and pushes it to PCIe-level bandwidth (3.48x for BFS, 4x for CC). With 4 SSDs, BaM matches or beats a host-memory DRAM-only solution that costs up to 21.7x more. The Uk graph is the hard case for BFS (deep traversal, small frontiers -> insufficient concurrent I/O to hide latency).

Cache-mode decomposition (Figure 8):

The naive cache alone gives ~12x (it minimizes I/O amplification); warp coalescing + reference reuse adds another 6-11x on top.

SSD-type sensitivity (Figure 9): with consumer Samsung 980pro SSDs, BaM is on average 3.21x (BFS) / 2.68x (CC) slower than with Optane — yet 980pro is "by far the lowest cost," so the trade is explicit. Datacenter Samsung DC PM1735 (S1735) performs similarly to Optane (both hit similar 4 KB read IOPs).

5.6 Data analytics vs RAPIDS (§5.3, Figure 12)

The gain source is reduced I/O amplification (on-demand fetch of only the touched columns) plus reduced GPU-memory-management overhead (RAPIDS pays CPU software overhead even with data preloaded). The advantage grows as queries become more data-dependent, because RAPIDS must move whole columns while BaM fetches on demand.

5.7 Cache-size and queue-pair sensitivity (§5.3, Figures 10-11)

The 1 GB cache matching 8 GB performance is the locality argument: the working set fits, so a tiny cache suffices. The 40-queue-pair knee is the structural floor — below it, NVMe protocol serialization per queue starts to bite.

5.8 Cache-overhead and floor analysis (§5.2)

With more SSDs the cache overhead share grows (metadata contention, long-latency atomics, warp scheduling under polling) — but BaM still beats the most-optimistic baseline. The floor is the storage I/O throughput (5-6.2 M IOPs, >80% of peak storage throughput); past this, more SSDs do not help and only application modification can.

5.9 vectorAdd (write-intensive, §5.4)

BaM is 1.51x slower than the proactive-tiling baseline on this write-intensive workload, because BaM does not yet overlap read-miss handling with write-back — it exposes the full write latency. This is an honest negative result with a named fix (asynchronous write-back, left as future work).

5.10 Register / SM utilization (§5.5, Figure 13)

BaM raises per-thread register usage substantially (up to 255 for RAPIDS Q5), causing register spilling in RAPIDS — but because all workloads are storage-I/O-bound, the reduced occupancy does not bottleneck performance. The hardware resource that matters is I/O, not registers.

6. Configuration-Regime Trade-off Tables

6.1 GPU-initiated (BaM) vs CPU-centric baseline

For an accelerator-centric stack, prefer GPU-initiated access for any data-dependent, fine-grained, read-dominated workload; the only regime where the CPU-centric tiling baseline still wins is pure write-intensive workloads, until asynchronous write-back lands.

6.2 Storage media regime (Table 2 + Figure 9)

For performance, prefer Optane (lowest latency -> smallest queue depth via Little's Law). For cost, prefer NAND — BaM still functions, just slower, and the 21.8x cost advantage may dominate the decision for capacity-bound deployments.

6.3 Cache-mode regime (Figure 8)

Prefer the Optimized cache mode whenever the application has warp- level locality to exploit — the bulk of BaM's software value lives in the gap between Naive and Optimized, and it requires the application to expose coalescable access patterns.

6.4 Scale-out regime (number of SSDs)

Prefer enough SSDs to saturate PCIe (≈4 for graph analytics, ≈10 for raw 512 B IOPs). Beyond the PCIe ceiling, additional SSDs are wasted; below it, BaM is bandwidth-starved and loses to host memory.

7. Bottlenecks & Insights Surfaced by the Measurements

7.1 The bottleneck is PCIe, not BaM's software

22.9 GBps at 45.8 M IOPs is 90% of measured Gen4 x16 peak — BaM's queue protocol and cache are not the limiter once enough SSDs are attached. This is the single most important measurement: it certifies that the software stack scales to the hardware ceiling, which is the whole claim of the paper. The corollary is that BaM's value will grow on PCIe Gen5 / NVLink-attached storage, where the ceiling is higher.

7.2 The CPU software stack is the real enemy of fine-grained I/O

GDS at 4 KB reaches only 23.6% of PCIe bandwidth; BaM reaches ~100%. The delta is entirely CPU software overhead (OS page-fault handler, kernel crossing, driver). The insight generalizes: as device latency drops (Optane, Z-NAND), software overhead becomes a larger fraction of total I/O time — the authors cite up to 36.4% for an optimized io_uring CPU stack. Removing the CPU from the path is therefore the right lever precisely because storage got fast.

7.3 Little's Law sets the queue-depth requirement, and only the GPU

can meet it

Sustaining peak throughput on high-latency consumer SSDs needs ~16,524 concurrent in-flight requests. No CPU can keep that many requests in flight; the GPU's thousand-fold thread-level parallelism is the only hardware that can. The GPU is not just a faster initiator — it is the only initiator that can hide modern SSD latency at peak throughput. This reframes BaM from "convenience" to "necessity."

7.4 Critical-section minimization is what unlocks the parallelism

The naive design (lock + enqueue + doorbell ring) would serialize thousands of threads on the doorbell write. BaM's ticket/turn protocol

• coalesced move_tail doorbell + clock-replacement-with-skip + warp coalescing all exist to shrink critical sections to near zero. The 45.8 M IOPs result is the proof that these mechanisms work; remove any one and the doorbell or the cache lock becomes the bottleneck.

7.5 I/O amplification is the data-analytics win, not raw bandwidth

Against RAPIDS, the 1.22x-to-5.3x advantage comes from fetching only the touched columns on demand rather than moving whole columns. The advantage grows with data-dependence (Q0 -> Q5) because that is exactly when over-fetching hurts most. The lesson: for irregular, data-dependent workloads, the right metric is bytes-actually-needed / bytes-moved, and on-demand fine-grained access optimizes it directly.

7.6 The honest negative: write-back latency is unhidden

vectorAdd's 1.51x slowdown is the one place BaM loses, and the authors name the cause precisely: no overlap between read-miss handling and write-back, so the full write latency is exposed. This is a clean, falsifiable bottleneck with a named remedy (async write-back), and its inclusion strengthens the paper's credibility.

7.7 Locality means a tiny cache suffices

A 1 GB cache matches an 8 GB cache on the K dataset. The working set's spatial/temporal locality is captured at 1 GB; the rest is cold misses that no cache size fixes. This argues against over-provisioning scarce GPU HBM for caching — small caches plus on-demand fetch beat large caches plus preloading.

8. Limitations of the Methodology

The most consequential scoping limit is the single-GPU, single-PCIe- link setup: every result is a story about saturating one link from many SSDs. The multi-GPU regime — where storage bandwidth must be shared or partitioned across GPUs, and where NVLink could change the ceiling entirely — is entirely future work. The second is the write-back gap, which is the one regime BaM loses and which the authors correctly flag rather than hide.

9. Analogy

BaM is a just-in-time warehouse-to-workshop delivery system that lets each individual worker order parts directly, instead of routing every order through a single dispatch manager.

In the CPU-centric world (GDS, page-fault tiling), there is one dispatch manager (the CPU). Every time a worker (GPU thread) needs a part (data), the worker stops, raises a hand, and waits for the manager to walk to the warehouse, guess which crate of parts might be needed, haul the whole crate back (I/O amplification — over-fetching), and hand it over. With thousands of workers, the manager is mobbed; the factory floor idles waiting on one overworked dispatcher. Worse, the manager cannot read the workers' minds, so the crates are full of parts nobody asked for.

BaM rips out the dispatcher. Now each worker holds a direct order terminal wired straight to the warehouse loading dock (the SSD doorbell mapped into GPU address space). A worker who needs a single bolt orders exactly that bolt (fine-grained, on-demand), and a fleet of forklifts (SSD DMA engines) deliver it straight to that worker's bench (P2P DMA into GPU memory) — never passing through the manager's office.

But a thousand workers all slamming order buttons at the same dock would jam the loading-dock paperwork (the doorbell write). So BaM adds a deli-counter ticket machine (the ticket/turn protocol): you grab a numbered ticket, and when your number is called you drop your order in the bin — and one runner (the move_tail winner) carries a whole batch of orders to the dock in a single trip, so the dock is rung once, not a thousand times. Next to the dock sits a shared parts shelf (the software cache): if the bolt you need is already on the shelf because a neighbor just ordered it, you take it off the shelf instantly (cache hit, lock-free) instead of bothering the warehouse at all — and workers reaching for the same shelf box elect one person to fetch for the group (warp coalescing).

The economics close the analogy: the warehouse (cheap NVMe SSDs) is far cheaper per part than keeping every part on the expensive workbench itself (DRAM) — 4x-22x cheaper per part. As long as enough forklifts (SSDs) feed the single delivery corridor (PCIe x16) fast enough to keep it full, the workshop runs as fast as if every part were already on the bench, at a fraction of the cost. The one thing the new system has not yet solved is shipping finished parts back out (write-back) while still receiving new ones — for now, outbound and inbound share the corridor serially, which is the single case where the old crate-hauling manager was actually faster.