[Hotstorage-chairs] [HotStorage 2026] Rejected, but invited as poster, submission #35 "Regular: Exploring High-Bandwidth Flash..."

[HotStorage 2026 HotCRP]

Dear authors,

The program committee for the 2026 ACM Workshop on Hot Topics in Storage 
and File Systems (HotStorage 2026) regrets to inform you that your 
submission 35 was rejected to appear in the proceedings of the workshop.

However, your work incited exciting discussion among the program 
committee members, and we would like to invite you to present your work 
during the poster session. Please let us know by August 12, 2026 
(Wednesday) whether or not you accept or decline the poster invitation.

* Title: Regular: Exploring High-Bandwidth Flash in GPU Memory Systems 
  with HBFSim
* Site: https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fhotstorage26.hotcrp.com%2fpaper%2f35&c=E,1,0puZjGP3Ai5xuZZhooGGXt9ioBpPW44lPV1R_Q02Y9MGvgLyFOl-ILCwHin_CamASQsjJYQ3HHn2aTRrKwZHZCNH-fT0Iq-SYUmVmNrRtVOH&typo=1

23 of 78 submissions were accepted.

Visit the submission site for reviews, comments, and related 
information. Reviews and comments are also included below.

Contact chairs26 at hotstorage.org with any questions or concerns.

Sincerely,
Young-ri Choi and Bryan S. Kim
HotStorage 2026 Program Co-Chairs

Review #35A
===========================================================================

Overall merit
-------------
2. Weak reject

Reviewer expertise
------------------
2. Some familiarity

Paper summary
-------------
The paper proposes a event-driven simulator for HBM+HBF memory systems with awareness of application-level memory semantics. The simulator consists of a workload module, a policy module, a device module and an attribution module. 

The simulator addresses a very timely gap in existing simulators, that spans the HBM/HBF boundary with workload semantics and placement policies. Overall, the paper is well written.

Calibration is performed against Ramulator and MQSim. They studied 9 placement policies under 8 concurrent LLM session serving scenarios. The work cover two Llama3.1, DeepSeek V4, and Qwen3 models.

However, the calibration is only against two conventional DRAM and NAND simulators which lack the application semantics (the main novelty in this work), no validation against running LLM serving workloads on real GPU is performed. Without such validation, the paper should consider narrowing its quantitative claims or clearly brand them as illustrative.


* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


Review #35B
===========================================================================

Overall merit
-------------
3. Weak accept

Reviewer expertise
------------------
2. Some familiarity

Paper summary
-------------
This paper presents HBFSim, an event-driven simulator for HBM+HBF memory systems in LLM serving. It models flat and tiered organizations while preserving LLM data-class semantics. A study across nine configurations and three workloads shows that semantic placement matters more than memory organization, and that uncontrolled write traffic to flash is the dominant failure mode.

Comments for authors
--------------------
The paper fills a real gap: HBF is pre-silicon, no existing simulator spans the HBM/HBF boundary with LLM-aware semantics. The simulator engineering is careful. Conservation identities run on every simulation as enforcement, not just logging, and they caught a real bug (a free-list leak that inflated throughput by 8%). The calibration is done transparently, with each residual traced to a known modeling difference and every device parameter labeled by provenance. The experimental structure is clean: Table 3 lets the reader trace each throughput delta back to the counter that moved, so the core claim is verifiable from the reported data.

A few concerns:

- The MQSim calibration has two items diverging by 1.5x and 2.1x IOPS, marked "by design." These reflect HBF assumptions with no silicon to validate against. A sensitivity analysis around plane parallelism and striping bandwidth would help show whether the main conclusions hold if these assumptions are off.

- The 1.7x GEMM re-read factor is treated as an optional overlay. For weight-dominated workloads like BurstGPT, it would be useful to see whether enabling it changes the relative ranking of placement strategies.

- Section 5 discusses training at length but no training workload is evaluated. Even one configuration would help back the claim that the trace format supports training traffic.

- The artifact link in the abstract returns 404. This may be due to anonymization, but I was unable to verify.

Grammar:

- Section 5: "The three questions of Section 4 converges on one answer." Should be "converge."
- Section 5: "A cache and an explicit placement behaves almost identically." Should be "behave."
- Section 3.2: "swapable" should be "swappable."


* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


Review #35C
===========================================================================

Overall merit
-------------
3. Weak accept

Reviewer expertise
------------------
2. Some familiarity

Paper summary
-------------
This paper presents HBFSim, an event-driven simulator for exploring heterogeneous HBM + High Bandwidth Flash (HBF) memory hierarchies for LLM serving. HBFSim preserves the semantic identity of weights, KV cache, activations, and workspace traffic while modeling flash-specific effects such as garbage collection, write amplification, queueing, SLC caching, and endurance. The paper evaluates nine flat-memory and HBM-as-cache configurations across three workload regimes. Its main finding is that semantic placement matters more than the memory organization itself: dominant read-heavy data should be placed in HBF, while transient and write-heavy data should remain in HBM.

Comments for authors
--------------------
I find the topic timely and well suited to HotStorage. The result, which shows that semantic placement can remain close to all-HBM performance while supporting workloads that exceed HBM capacity, is an important information to practical long-context LLM workloads.

My main concern is that the conclusions depend heavily on an assumed pre-silicon HBF model (Section 3.3). A brief clarification of which assumptions most strongly affect the main conclusions would be useful. Moreover, some conclusions, such as placing read-heavy objects in HBF and avoiding write-heavy transient data, are relatively intuitive. The paper would be stronger if the simulator revealed additional non-obvious insights that remain robust across these assumptions.


* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


Review #35D
===========================================================================

Overall merit
-------------
1. Reject

Reviewer expertise
------------------
3. Knowledgeable

Paper summary
-------------
This paper introduces HBFSim, an event-driven simulator for evaluating High Bandwidth Flash (HBF) in HBM+HBF memory systems for LLM serving. HBF offers large capacity at near-HBM bandwidth but introduces flash behaviors (GC, write amplification). Existing tools cannot model both flash internals and LLM semantics (weights, KV cache, activations). Using HBFSim, the authors evaluate 9 memory organizations and placement policies and find that HBF primarily alleviates capacity pressure rather than bandwidth bottlenecks; Workload-aware semantic placement matters more than flat vs. hierarchical organization; No single policy is best; optimal placement depends on the dominant data stream (weights vs. KV).

Comments for authors
--------------------
Thank you for submitting the paper to HotStorage! I like reading the paper; however, I have trouble fully understanding it. On the high level, it is very clear, but the details are very muddy, and I do not find the takeaways clear. For example, the authors compared different policies of data placement, but I am not able to understand the conclusion. 

The weight sizes shown in Figure 1 appear incorrect. Are these numbers based on FP4 quantization? Even under FP8, the reported sizes look too small relative to publicly available model configurations. Could the authors clarify the precision and calculation method used?

While the paper is generally well-written, several phrases reduce readability. In particular, terms such as “clean-room 70B/128K” are unclear without additional explanation. More precise and accessible wording would improve the overall clarity of the paper. 

I have no idea what Sec3.3 is talking about, which seems to be the only text on how the simulator handles HBF internals such as garbage collection and wear leveling. 

It is also unclear to me where the numbers (e.g., bandwidth) that the authors use come from. 

The anonymized repo does not exist.


Comment @A1 by Reviewer A
---------------------------------------------------------------------------
Dear Authors, thank you for submitting your work to HotStorage 2026. Your paper has been discussed by the PC members. Reviewers agree that the paper has chosen an interesting research topic. However, the evaluation part is not yet ready for publication in its current state. It has therefore been rejected.