[Hotstorage-chairs] [HotStorage 2026] Rejected, but invited as poster, submission #35 "Regular: Exploring High-Bandwidth Flash..."

[Jinheng Li]

Dear Young-ri and Bryan,

Thanks for the update on submission #35. We're sorry the paper didn't make
the proceedings, but glad it sparked some discussion — and yes, we'd be
happy to accept the poster invitation.

Two quick questions before we start putting things together:

1. Will the poster end up in the ACM Digital Library, or is the poster
session non-archival?

2. Once we send in the poster materials, will we get a separate
confirmation email? Such a confirmation would be helpful for our
institutional travel approvals.

Thanks again for the reviewers' feedback — we're looking forward to the
workshop.

Best,
Jinheng Li
MBZUAI


On Sat, Jul 25, 2026 at 11:03 AM HotStorage 2026 HotCRP <
noreply-hotstorage26 at hotcrp.com> wrote:

> Dear authors,
>
> The program committee for the 2026 ACM Workshop on Hot Topics in Storage
> and File Systems (HotStorage 2026) regrets to inform you that your
> submission 35 was rejected to appear in the proceedings of the workshop.
>
> However, your work incited exciting discussion among the program
> committee members, and we would like to invite you to present your work
> during the poster session. Please let us know by August 12, 2026
> (Wednesday) whether or not you accept or decline the poster invitation.
>
> * Title: Regular: Exploring High-Bandwidth Flash in GPU Memory Systems
>   with HBFSim
> * Site: https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fhotstorage26.hotcrp.com%2fpaper%2f35&c=E,1,nb_4PPeBnZU4IwUzouOTx3Of3SR5bmrxJQkBZ7-BsC25QloDTSC66XAPjUYS511rNJRRldRyKF-_BcdsfRL03NUAIQ0rANj9_sAd9djXHO-Yocx4SXZ0BhTWag,,&typo=1
>
> 23 of 78 submissions were accepted.
>
> Visit the submission site for reviews, comments, and related
> information. Reviews and comments are also included below.
>
> Contact chairs26 at hotstorage.org with any questions or concerns.
>
> Sincerely,
> Young-ri Choi and Bryan S. Kim
> HotStorage 2026 Program Co-Chairs
>
> Review #35A
> ===========================================================================
>
> Overall merit
> -------------
> 2. Weak reject
>
> Reviewer expertise
> ------------------
> 2. Some familiarity
>
> Paper summary
> -------------
> The paper proposes a event-driven simulator for HBM+HBF memory systems
> with awareness of application-level memory semantics. The simulator
> consists of a workload module, a policy module, a device module and an
> attribution module.
>
> The simulator addresses a very timely gap in existing simulators, that
> spans the HBM/HBF boundary with workload semantics and placement policies.
> Overall, the paper is well written.
>
> Calibration is performed against Ramulator and MQSim. They studied 9
> placement policies under 8 concurrent LLM session serving scenarios. The
> work cover two Llama3.1, DeepSeek V4, and Qwen3 models.
>
> However, the calibration is only against two conventional DRAM and NAND
> simulators which lack the application semantics (the main novelty in this
> work), no validation against running LLM serving workloads on real GPU is
> performed. Without such validation, the paper should consider narrowing its
> quantitative claims or clearly brand them as illustrative.
>
>
> * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
>
>
> Review #35B
> ===========================================================================
>
> Overall merit
> -------------
> 3. Weak accept
>
> Reviewer expertise
> ------------------
> 2. Some familiarity
>
> Paper summary
> -------------
> This paper presents HBFSim, an event-driven simulator for HBM+HBF memory
> systems in LLM serving. It models flat and tiered organizations while
> preserving LLM data-class semantics. A study across nine configurations and
> three workloads shows that semantic placement matters more than memory
> organization, and that uncontrolled write traffic to flash is the dominant
> failure mode.
>
> Comments for authors
> --------------------
> The paper fills a real gap: HBF is pre-silicon, no existing simulator
> spans the HBM/HBF boundary with LLM-aware semantics. The simulator
> engineering is careful. Conservation identities run on every simulation as
> enforcement, not just logging, and they caught a real bug (a free-list leak
> that inflated throughput by 8%). The calibration is done transparently,
> with each residual traced to a known modeling difference and every device
> parameter labeled by provenance. The experimental structure is clean: Table
> 3 lets the reader trace each throughput delta back to the counter that
> moved, so the core claim is verifiable from the reported data.
>
> A few concerns:
>
> - The MQSim calibration has two items diverging by 1.5x and 2.1x IOPS,
> marked "by design." These reflect HBF assumptions with no silicon to
> validate against. A sensitivity analysis around plane parallelism and
> striping bandwidth would help show whether the main conclusions hold if
> these assumptions are off.
>
> - The 1.7x GEMM re-read factor is treated as an optional overlay. For
> weight-dominated workloads like BurstGPT, it would be useful to see whether
> enabling it changes the relative ranking of placement strategies.
>
> - Section 5 discusses training at length but no training workload is
> evaluated. Even one configuration would help back the claim that the trace
> format supports training traffic.
>
> - The artifact link in the abstract returns 404. This may be due to
> anonymization, but I was unable to verify.
>
> Grammar:
>
> - Section 5: "The three questions of Section 4 converges on one answer."
> Should be "converge."
> - Section 5: "A cache and an explicit placement behaves almost
> identically." Should be "behave."
> - Section 3.2: "swapable" should be "swappable."
>
>
> * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
>
>
> Review #35C
> ===========================================================================
>
> Overall merit
> -------------
> 3. Weak accept
>
> Reviewer expertise
> ------------------
> 2. Some familiarity
>
> Paper summary
> -------------
> This paper presents HBFSim, an event-driven simulator for exploring
> heterogeneous HBM + High Bandwidth Flash (HBF) memory hierarchies for LLM
> serving. HBFSim preserves the semantic identity of weights, KV cache,
> activations, and workspace traffic while modeling flash-specific effects
> such as garbage collection, write amplification, queueing, SLC caching, and
> endurance. The paper evaluates nine flat-memory and HBM-as-cache
> configurations across three workload regimes. Its main finding is that
> semantic placement matters more than the memory organization itself:
> dominant read-heavy data should be placed in HBF, while transient and
> write-heavy data should remain in HBM.
>
> Comments for authors
> --------------------
> I find the topic timely and well suited to HotStorage. The result, which
> shows that semantic placement can remain close to all-HBM performance while
> supporting workloads that exceed HBM capacity, is an important information
> to practical long-context LLM workloads.
>
> My main concern is that the conclusions depend heavily on an assumed
> pre-silicon HBF model (Section 3.3). A brief clarification of which
> assumptions most strongly affect the main conclusions would be useful.
> Moreover, some conclusions, such as placing read-heavy objects in HBF and
> avoiding write-heavy transient data, are relatively intuitive. The paper
> would be stronger if the simulator revealed additional non-obvious insights
> that remain robust across these assumptions.
>
>
> * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
>
>
> Review #35D
> ===========================================================================
>
> Overall merit
> -------------
> 1. Reject
>
> Reviewer expertise
> ------------------
> 3. Knowledgeable
>
> Paper summary
> -------------
> This paper introduces HBFSim, an event-driven simulator for evaluating
> High Bandwidth Flash (HBF) in HBM+HBF memory systems for LLM serving. HBF
> offers large capacity at near-HBM bandwidth but introduces flash behaviors
> (GC, write amplification). Existing tools cannot model both flash internals
> and LLM semantics (weights, KV cache, activations). Using HBFSim, the
> authors evaluate 9 memory organizations and placement policies and find
> that HBF primarily alleviates capacity pressure rather than bandwidth
> bottlenecks; Workload-aware semantic placement matters more than flat vs.
> hierarchical organization; No single policy is best; optimal placement
> depends on the dominant data stream (weights vs. KV).
>
> Comments for authors
> --------------------
> Thank you for submitting the paper to HotStorage! I like reading the
> paper; however, I have trouble fully understanding it. On the high level,
> it is very clear, but the details are very muddy, and I do not find the
> takeaways clear. For example, the authors compared different policies of
> data placement, but I am not able to understand the conclusion.
>
> The weight sizes shown in Figure 1 appear incorrect. Are these numbers
> based on FP4 quantization? Even under FP8, the reported sizes look too
> small relative to publicly available model configurations. Could the
> authors clarify the precision and calculation method used?
>
> While the paper is generally well-written, several phrases reduce
> readability. In particular, terms such as “clean-room 70B/128K” are unclear
> without additional explanation. More precise and accessible wording would
> improve the overall clarity of the paper.
>
> I have no idea what Sec3.3 is talking about, which seems to be the only
> text on how the simulator handles HBF internals such as garbage collection
> and wear leveling.
>
> It is also unclear to me where the numbers (e.g., bandwidth) that the
> authors use come from.
>
> The anonymized repo does not exist.
>
>
> Comment @A1 by Reviewer A
> ---------------------------------------------------------------------------
> Dear Authors, thank you for submitting your work to HotStorage 2026. Your
> paper has been discussed by the PC members. Reviewers agree that the paper
> has chosen an interesting research topic. However, the evaluation part is
> not yet ready for publication in its current state. It has therefore been
> rejected.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fsl.cs.sunysb.edu/pipermail/hotstorage-chairs/attachments/20260725/96035962/attachment.htm>