<div dir="auto">There wasn't much that the reviewers disliked that we didn't expect, so let's go and improve the overhead discussion. Onwards and upwards!</div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Fri, Jul 24, 2026, 8:06 PM HotStorage 2026 HotCRP <<a href="mailto:noreply-hotstorage26@hotcrp.com">noreply-hotstorage26@hotcrp.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Dear authors,<br>
<br>
The program committee for the 2026 ACM Workshop on Hot Topics in Storage <br>
and File Systems (HotStorage 2026) regrets to inform you that your <br>
submission 120 was rejected to appear in the proceedings of the <br>
workshop.<br>
<br>
* Title: Regular: FlexiPool: Towards Flexible Cache Policy Switching<br>
* Site: <a href="https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fhotstorage26.hotcrp.com%2fpaper%2f120&c=E,1,kLvnHox-ew_0hdv_1mSrweYbmMS4DftNc9GurxAzAoTzmBeIqXaTpLNAP8nzpvR0d7EfgmPE7CiuTlD-CKvWdjcDVqs-0YIlIC8Oa8d189M,&typo=1" rel="noreferrer noreferrer" target="_blank">https://hotstorage26.hotcrp.com/paper/120</a><br>
<br>
23 of 78 submissions were accepted.<br>
<br>
Visit the submission site for reviews, comments, and related <br>
information. Reviews and comments are also included below.<br>
<br>
Contact <a href="mailto:chairs26@hotstorage.org" target="_blank" rel="noreferrer">chairs26@hotstorage.org</a> with any questions or concerns.<br>
<br>
Sincerely,<br>
Young-ri Choi and Bryan S. Kim<br>
HotStorage 2026 Program Co-Chairs<br>
<br>
Review #120A<br>
===========================================================================<br>
<br>
Overall merit<br>
-------------<br>
2. Weak reject<br>
<br>
Reviewer expertise<br>
------------------<br>
4. Expert<br>
<br>
Paper summary<br>
-------------<br>
The paper proposes an automated cache policy switching framework called FlexiPool. FlexiPool have the ability to switch between arbitrary policies (not necessarily previously predefined). It uses an LLM-based agent to synthesize the switching rules. Three challenges were identified for which a solution has been proposed: 1) maintaining multiple eviction strategies meta data, 2) "switch logic" design, and 3) overfitting of the switch logic.<br>
The results show some benefit as compared to SOTA work.<br>
<br>
Comments for authors<br>
--------------------<br>
Strengths of the paper<br>
- using agents to explore cache eviction policy selection is interesting<br>
- the paper addresses an impactful topic about cache strategy selection<br>
- the paper falls clearly within the topics of HotStorage<br>
- the authors evaluated exhaustively against several workloads<br>
<br>
Main weaknesses<br>
- the overheads are not studied<br>
- the gains seem modest<br>
- the motivation could be enhanced<br>
- the writing could be enhanced <br>
<br>
<br>
Discussion <br>
We would like to thank the authors for their contribution<br>
<br>
- the overheads are not studied: one important aspect in deploying a cache strategy is its overhead. Even if the authors ackowledged this aspect in the introduction no overhead related to the simulations, to the agent were given. This is a real issue as the reader does not have any idea about the benefit in terms of BHR with regards to the added overhead. Is it worth it to run a voluminous LLM for such performance gain ? what is the simulator overhead, how does it scale with the number of policies ?...<br>
<br>
- the gains seem modest: the authors show in the evaluation part that their policy gives BHR percentage gains that are mostly under 3%. It would be interesting to discuss how such an enhancement is worth the deployment.<br>
<br>
- the motivation could be enhanced: in the introduction, the motivation behind this study could be enhanced. It is not clear what is the limitation in previous work and what is the impact in terms of performance of those limitations. Why is it so important to embed more than 10 caching strategies?<br>
<br>
<br>
- the writing could be enhanced: even if HotStorage is a topical workshop, there are several terminology used without being defined: <br>
- BHR acronym is used without being defined, <br>
- "switching logic" is a concept used without being properly defined, the challenges are given but the reader do not understand how were they identified .... <br>
- Table 1 reports some numbers but the reader does not have any clue to know which metric the authors are talking about<br>
- Figure one is not clear for the reviewer<br>
- the authors suppose that the reviewer has full knowledge of past work they discuss in the paper<br>
- Several paragraphs are not clear (e.g. the paragraph on the agent in the intro, eq 1, ...)<br>
<br>
- the strategies FlexiPool is evaluated against are quite old.<br>
<br>
- Section 2.1 : "... the next workload phase", how is this prediction done ?<br>
<br>
- For the simulation part, it is not clear on which time window the simulation is performed: is it on time windows N-1 during time window N ?<br>
<br>
- Why is the choice of cache policies the same as the cache policies on libCacheSim ? should the choice of the caching strategies be as random as that ?<br>
<br>
- an overview figure of the contribution could be helpful.<br>
<br>
<br>
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *<br>
<br>
<br>
Review #120B<br>
===========================================================================<br>
<br>
Overall merit<br>
-------------<br>
2. Weak reject<br>
<br>
Reviewer expertise<br>
------------------<br>
3. Knowledgeable<br>
<br>
Paper summary<br>
-------------<br>
This paper proposes FlexiPool, a framework that dynamically switches among multiple cache eviction policies to adapt to changing workloads. It reduces online switching overhead by simulating candidate policies in the background and preserving their respective cache states and metadata. The paper also introduces an LLM-based coding agent that automatically generates interpretable and editable switching rules from historical byte hit rates. Experiments show that FlexiPool generally outperforms several state-of-the-art cache eviction methods and matches or surpasses the best static policy on most traces.<br>
<br>
Comments for authors<br>
--------------------<br>
Thank you for submitting this paper to HotStorage 2026. The paper presents FlexiPool, which simulates multiple cache policies in the background and uses an LLM-based coding agent to generate executable switching rules. The paper has two main strengths. <br>
<br>
First, it addresses an important problem with clear motivation: different cache policies may perform best during different workload phases, while existing approaches are often limited to predefined policy sets or fixed switching rules. <br>
<br>
Second, its use of LLMs is relatively novel. Rather than producing black-box decisions, the agent generates executable, inspectable, and editable code, making the switching algorithm itself automatically generated and refinable.<br>
<br>
However, several issues prevent me from leaning toward acceptance.<br>
<br>
First, the paper lacks essential system-overhead experiments. The evaluation focuses mainly on BHR, but does not quantify the CPU, memory, storage-bandwidth, switching-latency, or tail-latency costs of background simulation and state migration. The cost of LLM-based code generation is also not reported. More importantly, the paper does not show whether these overheads offset the BHR gains or how they affect end-to-end throughput and latency.<br>
<br>
Second, FlexiPool’s overall gains are modest. On several Cloudflare traces, it only matches the best static policy. On CF1, the gain is 0.12 percentage points, compared with an Oracle gap of 1.02 points, while the gains on CF5–CF8 are near zero. Given the additional complexity introduced by background simulation and LLM-based code generation, it is unclear whether these modest BHR improvements justify the proposed design.<br>
<br>
Third, the LLM-based coding agent is one of the key novelties of the paper. However, it is not yet clear why an LLM is the appropriate solution for this problem, rather than simpler rule-generation or program-synthesis techniques. A discussion of what unique capabilities the LLM provides, and why these capabilities are essential for generating effective switching policies, would better justify this design choice.<br>
<br>
Finally, the LLM-based coding agent is not described clearly enough. The paper provides no concrete examples of generated rules and lacks details about prompts, interfaces, feedback formats, stopping criteria, and the multi-round refinement process. This makes the core method difficult to understand and reproduce. The paper would benefit from representative generated rules and a clearer description of the agent workflow.<br>
<br>
<br>
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *<br>
<br>
<br>
Review #120C<br>
===========================================================================<br>
<br>
Overall merit<br>
-------------<br>
2. Weak reject<br>
<br>
Reviewer expertise<br>
------------------<br>
3. Knowledgeable<br>
<br>
Paper summary<br>
-------------<br>
This paper presents FlexiPool, a cache-policy switching framework that supports an extensible pool of eviction policies. FlexiPool runs only the active policy on the request path, while periodically simulating candidate policies in the background and retaining their cache states and metadata for switching. The paper further uses an LLM-based coding agent to synthesize executable and editable switching rules from recent per-policy byte-hit-rate histories. Experiments using different traces show that FlexiPool generally matches the best static policy and outperforms baselines. On the Cloudflare traces, the agent-generated rules improve over recent-best Search, although the gains are generally small.<br>
<br>
Comments for authors<br>
--------------------<br>
The idea of separating request serving from background policy simulation is useful, and preserving each simulated policy’s warm state makes switching more flexible.<br>
<br>
However, the novelty of using an LLM to generate switching rules feels incremental, and the evaluation does not yet demonstrate a compelling benefit. Further, the agent is developed separately for each trace and cache-size configuration using training and validation segments, so the evaluation demonstrates offline workload-specific rule generation, rather than the claimed ability to adapt automatically to a new workload at runtime.<br>
<br>
<br>
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *<br>
<br>
<br>
Review #120D<br>
===========================================================================<br>
<br>
Overall merit<br>
-------------<br>
2. Weak reject<br>
<br>
Reviewer expertise<br>
------------------<br>
3. Knowledgeable<br>
<br>
Paper summary<br>
-------------<br>
This paper presents a dynamic switching eviction cache policy at runtime. It solves three challenges: 1) for the memory overhead of running multiple policies, it just run one policy for online requests, 2) for an expressive switching logic, it uses a LLM-based coding agent, 3) for removing bias toward specific phases or transient behavior, it uses a window with a train/validation split and selects the candidate with the best held-out validation performance.<br>
<br>
Comments for authors<br>
--------------------<br>
Strengths:<br>
• As workload behavior changes over time, the policy must be dynamically adjusted. This dynamic adjustment is clearly defined and adequately demonstrates the impact of policy changes.<br>
• It compares the work with the most related work, such as SOPA.<br>
• It considers cache replacement, memory‑cost pressure, and LLM agents for systems design, which is valuable.<br>
<br>
Weaknesses:<br>
• The overhead of the proposed approach is not presented in terms of CPU and memory utilization.<br>
• The evaluation is not complete. It does not show the switch latency between policies or the resulting impact on hit rate.<br>
• Some policies require specific metadata and data structures, and therefore switching to other policies requires high operational cost. This work does not address such policies.<br>
• The work does not discuss how to select the window time.<br>
• The improvement is low compared to SOPA (Table 2), and this paper does not discuss the overhead of their approach relative to SOPA.<br>
• The paper does not compare their approach against recent work such as "3L-CACHE: Low Overhead and Precise Learning‑based Eviction Policy for Web Caches" or "Learning Cache Replacement with CACHEUS".<br>
• The paper focuses on trace-driven simulations using libCacheSim. There is no evaluation on a real-world workloads that have complexities.<br>
• The evaluation uses relatively short windows (100K requests) and specific traces. Generalization to completely different workload types or much longer time horizons has not been explored.<br>
• All experiments use only a 0.1% cache size relative to working-set size. This is extremely small and may favor certain policies; results may not generalize to larger, more typical cache sizes where behavior differs significantly. <br>
<br>
Comments for Authors:<br>
• It would be better to measure CPU and memory utilization and compare their approach against state‑of‑the‑art articles.<br>
• It would be better to implement their approach in an enterprise cache module (e.g., OpenCAS or EnhanceIO) to evaluate a real system rather than relying on simulation alone.<br>
• It would be better to classify policies that share the same metadata and data structures so that they can switch between them at runtime without requiring extra metadata.<br>
• Including experiments on a live cache system or evaluating the framework under larger policy pools and more diverse workload characteristics would provide stronger evidence of FlexiPool's scalability and practical applicability.<br>
• Although the reported byte hit rate improvements are consistent, discussing their impact on end-to-end system performance (e.g., latency reduction, backend traffic, or infrastructure cost savings) would better demonstrate the practical value of the proposed approach.<br>
• Figure 1 is referenced but is not described in details in the manuscript. The caption alone is not descriptive.<br>
• In this manuscript only byte-miss cost is considered. In real systems, switching may involve CPU bursts, lock contention, or application stalls. Modeling or measuring these temporal costs would make the evaluation more realistic.<br>
<br>
<br>
Comment @A1 by Reviewer B<br>
---------------------------------------------------------------------------<br>
Dear Authors,<br>
<br>
All reviewers agree that the paper addresses an important problem. However, there is broad consensus that the novelty and motivation for the LLM-based approach are not yet sufficiently justified, the reported performance gains are relatively modest, and the evaluation needs to be strengthened. Overall, the reviewers do not believe the current submission provides sufficient evidence to support acceptance. We encourage the authors to address these concerns in a future revision.<br>
<br>
</blockquote></div>