Self-Optimizing Memory Controllers
Engin İpek, Onur Mutlu, José F. Martínez et al. · ACM SIGARCH Computer Architecture News · 2008
Efficiently utilizing off-chip DRAM bandwidth is a critical issuein designing cost-effective, high-performance chip multiprocessors(CMPs). Conventional memory controllers deliver relativelylow performance in part because they often employ fixed,rigid access scheduling policies designed for average-case applicationbehavior. As a result, they cannot learn and optimizethe long-term performance impact of their scheduling decisions,and cannot adapt their scheduling policies to dynamic workloadbehavior.We propose a new, self-optimizing memory controller designthat operates using the principles of reinforcement learning (RL)to overcome these limitations. Our RL-based memory controllerobserves the system state and estimates the long-term performanceimpact of each action it can take. In this way, the controllerlearns to optimize its scheduling policy on the fly to maximizelong-term performance. Our results show that an RL-basedmemory controller improves the performance of a set of parallelapplications run on a 4-core CMP by 19% on average (upto 33%), and it improves DRAM bandwidth utilization by 22%compared to a state-of-the-art controller.
Cited by 362 works. doi:10.1145/1394608.1382172
Authors
References
- Neuro-dynamic Programming
- Advances in neural information processing systems 11
- The SPLASH-2 programs
- Reinforcement Learning
- 10. Reinforcement learning
- Greedy Attribute Selection
- Niagara: A 32-Way Multithreaded Sparc Processor
- Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding
- The Alpha 21264 microprocessor
- Memory access scheduling