Tail Latency Meets Caching: An Unusual Alliance

Abstract: In today’s world of interactive computing, web services need to achieve low latency for almost all user requests (e.g., low 99-th percentile latency). Reducing the latency tail is challenging because tail events are rate and often have complex causes. Hence, tail latency has been a recurring theme in academic and industry systems research for over a decade.
In this talk, I will demonstrate that caching can be a strong ally in the pursuit of low tail latency. My talk will present two concrete examples, in datacenter and edge caching, where redesigned caching systems lead to order-of-magnitude reductions in tail latency. These results contradict the common belief that, due to their non-negligible miss ratios, caches are of little benefit for reducing tail latency. I will show how to overcome this common belief by exploiting analytical performance modeling to guide the design of caching systems.
While rooted in theory, my research is highly practical and conducted in continuous collaboration with partners from industry. I will describe how collaboration and open-source-prototypes have enabled production use of my research at a top-ten US website and deployment tests at several other companies.

Biography: Daniel S. Berger is a postdoctoral fellow in computer science at CMU since July 2018. Daniel's research intersects systems and mathematical modeling and focuses on reducing tail latency in large Internet content serving systems. He is the recipient of the Mark-Stehlik Postdoctoral Fellowship at CMU (2018) and a best paper award at Performance (2014). Daniel received his Ph.D (2018) from Kaiserslautern, Germany, while working as a project scientist at CMU (2017) and completing his graduate course work at ETH Zurich (2012-2013).


Share this event:

facebook linked in twitter email

Media Contact: Timothy Zhu



The School of Electrical Engineering and Computer Science was created in the spring of 2015 to allow greater access to courses offered by both departments for undergraduate and graduate students in exciting collaborative research in fields.

We offer B.S. degrees in electrical engineering, computer science, computer engineering and data science and graduate degrees (master's degrees and Ph.D.'s) in electrical engineering and computer science and engineering. EECS focuses on the convergence of technologies and disciplines to meet today’s industrial demands.

School of Electrical Engineering and Computer Science

The Pennsylvania State University

207 Electrical Engineering West

University Park, PA 16802


Department of Computer Science and Engineering


Department of Electrical Engineering