Research on disaggregated architectures and the systems built on them is challenging to evaluate. Such work by definition involves radical changes to performance critical system components (e.g., memory subsystem) with complex impact on overall system performance at scales ranging from racks to complete data centers. We examine the challenges faced in evaluation for work in this space, along with the inherent trade-offs chosen.
We then describe modular full system simulation with SimBricks as a means of evaluating disaggregated systems end-to-end. Simulation in SimBricks offers the required flexibility to modify and re-combine components, while maintaining control over performance across components. Looking forward, we discuss future steps to further improve and streamline disaggregated system simulation.