The Datacenter Observatory

The Datacenter Observatory is an experimental research facility for cloud computing research. While relatively small on the scale of industrial datacenters, the system nonetheless allows cloud computing experiments on a scale that is relatively unique in the academic world and that is relevant to industrial practice. It contains 7 racks hosting 178 2-processor servers with 8-core processors, for a total of 2848 processing cores. The system further contains 22.3TB of memory, 408TB of local disk storage (composed of 52TB SSD and 356TB SATA disks), and a storage array of 72TB of disks, for an aggregate of around 0.5PB of storage. The various components will be connected by two 96-port 10 Gigabit Ethernet network switches.

For various reasons, including economies of scale, datacenters in current use are very large, and this will continue and even increase to be the case. Many of the interesting research problems are therefore concerned with scalability: how to build the systems that can scale to this size of installation. Very few academic researchers have access to this kind of facility. Many researchers use smaller clusters in their labs, and these can serve as prototyping facilities, but they need access to a large datacenter for experimentation and measurement. They cannot use commercially available cloud computing services or datacenters, because they want to experiment with the hardware, the network, the operating system, and all other levels of the software stack. They could not possibly hope to do experimentation at such low levels of the system without upsetting production use.