On this page:
The Cloud  Lab Manual
2021-11-04 (aac7cb5)

The CloudLab Manual

The CloudLab Team

This document is also available as a single page

CloudLab is a "meta-cloud"—that is, it is not a cloud itself; rather, it is a facility for building clouds. It provides bare-metal access and control over a substantial set of computing, storage, and networking resources; on top of this platform, users can install standard cloud software stacks, modify them, or create entirely new ones.

The current CloudLab deployment consists of more than 25,000 cores distributed across three sites at the University of Wisconsin, Clemson University, and the University of Utah. CloudLab interoperates with existing testbeds including GENI and Emulab, to take advantage of hardware at dozens of sites around the world.

The control software for CloudLab is open source, and is built on the foundation established for Emulab, GENI, and Apt. Pointers to the details of this control system can be found on CloudLab’s technology page.

Get started!

    1 CloudLab Status Notes

    2 Getting Started

      2.1 Next Steps

    3 CloudLab Users

      3.1 GENI Users

      3.2 Register for an Account

        3.2.1 Join an existing project

        3.2.2 Create a new project

        3.2.3 Setting up SSH access

        3.2.4 Setting up X11

    4 CloudLab and Repeatable Research

    5 Creating Profiles

      5.1 Creating a profile from an existing one

        5.1.1 Preparation and precautions

        5.1.2 Cloning a Profile

        5.1.3 Copying a Profile

        5.1.4 Creating the Profile

        5.1.5 Updating a profile

      5.2 Creating a profile with a GUI

      5.3 Repository-Based Profiles

        5.3.1 Updating Repository-Based Profiles

        5.3.2 Branches and Tags in Repository-Based Profiles

      5.4 Creating a profile from scratch

      5.5 Sharing Profiles

      5.6 Versioned Profiles

    6 Basic Concepts

      6.1 Profiles

        6.1.1 On-demand Profiles

        6.1.2 Persistent Profiles

      6.2 Experiments

        6.2.1 Extending Experiments

      6.3 Projects

      6.4 Physical Machines

      6.5 Virtual Machines and Containers

    7 Resource Reservations

      7.1 What Reservations Guarantee

      7.2 How Reservations May Affect You

      7.3 Making a Reservation

      7.4 Using a Reservation

    8 Describing a profile with python and geni-lib

      8.1 A single XEN VM node

      8.2 A single physical host

      8.3 Two XenVM nodes with a link between them

      8.4 Two ARM64 servers in a LAN

      8.5 A VM with a custom size

      8.6 Set a specific IP address on each node

      8.7 Specify an operating system and set install and execute scripts

      8.8 Profiles with user-specified parameters

      8.9 Add storage to a node

      8.10 Debugging geni-lib profile scripts

    9 Virtual Machines and Containers

      9.1 Xen VMs

        9.1.1 Controlling CPU and Memory

        9.1.2 Controlling Disk Space

        9.1.3 Setting HVM Mode

      9.2 Docker Containers

        9.2.1 Basic Examples

        9.2.2 Disk Images

        9.2.3 External Images

        9.2.4 Dockerfiles

        9.2.5 Augmented Disk Images

        9.2.6 Remote Access

        9.2.7 Console

        9.2.8 ENTRYPOINT and CMD

        9.2.9 Shared Containers

        9.2.10 Privileged Containers

        9.2.11 Remote Blockstores

        9.2.12 Temporary Block Storage

        9.2.13 DockerContainer Member Variables

    10 Storage Mechanisms

      10.1 Overview of Storage Mechanisms

      10.2 Node-Local Storage

        10.2.1 Specifying Storage in a Profile – Local Datasets

        10.2.2 Allocating Storage in a Running Experiment

        10.2.3 Persisting Local Data

      10.3 Image-backed Datasets

      10.4 Remote Datasets

      10.5 NFS Shared Filesystems

      10.6 Storage Type Summary (TL;DR)

      10.7 Example Storage Profiles

        10.7.1 Creating a Node-local Dataset

        10.7.2 Creating an Image-backed Dataset from a Node-local Dataset

        10.7.3 Using and Updating an Image-backed Dataset

        10.7.4 Creating a Remote Dataset

        10.7.5 Using a Remote Dataset on a Single Node

        10.7.6 Using a Remote Dataset on Multiple Nodes via a Shared Filesystem

        10.7.7 Using a Remote Dataset on Multiple Nodes via Clones

    11 Advanced Topics

      11.1 Disk Images

      11.2 RSpecs

      11.3 Public IP Access

        11.3.1 Dynamic Public IP Addresses

      11.4 Markdown

      11.5 Introspection

        11.5.1 Client ID

        11.5.2 Control MAC

        11.5.3 Manifest

        11.5.4 Private key

        11.5.5 Profile parameters

      11.6 User-controlled switches and layer-1 topologies

      11.7 Portal API

    12 Hardware

      12.1 CloudLab Utah

      12.2 CloudLab Wisconsin

      12.3 CloudLab Clemson

      12.4 Apt Cluster

      12.5 Mass

      12.6 OneLab

    13 Planned Features

      13.1 Improved Physical Resource Descriptions

    14 CloudLab OpenStack Tutorial

      14.1 Objectives

      14.2 Prerequisites

      15.4 Logging In

        15.4.1 Using a CloudLab Account

        15.4.2 Using a GENI Account

      14.4 Building Your Own OpenStack Cloud

      14.5 Exploring Your Experiment

        14.5.1 Experiment Status

        14.5.2 Profile Instructions

        14.5.3 Topology View

        14.5.4 List View

        14.5.5 Manifest View

        14.5.6 Graphs View

        14.5.7 Actions

        14.5.8 Web-based Shell

        14.5.9 Serial Console

      14.6 Bringing up Instances in OpenStack

      14.7 Administering OpenStack

        14.7.1 Log Into The Control Nodes

        14.7.2 Reboot the Compute Node

      14.8 Terminating the Experiment

      14.9 Taking Next Steps

    15 CloudLab Chef Tutorial

      15.1 Objectives

      15.2 Motivation

      15.3 Prerequisites

      15.4 Logging In

        15.4.1 Using a CloudLab Account

        15.4.2 Using a GENI Account

      15.5 Launching Chef Experiments

      15.6 Exploring Your Experiment

        15.6.1 Experiment Status

        15.6.2 Profile Instructions

        15.6.3 Topology View

        15.6.4 List View

        15.6.5 Manifest View

        15.6.6 Actions

      15.7 Brief Introduction to Chef

      15.8 Logging in to the Chef Web Console

        15.8.1 Web-based Shell

        15.8.2 Chef Web Console

      15.9 Configuring NFS

        15.9.1 Exploring The Structure

      15.10 Apache Web Server and ApacheBench Benchmarking tool

        15.10.1 Understanding the Internals

      15.11 Final Remarks about Chef on CloudLab

      15.12 Terminating Your Experiment

      15.13 Future Steps

    16 Citing CloudLab

    17 Getting Help