Skip to content

Capacity Reservations

Copy/Paste Quick Reference

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
capacity_reservation_group = {
    test = {
        resource_group = "test_rg"
        zones          = ["1", "2"]
        capacity_reservations = {
            e2s = {
                zone = "2"
                sku = {
                    name     = "Standard_E2s_v7"
                    capacity = 1
                }
            }
        }
    }
}

1
2
3
4
5
6
7
8
9
# When this is set, all VMs will use the listed CRG by default
default_capacity_reservation_group = "test"

# The default can be overridden to use something different, or set to "" to opt a specific VM out of using the CRG
capacity_reservation_group = "test"
capacity_reservation_group = ""

# Use the CRG ID to reference an existing CRG to use with a VM
capacity_reservation_group = /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/existing-rg/providers/Microsoft.Compute/capacityReservationGroups/existing-crg"

Overview

Capacity reservations let us reserve compute capacity in a region ahead of time so that a VM is guaranteed to be able to start, rather than competing for capacity with everyone else in the region at boot time. This matters for customers who cannot tolerate an allocation failure during a DR failover, a planned migration, or a scheduled restart of a large workload.

There are two resources, and one is nested inside the other. The capacity_reservation_group is a container that holds reservations and is the thing a VM is attached to. The capacity_reservations neseted insde the group are the actual reserved capacity, one entry per VM size.

A group on its own reserves nothing and costs nothing. The reservations inside it are what reserve capacity, and they bill from the moment they are created. See Cost below before adding these to a customer environment.

Naming

Both resources follow our standard naming convention and need their keys added to the environment's name_prefixes and name_suffixes. Unlike some other modules, there is no fallback if the key is missing, so terraform will fail at plan time.

1
2
3
4
5
6
7
8
9
name_prefixes = {
  capacity_reservation_group = "dev-"
  capacity_reservation       = "dev-"
}

name_suffixes = {
  capacity_reservation_group = "-crg"
  capacity_reservation       = "-cr"
}

The entry key is used as the middle of the name, so the example above produces dev-test-crg and dev-e2s-cr. As with our other modules, setting name on an entry overrides the generated name entirely.

Zones

The zones on the group and the zone on each reservation must agree, and a VM can only use a reservation that matches both its size and its zone. A group defined with zones = ["1", "2"] can hold reservations in zone 1 and zone 2, but a VM in zone 3 cannot use that group at all.

A group with no zones is a regional reservation, and reservations inside it must also omit zone. Since nearly all of our VMs set a zone, a zoned group is almost always what you want. Mismatches are rejected by Azure at apply time rather than by terraform at plan time.

Each reservation covers capacity VMs of that size in that zone. Reserving for four identical VMs means one reservation entry with capacity = 4, not four entries.

Assigning VMs to a Capacity Reservation Group

The default_capacity_reservation_group variable sets a group for every VM in the environment, in the same way location sets a default region. Its value is the key of an entry in capacity_reservation_group.

1
default_capacity_reservation_group = "test"

Individual VMs override the default with their own capacity_reservation_group attribute, which works on both windows_vms and linux_vms. Setting it to an empty string opts that VM out of the default and places it in no group at all.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
linux_vms = {
    crgtest = {
        size                       = "Standard_E2s_v7"
        capacity_reservation_group = "test"
    }
    other = {
        size                       = "Standard_E2s_v7"
        capacity_reservation_group = "othergroup"
    }
    excluded = {
        size                       = "Standard_D2s_v5"
        capacity_reservation_group = ""
    }
}

The VM's size must exactly match the sku.name of a reservation in the group, and the VM's zone must match that reservation's zone. A VM pointed at a group with no matching reservation will fail to deploy.

Note that default_capacity_reservation_group is only resolved for VMs that actually use it. Setting it to a key that does not exist will not fail on its own, but every VM that inherits it will fail at plan time with a lookup error naming the missing key.

Using a Group Created Outside Terraform

The azurerm provider does not publish data sources for either of these resources, so our usual existing = true pattern is not available here. To attach VMs to a group that we did not create, pass the group's resource ID instead of a key. Any value beginning with / is treated as a resource ID and used as-is.

1
default_capacity_reservation_group = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/existing-rg/providers/Microsoft.Compute/capacityReservationGroups/existing-crg"

This works at the VM level as well. Because there is no data source behind it, terraform cannot verify that the group exists, so a wrong ID surfaces as an API error when the VM is created.

Cost

A capacity reservation bills at the pay-as-you-go compute rate for its size and capacity from the moment it is created, whether or not a VM is using it. Reserving a Standard_E2s_v7 with capacity = 1 costs roughly the same as leaving that VM running continuously. When a VM does run on the reservation, the VM is billed and the reservation is not, so there is no double charge.

Coordinate with customers on timing so they do not see a large, unexpected bill. Be extra careful to tear down when testing.

Subscription Requirements

Capacity reservation is not available on every subscription or for every VM size. The most common failure is SkuNotAvailable at apply time, which means the subscription is not entitled to that size in that region, and is resolved through a quota or SKU access request rather than a code change.

You will most likely run into this in our dev environment. Regular customer subscriptions should be of a type that supports capacity reservations.