Just like with a physical machine, some user operations (such as a major upgrade of the operating system or a modification of the partition layout) could result in a Compute instance being unable to boot normally.

For times such as these when an instance cannot boot normally, a rescue mode exists in order to access the root disk of the instance by booting from a different source. In this case, we use an open source program called netboot.xyz that can utilize a variety of Linux, BSD, ISO images, specialized installers or utilities.

Pre-Requisites to Work with Rescue Mode

Rescue mode is not a default operation, and can be operated only via the CLI or the API. Rescue mode requires:

If you wish to use a custom ISO, it must be loaded in a publicly accessible URL. You can always use Simple Object Storage.

Warning

Exoscale is solely responsible for the instance to boot in rescue mode with the netboot.xyz loader. Any operation performed in netboot.xyz, and the repair of the instance itself, is out of Exoscale’s standard support scope. The examples provided in this guide are solely intended to point the user in the right direction, with no guarantee of their outcome. Knowledge of tooling and operations is required.

Booting Into Rescue Mode

To reboot in rescue mode, first start by shutting down your instance via the CLI or the Portal in the instance details:

    exo compute instance stop broken-vm

Then start your instance in either BIOS or UEFI rescue mode, depending on the template you are using.

To start an instance in BIOS rescue mode:

    exo compute instance start --rescue-profile=netboot broken-vm

To start an instance in UEFI rescue mode:

    exo compute instance start --rescue-profile=netboot-efi broken-vm

Start instance rescue mode

The instance will now boot the netboot.xyz image.

Accessing the Rescue Mode

To access the rescue mode, log into the Portal and navigate to the instance detail view. Select Console in the ... menu at the top right header.

A new window will open and display the instance console and the netboot.xyz menu:

netboot.xyz rescue menu from console

Note

When booting a Private Instance you need to enter Failsafe Menu from netboot.xyz to perform a manual network configuration.

You can now perform any operations you need to repair the instance and then reboot your instance normally after you have finished. After the instance has rebooted, it will exit rescue mode and try to boot from its root disk.

Booting from a Custom ISO Image in BIOS Rescue Mode

Through rescue mode, it is possible to boot an instance from an external ISO as long as the ISO supports VirtIO storage and network drivers. With this method, a bare metal recovery is possible, for example.

Note

The custom ISO must be located in a publicly accessible URL. You can always use our Simple Object Storage.

To boot from an external ISO:

  • Start your instance in rescue mode as described above
  • From the main netboot.xyz menu, select Tools -> Utilities
  • Then select netboot.xyz tools -> Test Distribution ISO
  • Enter the public URL of the target ISO (for example: https://releases.ubuntu.com/24.04/ubuntu-24.04-live-server-amd64.iso boots an Ubuntu Live CD)

Netboot_rescue_menu_from_console_iso_image

Booting from a Windows Recovery Image

If you need to recover a Windows instance from a Windows Recovery Image, you can use two ISOs we have provided, where all the needed virtIO drivers have already been integrated:

  • http://sos-de-muc-1.exo.io/windows-recovery-image/w2k16 a Windows 2016 recovery image
  • http://sos-de-muc-1.exo.io/windows-recovery-image/w2k19 a Windows 2019 recovery image
  • http://sos-de-muc-1.exo.io/windows-recovery-image/w2k22 a Windows 2022 recovery image

Warning

Although provided for public use, the above Windows Recovery Images are provided as is, without warranties of any kind. As for any other rescue mode operation, the user is solely responsible for the outcome of their actions, and rescue operations beyond the booting of the netboot.xyz loader are out of the Exoscale standard support scope.

Netboot_rescue_menu_from_console_iso_image

To proceed:

  • Start your instance in rescue mode as described above
  • From the main netboot.xyz menu, select Distributions -> Windows
  • Select Options -> Base URL
  • Enter the URL corresponding to your instance’s Windows version from the options listed above. Then enter any key to return to the Windows menu
  • After returning to the Windows menu, select Load Microsoft Windows Installer...
  • The Windows Recovery Image will load and you will be prompted to enter some basic options. click on Next when ready
  • On the next screen, click on Repair Your Computer

You can now recover your Windows instance.

Linux Password Reset via Rescue Environment

If you need to reset a Linux password, such as the root password, and cannot access the system normally, you can boot into a rescue environment and manually reset the password. Here’s a step-by-step guide to doing this using a rescue environment like netboot.xyz.

Step 1: Access a Linux Rescue Environment via netboot.xyz

  • Boot from netboot.xyz: Use the netboot.xyz tool to boot into a live or rescue environment for Linux: Stop the VM, and then boot it in rescue mode. Make sure your VM type is large or bigger.
      exo compute instance start --rescue-profile=netboot instance_name
  • Select a Distribution: In the netboot.xyz menu, choose an option to boot into Rescue Mode or a minimal environment. In this example we will use Ubuntu rescue environment, but you can use distribution of your choice available in the netboot.xyz.

Step 2: Mount the Root Filesystem

Once you’re in the live environment, you’ll need to identify and mount the root partition of your system. Follow these steps:

  • Identify the Root Partition:

Use the following command to list available partitions and find your root partition:

      fdisk -l

Look for a partition named something like /dev/sda1 or /dev/vda1. This is likely your root partition where the Linux system is installed.

  • Mount the Root Filesystem:

Mount the root filesystem to a directory like /mnt:

      mount /dev/vdX /mnt  # Replace /dev/vdX with the root partition (e.g., /dev/vda1)
  • Mount Additional Filesystems:

Mount the necessary system filesystems required for chroot:

    mount -t proc /proc /mnt/proc
    mount --rbind /sys /mnt/sys
    mount --rbind /dev /mnt/dev

Step 3: Chroot into the Mounted Filesystem

  • Enter the chroot environment:

Use chroot to change your root directory to the mounted filesystem:

      chroot /mnt
  • Reset the Password:

To reset the root password, run the following command:

      passwd root

If you need to reset the password for a specific user, replace root with the username:

      passwd <username>
  • Exit the Chroot Environment:

After resetting the password, exit the chroot environment by typing:

      exit

Step 4: Unmount Filesystems and Reboot

  • Unmount the filesystems:

Unmount the filesystems you mounted earlier:

    umount --recursive --lazy /mnt
  • Reboot the system:

Reboot the system with the following command:

      exo compute instance reboot "instance-name"

After the reboot, you should be able to log into the system with the new password via the console or SSH.