# Manage Flexible Storage

The Exoscale __Flexible Storage Template__ aims to empower users so they can manage
their instance's disk/storage as they deem fit. The storage template provides:

- A minimalistic [Debian][debian-os]-based template with no extraneous components and only what is required to get you started (namely, `cloud-init` and `sudo`)

- A built-from-scratch template using the [Debian][debian-cdimage] `netinst` installation media  and the `ssh-server` installation scenario (AKA `tasksel`)

- Uses [UEFI][wikipedia-uefi] boot mode and a [GPT][wikipedia-gpt] partition table (as opposed to Debian's stock cloud image) to enable support for disk images larger than 2 TB (along [Exoscale Storage Optimized Instances][exoscale-offering-storage])

- Uses [LVM][wikipedia-lvm] logical partitioning (as opposed to Debian's stock cloud image) to allow you to manage partitions and file systems as you deem fit

The flexible storage template is based on - but deviates minimally from - Debian's stock installation configuration, so it is easy to embrace for users accustomed to Debian or Ubuntu.

[debian-os]: https://www.debian.org/
[debian-cdimage]: https://cdimage.debian.org/cdimage/release/current/amd64/iso-cd/
[exoscale-offering-storage]: https://www.exoscale.com/pricing/#storage-optimized-instances
[wikipedia-uefi]: https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface
[wikipedia-gpt]: https://en.wikipedia.org/wiki/GUID_Partition_Table
[wikipedia-lvm]: https://en.wikipedia.org/wiki/Logical_Volume_Manager_%28Linux%29


## Deploying a Flexible Storage Instance

The flexible storage template is based on - but deviates minimally from - Debian’s stock installation configuration, so it is easy to embrace for users accustomed to Debian or Ubuntu.


```bash
exo compute instance create storage \
  --zone <desired-zone> \
  --template 'Exoscale Flexible Storage 10' \
  --disk-size 50 \
  --ssh-key <your-SSH-keypair>
```

[exoscale-doc-cli]: https://community.exoscale.com/tools/command-line-interface/

### Log In
After the instance is deployed, log into it via SSH with the CLI:

```bash
exo compute instance ssh storage
# [output]
debian@storage:~$
```

Then use `sudo` to obtain administrative privileges:

```bash
debian@storage:~$ sudo -i
# [output]
root@storage:~#
```

(For the sake of readability, we will hide the `root@storage:...#` command
prompt in further shell commands examples.)


## Flexible Storage Management
### Create Flexible Storage Instance

Creating a flexible storage instance is similar to creating any other Exoscale Compute instance. Here is an example with the [CLI]({{< ref "/tools/command-line-interface/" >}}):

```bash
exo compute instance create storage \
  --zone <desired-zone> \
  --template 'Exoscale Flexible Storage 10' \
  --disk-size 50 \
  --ssh-key <your-SSH-keypair>
```

### Log In via SSH
After the instance is deployed, log into it via SSH with the CLI:

```bash
exo compute instance ssh storage
# [output]
debian@storage:~$
```

Then use `sudo` to obtain administrative privileges:

```bash
debian@storage:~$ sudo -i
# [output]
root@storage:~#
```

> [!NOTE]
> For readability, further shell commands will omit the `root@storage:...#` command prompt.

## Default Partition Layout

Thanks to the [Linux Logical Volume Manager (LVM)][wikipedia-lvm], as well as
following the [Linux Filesystem Hierarchy Standard (FHS)][wikipedia-fhs] and
general best practices, the flexible storage template's initial partitioning
separates a few systems. These are `ext4`-formatted partitions, namely
`/` (root), `/var`, `/tmp` (as well as `/boot` and `/boot/efi`):

```bash
lsblk
```

List block devices (disks/partitions):

```bash
# [output]
NAME                MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda                 254:0    0   50G  0 disk
|-vda1              254:1    0  511M  0 part /boot/efi
|-vda2              254:2    0  512M  0 part /boot
`-vda3              254:3    0    9G  0 part
  |-vg.flex-lv.root 253:0    0    5G  0 lvm  /

  |-vg.flex-lv.var  253:1    0    2G  0 lvm  /var
  `-vg.flex-lv.tmp  253:2    0    1G  0 lvm  /tmp
```

> [!NOTE]
> The `50G` disk (`vda`) space not being entirely used by underlying partitions (`vda<N>`).

[wikipedia-fhs]: https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

As opposed to most other templates, partition sizes will
__not automatically grow__ after an instance's disk is
[resized]({{< ref "/product/compute/instances/how-to/instance-scaling/#scaling-the-disk-size" >}}). 
You can create or resize partitions according to your requirements.

## Storage Management Operations

Further, as an example, we will demonstrate both:
- how to create a new `/data` partition with the `xfs` filesystem
- how to resize the `/var` partition

### Create New Logical Volume

You can create as many partitions or file systems as you need. Here is an example
with a `xfs`-formatted `/data` partition - install XFS utilities:

```bash
apt-get install xfsprogs
```

Create the LVM Logical Volume (LV):

```bash
lvcreate -n lv.data -L 20G vg.flex
```

Output:

```bash
Logical volume "lv.data" created.
```

Show the LVM Logical Volume (LV) details:

```bash
lvdisplay /dev/mapper/vg.flex-lv.data
```

Output (partial):

```bash
  --- Logical volume ---
  LV Path                /dev/vg.flex/lv.data
  LV Name                lv.data
  VG Name                vg.flex
  LV Status              available
  LV Size                20.00 GiB
```

Create the filesystem:

```bash
$ mkfs.xfs -L DATA /dev/mapper/vg.flex-lv.data
```

Output (partial):

```bash
meta-data=/dev/mapper/vg.flex-lv.data isize=512    agcount=4, agsize=1310720 blks
```

Create the filesystem mountpoint:

```bash
mkdir -p /data
```

Mount the filesystem:

```bash
$ mount /dev/mapper/vg.flex-lv.data /data
```

Show the filesystem capacity/usage:

```bash
df -h /data
```

Output:

```bash
Filesystem                   Size  Used Avail Use% Mounted on
/dev/mapper/vg.flex-lv.data   20G   53M   20G   1% /data
```

You may want to add to `/etc/fstab` to allow automatic mounting at boot time:

```bash
grep /data /etc/fstab
```

Output:

```bash
/dev/mapper/vg.flex-lv.data /data xfs defaults 0 2
```

### Resize the Physical Volume

The first step is to grow the `/dev/vda3` partition, corresponding to the LVM
Physical Volume (PV) - show LVM Volume Groups (VGs)

```bash
vgs
```

Output:

```bash
  VG      #PV #LV #SN Attr   VSize   VFree
  vg.flex   1   3   0 wz--n- <9.00g  1.00g
```

Grow the LVM-backing disk partition:

```bash
growpart /dev/vda 3
```

Output:

```bash
CHANGED: partition=3 start=2097152 old: size=18872320 end=20969472 new: size=102760415,end=104857567
```

Grow the LVM Physical Volume (PV):

```bash
pvresize /dev/vda3
```

Output:

```bash
  Physical volume "/dev/vda3" changed
  1 physical volume(s) resized or updated / 0 physical volume(s) not resized
```

Show LVM Volume Groups (VGs):

```bash
$ vgs
```

Output:

```bash
  VG      #PV #LV #SN Attr   VSize   VFree
  vg.flex   1   3   0 wz--n- <49.00g 41.00g
```

> [!NOTE]
> The increase of `VFree` space in the LVM Volume Group (VG).


### Resize a Logical Volume
To resize (or extend) an existing LVM Logical Volume (LV) and partition
(with `/var` as an example) - grow the LVM Logical Volume (LV):

```bash
lvresize -L +10G /dev/mapper/vg.flex-lv.var
```

Output:

```bash
  Size of logical volume vg.flex/lv.var changed from <2.00 GiB (511 extents) to <12.00 GiB (3071 extents).
  Logical volume vg.flex/lv.var successfully resized.
```

Grow the filesystem:

```bash
resize2fs /dev/mapper/vg.flex-lv.var
```

Output:

```bash
resize2fs 1.44.5 (15-Dec-2018)
Filesystem at /dev/mapper/vg.flex-lv.var is mounted on /var; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 2
The filesystem on /dev/mapper/vg.flex-lv.var is now 3144704 (4k) blocks long.
```

Show the filesystem capacity/usage:

```bash
df -h /var
```

Output:

```bash
Filesystem                  Size  Used Avail Use% Mounted on
/dev/mapper/vg.flex-lv.var   12G  217M   12G   2% /var
```


[ansible]: https://www.ansible.com/
[cloudinit]: https://cloudinit.readthedocs.io/en/latest/index.html
[exotemplates]: https://www.exoscale.com/templates/
[gh-exoscale-packer-builder-doc]: https://github.com/exoscale/packer-plugin-exoscale/blob/main/docs/builders/exoscale.md
[gh-exoscale-packer-post-processor-doc]: https://github.com/exoscale/packer-plugin-exoscale/blob/main/docs/post-processors/exoscale-import.md
[packer-builder-file-doc]: https://www.packer.io/docs/builders/file
[packer-builder-qemu-doc]: https://www.packer.io/plugins/builders/qemu
[packer-doc-init]: https://www.packer.io/docs/commands/init
[packer-exoscale-plugin]: https://github.com/exoscale/packer-plugin-exoscale
[packer-qemu]: https://www.packer.io/docs/builders/qemu.htm
[packer]: https://www.packer.io/
[packerex]: https://github.com/exoscale/packer-examples
[puppet]: https://puppet.com/
[snapshottemplatepost]: https://www.exoscale.com/syslog/exporting-snapshots-from-exoscale-registering-as-custom-template/
[zone]: https://www.exoscale.com/datacenters/