Storage

Local Disks

By default, each server node is attached with a fixed amount of local disk space under "/dev/sda". You can check the total amount of disk space by running:

root@NODE:~# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda      8:0    0 446.6G  0 disk 
|-sda1   8:1    0   150G  0 part /
`-sda2   8:2    0   250G  0 part /mnt/sda2

Per different setups of each disk image, the size of actual mounted disk space under "/dev/sda1" may vary. You can mount extra disk space (if available) to any mount points under the file system of your server node using fdisk command:

root@NODE:~# fdisk /dev/sda

Welcome to fdisk (util-linux 2.37.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): ...

NFS Mounting

Since local storage space is limited, large files (e.g. datasets and trained models) are better kept remotely. This can be done efficiently with the help of Network File System (NFS), which allows you to mount remote directories and do read/write as if they are local directories.

Below are two common use cases of NFS.

COSMOS Storage

The easiest way is to use the storage spaces provided by the COSMOS ecosystem: "cosmos_data" and "datasets".

To mount these,

1. Install ​nfs-common:

   apt-get update
   apt-get install nfs-common
   

2. Open "/etc/fstab" and append (NOT overwrite with) the following lines:

   10.0.0.41:/datasets     /mnt/datasets   nfs     sec=sys,nfsvers=4.1,rw,defaults      0   0
   10.108.0.21:/cosmos_data     /mnt/cosmos_data   nfs     sec=sys,nfsvers=4.1,rw,defaults      0   0
   

3. Create mounting points under your local file system:

   mkdir /mnt/datasets
   mkdir /mnt/cosmos_data
   

4. Mount the NFS:

   mount -a
   

5. Check the mounted NFS storage space using df -h command:

   root@NODE:~# df -h
   Filesystem                Size  Used Avail Use% Mounted on
   /dev/sda1                 148G  108G   33G  77% /
   /dev/sda2                 246G  246G     0 100% /mnt/sda2
   tmpfs                      95G   65M   94G   1% /dev/shm
   10.108.0.21:/cosmos_data  5.1T  931G  4.1T  19% /mnt/cosmos_data
   10.0.0.41:/datasets       2.1T  284G  1.8T  14% /mnt/datasets
   

Google Cloud Buckets

Apart from native COSMOS storage, you may also mount external NFS like Google Cloud buckets.

To do so,

1. Prepare an alive project and a GCS bucket
2. Install Google Cloud CLI and GCSFuse
3. In your console, login to your GCP account and configure the project
4. Create mounting point and mount the bucket

Find and follow detailed instructions in ​here.