Azure Red Hat OpenShift (ARO) Persistent Volume on Azure Files

What is Red Hat OpenShift? What are Persistent Volumes? Why should I choose to provision in Azure files?

Openshift — Switching to a robust storage

To anyone that isn’t familiar with Red Hat OpenShift —

It is a unified platform to build, modernize, and deploy applications at scale. ref

Azure Red Hat OpenShift extends Kubernetes. Running containers in production with Kubernetes requires additional tools and resources. This often includes needing to juggle image registries, storage management, networking solutions, and logging and monitoring tools — all of which must be versioned and tested together. Building container-based applications requires even more integration work with middleware, frameworks, databases, and CI/CD tools. Azure Red Hat OpenShift combines all this into a single platform, bringing ease of operations to IT teams while giving application teams what they need to execute. ref

TL;DR Red Hat OpenShift features an advanced developer experience including automated image builds, built-in image registry, options for Enterprise users including support, patching and more on top of a Kubernetes-like based container orchestration platform.

Azure Red Hat OpenShift offers a fully functional OpenShift cluster running and managed on the Azure cloud.

What are Persistent Volumes? Persistent volumes are storage pieces represented resources based on the cluster to be used by a workload (pod/deployment/dc). They are provisioned either manually by an admin or by a storageClass.

Then, what is a storageClass? storageClass resources defines the storage type which allows the user to request a type of storage for the above explained workloads. By default, Azure Red Hat OpenShift provisions Azure Disks as the default provisioner. That means that when a PV is created it will automatically provision an Azure disk on the cloud platform.

Then, why should i choose to provision on Azure Files?

1. Support the following access modes — ReadWriteOnce, ReadOnlyMany, ReadWriteMany.
2. Can be shared /across multiple compute resources (NFS, SMB, Rest API).
3. Easy to handle content through Azure portal/CLI/API, support Shared Access Signatures tokens and Azure identity features.

In the following tutorial i will explain how to create a connection between Azure Red Hat Openshift to Azure Files in an Azure Storage Account shown by this architecture below:

Prerequisites

• Azure Subscription
• Azure Storage Account
• Azure Red Hat Openshift Cluster

Setting up a test web app

## Create a new project named poc1
oc new-project poc1
## Create a new s2i deploymentConfig based on nodejs.
oc new-app https://github.com/nodeshift-starters/devfile-sample.git --as-deployment-config=true
## If the pull from github fails make sure the 80/443 to and from github is open on the NSG.
## Track the build process, wait for the build to finish: 
oc logs -f buildconfig/devfile-sample
## Exposing the service
oc expose service/devfile-sample

Setting up the variables

AZURE_FILES_RESOURCE_GROUP=<storage_account_rg>
LOCATION=<region>
SUB_ID=<your_sub_id>
AZURE_STORAGE_ACCOUNT_NAME=<storage_account_name>
ARO_RESOURCE_GROUP=<aro_rg_here>
CLUSTER_NAME=<aro_cluster_name>
ARO_SERVICE_PRINCIPAL_ID=$(az aro show -g $ARO_RESOURCE_GROUP -n $CLUSTER_NAME --query servicePrincipalProfile.clientId -o tsv)
ARO_API_SERVER=$(az aro list --query "[?contains(name,'$CLUSTER_NAME')].[apiserverProfile.url]" -o tsv)
SECRET_NAME=secret-$AZURE_STORAGE_ACCOUNT_NAME
FILESHARE_NAME=<fileshare_name>

Setting up permissions

# Assign contributor role to the ARO SP on SA resource group
az role assignment create --role Contributor --scope /subscriptions/$SUB_ID/resourceGroups/$AZURE_FILES_RESOURCE_GROUP --assignee $ARO_SERVICE_PRINCIPAL_ID

## login to the ARO Cluster
oc login -u kubeadmin -p $(az aro list-credentials -g $ARO_RESOURCE_GROUP -n $CLUSTER_NAME --query=kubeadminPassword -o tsv) $ARO_API_SERVER
## Create a cluster role for the secret reader
oc create clusterrole azure-secret-reader --verb=create,get --resource=secrets
oc adm policy add-cluster-role-to-user azure-secret-reader system:serviceaccount:kube-system:persistent-volume-binder

Creating the storageClass for Azure Files provisioning

cat << EOF >> azure-storageclass-azure-file.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: azure-file-storageclass
provisioner: kubernetes.io/azure-file
parameters:
  location: $LOCATION
  secretNamespace: kube-system
  skuName: Standard_LRS
  storageAccount: $AZURE_STORAGE_ACCOUNT_NAME
  resourceGroup: $AZURE_FILES_RESOURCE_GROUP
reclaimPolicy: Retain
volumeBindingMode: Immediate
EOF

oc create -f azure-storageclass-azure-file.yaml

Changing the default storageClass used

oc patch storageclass managed-premium -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
oc patch storageclass azure-file-storageclass -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
# Check the storageClass with
oc describe sc/$azure-file-storageclass

In case the ARO is private, and the storage account isn’t public and is integrated to a VNET then you will need to authorize connection from the ARO Vnet to the Storage Account’s allowed VNET list as shown on the image below

Create a Secret for the storage account key

oc create secret generic $SECRET_NAME --from-literal=azurestorageaccountname=$AZURE_STORAGE_ACCOUNT_NAME --from-literal=azurestorageaccountkey=’<storage-account-key>’
## must have ‘ before and after
## on the above command please copy the storage account key from the portal.

Create a new file share in the Azure portal

Create a new persistent volume

cat << EOF >> pv-fileshare.yaml
apiVersion: "v1"
kind: "PersistentVolume"
metadata:
  name: "pv-fileshare" 
spec:
  capacity:
    storage: "3Gi" 
  accessModes:
    - "ReadWriteMany"
  storageClassName: azure-file-storageclass
  azureFile:
    secretName: $SECRET_NAME
    shareName: $FILESHARE_NAME
    readOnly: false
EOF

oc create -f pv-fileshare.yaml

Create a new persistent volume claim

cat << EOF >> pvc-fileshare.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: "pvc-fileshare"
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: azure-file-storageclass
  resources:
     requests:
       storage: 3Gi
  storageClassName: azure-file-storageclass
  volumeName: "pv-fileshare"
EOF

oc create -f pvc-fileshare.yaml

#Check that the claim is bound successfully to the pv
oc get pvc

Setting the mountpoint on the DeploymentConfig

oc set volume dc/devfile-sample --add --name=pv-fileshare1 -t pvc-fileshare1 --claim-size=3G -m /data --claim-class='azure-file-storageclass'

Testing the mount

export POD=$(oc get pods --field-selector=status.phase==Running -o jsonpath={.items[].metadata.name})
oc exec $POD -- bash -c "echo 'azure file storage' >> /data/test.txt"

Next step is to check on the portal that the file exists

Troubleshooting common connection issues: Check connection from the pod to the file share

nc -zvw3 xxxx.file.core.windows.net 445

Check NSGs on both subnets in each Vnet/Interface. If placed on different Vnets make sure Vnet peering is connected.

That is All. If you liked this story, please show your support by using the 👏 button below this story. Thanks for reading!