Managing the Physical Infrastructure of Telco Cloud

When it comes to cloud computing and virtualization, most management paradigms focus on managing the workloads, whether they are in the form of virtual machines (VM), Kubernetes Pods or some other form of application virtualization. However, managing the underlying physical infrastructure on top of which the virtualization and workload orchestration magic takes place to make cloud a reality is crucial when dealing with a complete top-to-bottom management of on-prem clouds.

Telco Cloud deployments are widely based on the Networks Function Virtualization (NFV) technology. The NFV Management and Orchestration (NFV-MANO) framework offers a way to orchestrate and manage network services and functions in the form of VMs or Kubernetes Pods that are deployed on the NFV infrastructure (NFVI), also referred sometimes as Telco Cloud infrastructure. The physical infrastructure management (PIM) feature in NFV-MANO enables network operators to control, manage, and monitor hardware resources that make up the Telco Cloud infrastructure.

Why the need for PIM?

Previously, the NFV framework (and its technical specifications developed and maintained by ETSI ISG NFV) did not go into too much detail of the NFVI in terms of its composition, its hardware resources and their fine-grained control and management. Rather the orchestration and management aspects of NFV-MANO relied on the assumption that the NFVI physical resources were already set up, provisioned, configured and ready for use by higher layer, specialized infrastructure managers such as the Virtualised Infrastructure Manager (VIM), e.g. OpenStack Nova offering VM management, Container Infrastructure Service Manager (CISM), e.g., Kubernetes Control Plane offering containerized workload management, and CIS Cluster Manager (CCM), e.g., Kubernetes Cluster API offering cluster management.

However, a detailed framework for managing physical resources is necessary for enabling advanced NFV features and deployment scenarios, such as energy efficiency (Green NFV), virtualised RAN (vRAN) use cases, bare-metal Kubernetes deployments etc.

PIM function inside the NFV-MANO architectural framework

Technical aspects of the PIM Feature

A comprehensive support for PIM has been introduced in the NFV technical framework and its specifications under a Release 5 feature on physical infrastructure management (referred to as FEAT33: PIM inside NFV standards community). The following technical aspects summarize the PIM support inside NFV-MANO.

PIM specifications

First and foremost, detailed requirements and interface specifications for PIM are introduced in a brand new Release 5 standard document, ETSI GS NFV-IFA 053 (latest version v5.2.1). ETSI GS NFV-IFA 053, hereafter referred to as IFA053 for ease, defines detailed requirements related to managing physical infrastructure in light of NFV use cases, specifies a standard PIM function, its interfaces, and requirements related to each interface. There are six separate interfaces specified that sum up the main functionality provided by the standard PIM function:

1. Physical resource provisioning and lifecycle management interface
2. Physical resource inventory management interface
3. Physical resource topology management interface
4. Physical resource performance management interface
5. Physical resource fault management interface
6. Physical resource log management interface

A high-level information model and analysis of relevant solutions like OpenStack Ironic, Metal3 and DMTF’s Redfish standard is also included in the IFA053 standard specification.

Integration of PIM inside the NFV-MANO

The PIM function is integrated into the NFV-MANO framework in order to enable automated and standardized way to holistically manage physical infrastructure for NFV-based Telco Cloud deployments. Higher layer management functions like VIM, CCM, NFV Orchestrator (NFVO) and other operations & business support systems (OSS/BSS) in the telecom network can invoke PIM operations over its interfaces. A thorough normative work has been done to integrate PIM into the NFV-MANO stack.

1. PIM as an NFV-MANO function: The PIM function has been added as a logical function within NFV-MANO architectural framework in ETSI GS NFV 006 v5.2.1 along with corresponding functional requirements in ETSI GS NFV-IFA 010 v5.2.1.
2. Interactions between PIM and other NFV-MANO entities: ETSI GS NFV-IFA 010 also specifies interactions between the PIM and other NFV-MANO entities like NFVO, VIM, CCM, WIM for holistic management of underlying physical resources.
3. Interactions between PIM and non NFV-MANO entities: Due to SBA-like approach to specifying the PIM function and its service interfaces, the interface consumers are not limited to NFV-MANO entities. Non NFV-MANO entities like operator’s OSS/BSS may also invoke management operations related to physical infrastructure by directly interacting with the PIM and utilize its services as needed.

Interactions between the three infrastructure managers of NFV-MANO

Measurements and fault/alarms related to physical resources

Performance measurements, faults and alarm models associated with physical resources managed by the PIM function are important in order to complete the support for physical infrastructure management in NFV. Currently, performance measurements related to physical compute, networking, storage resources along with their power and energy consumption related measurements are being specified in respective standard documents.

Protocol and Data Model specification of PIM APIs

As comprehensive as the IFA053 specification is in terms of PIM requirements and interface specifications, it does not offer detailed protocol and data model specification for PIM service interfaces. A new work item SOL026 aims to specify these API level details, e.g., HTTP endpoints, supported methods, request and response schema etc., based on pre-existing standard (like Redfish) and de-facto open source solutions (like Ironic).

Benefits of PIM

A standard approach to managing physical infrastructure in the Telco Cloud offers network operators and their orchestration & management solutions a fine-grained control of underlying hardware resources, enabling advanced use cases and deployment scenarios, e.g., energy efficiency, vRAN deployments etc. Detailed standard PIM specifications allow telco operators and service providers flexibility in choosing standards-compliant products during procurement, and at the same time offer vendors to support latest standard requirements, features and APIs in their products and solutions.

On the technical side, introduction of the PIM feature in NFV-MANO offers the following advantages (non-exhaustive list):

1. Full stack Telco Cloud automation: via enabling automated infrastructure management to orchestrate Telco Cloud workloads using standard services, interfaces and interactions between different management entities.
2. Energy efficient NFV deployments: via maintaining resource pools of physical resources according to their power profiles, controlling power states of hardware resources, monitoring and providing power/energy consumption information etc.
3. Bare-metal Kubernetes clusters: via setting up (composing), provisioning, configuring servers as nodes for bare metal CIS (Kubernetes) clusters.

Challenges and Learning Experiences

Introduction of such a fundamental feature as physical infrastructure management in a mature technical framework (NFV framework was already 10 year old when FEAT33 started) raised some challenges.

The first challenge was related to the selection of the right approach for adding the physical infrastructure management functionality within the NFV-MANO framework, i.e., whether to enhance an pre-existing management entity like VIM or add a new entity. After some evaluation, it was decided to add a new logical function called the PIM in NFV-MANO. Secondly, the choice of interface design related to PIM interfaces was debated and eventually an SBA-like approach was used to specify PIM service interfaces. Annex A in the IFA053 standard document summarizes the architectural analysis that was carried out before normative provisions were added for PIM.

A major challenge was to integrate PIM inside NFV-MANO in terms of clarifying management scopes amongst different infrastructure managers, aligning terminologies, defining interactions between different entities etc. I like to think of this whole exercise as a kind of like a reverse Jenga move, where you have to carefully lift the whole stack up and slide one fundamental layer under it without letting the above pieces fall apart… but it was fun!

I had the privilege of serving as Feature Prime for this significant feature in NFV. I also served as the rapporteur of the new specification document for PIM, ETSI GS NFV-IFA 053. Leading the development of this feature over the past two years has been a fulfilling journey. This is the first comprehensive feature that I led and developed in a standards or opensource ecosystem. I look forward to developing more cutting-edge solutions for cloud management and Telco modernization.