Skip to content

Latest commit

 

History

History
874 lines (661 loc) · 36 KB

manual_deployment_guide.md

File metadata and controls

874 lines (661 loc) · 36 KB

Manual Deployment Guide

Version History

Version Date Changes
1.0 2025-05-30 First version with PCI passthrough of complete PF and PTP4L running in VM
2.0 2025-06-13 Updated architecture: PTP4L Grandmaster runs on host, VM uses VF instead of complete PF, VM synchronizes via ptp_kvm
3.0 2025-07-04 Added RU LiteON configuration and verification section, updated Table of Contents positioning, refined deployment workflow
4.0 2025-07-07 Updated open5GS sections to reference install_open5gs.sh script automation, added WebUI access configuration, integrated VM appliances information in Future Automation section
5.0 2025-07-15 Updated PTP service architecture: host now runs both ptp4l.service and phc2sys.service, corrected service references throughout documentation, updated monitoring and troubleshooting commands

Introduction

This document provides a comprehensive manual deployment guide for setting up a complete 5G O-RAN private network infrastructure. This guide covers the manual deployment process that will eventually be automated using OneDeploy in another document.

The deployment process involves multiple components working together to create a functional 5G network, including virtualized network functions, precise timing synchronization, and hardware integration.

Deployment Architecture Overview

Target Setup Description

The deployment consists of the following main components:

KVM OpenNebula Server Node

A physical server running KVM hypervisor with OpenNebula cloud management platform that hosts:

  • srsRAN CU/DU Virtual Machine: Handles the Central Unit and Distributed Unit functions of the 5G O-RAN
  • open5GS Virtual Machine: Provides the 5G Core Network functionality

External Hardware Components

  • RU LiteON: Radio Unit for RF transmission and reception
  • 5G UE (User Equipment): End-user devices such as smartphones, IoT devices, and other 5G-capable equipment that connect to the network through the radio interface

Critical Hardware Requirements

  • High-Precision PTP NIC Card: The KVM host server must be equipped with a Network Interface Card that supports high-quality PTP (Precision Time Protocol) synchronization with nanosecond-level accuracy. This is essential for proper 5G network timing requirements. For example, the Intel NIC E810 has been tested and proven to work well for this purpose.
  • PTP Grandmaster Architecture: The KVM host server runs the PTP4L grandmaster service on the Physical Function (PF) of the NIC connected to the RU LiteON, providing precise timing synchronization. The srsRAN VM receives a Virtual Function (VF) from this PF via SR-IOV, allowing direct hardware access while maintaining the PTP synchronization capabilities.
  • VM Clock Synchronization: The srsRAN VM synchronizes its system clock with the KVM host clock using ptp_kvm mechanism, ensuring nanosecond-level timing accuracy throughout the virtualized infrastructure.

Network Architecture

┌─────────────────────────────────────────────────────────────┐
│                    KVM OpenNebula Host                      │
│                   PTP4L Grandmaster                         │
│  ┌─────────────────┐         ┌─────────────────────────┐    │
│  │   srsRAN VM     │         │      open5GS VM         │    │
│  │   (CU/DU)       │◄────────┤    (5G Core)            │    │
│  │ 32 CPU/32GB RAM │         │  8 CPU/16GB RAM         │    │
│  │   VF (SR-IOV)   │         │                         │    │
│  │   ptp_kvm sync  │         │                         │    │
│  └─────────────────┘         └─────────────────────────┘    │
│           │                                                 │
│           │ VF from PF (SR-IOV)                             │
│           │ Host: PTP4L on PF                               │
└───────────┼─────────────────────────────────────────────────┘
            │ PF with PTP4L
            │
    ┌───────▼──────┐              ┌─────────────┐
    │  RU Liteon   │              │   5G UE     │
    │  (Radio Unit)│◄─────────────┤ (Smartphones│
    └──────────────┘   RF Link    │  Devices,   │
                                  │  IoT, etc.) │
                                  └─────────────┘

Table of Contents

  1. OpenNebula Installation
  2. KVM Host Configuration
  3. RU LiteON Configuration and Verification
  4. Virtual Machine Instantiation
  5. srsRAN Virtual Machine Configuration
  6. open5GS Virtual Machine Configuration
  7. Testing and Validation
  8. Troubleshooting and Maintenance
  9. Future Automation
  10. References
  11. Appendices

1. OpenNebula Installation

This deployment uses a simple OpenNebula setup consisting of:

  • 1 OpenNebula Front-end: Management and orchestration node
  • 1 KVM Host: Physical server that will host the srsRAN and open5GS virtual machines

For OpenNebula installation, refer to the official OpenNebula documentation:

2. KVM Host Configuration

2.1 Software Prerequisites

Before configuring the KVM host, ensure the following software components are installed:

2.1.1 Intel ice Driver Installation

Install Intel ice driver version 1.17.2 for the Intel E810 NIC:

# Install Intel ice driver for Intel E810 NIC
sudo ./scripts/install/install_ice_driver.sh

This driver is required for:

  • SR-IOV Virtual Function (VF) creation
  • Hardware timestamping support for PTP
  • High-performance network operations

2.1.2 LinuxPTP Installation

Install LinuxPTP version 4.3 for PTP4L grandmaster functionality:

# Install LinuxPTP for PTP4L grandmaster
sudo ./scripts/install/install_linuxptp.sh

LinuxPTP provides:

  • PTP4L grandmaster service for precise timing
  • Hardware clock synchronization capabilities
  • Integration with Intel E810 NIC timestamping

Note: These software prerequisites must be installed before proceeding with SR-IOV configuration and PTP grandmaster setup.

2.2 SR-IOV and Virtual Function Configuration

Configure SR-IOV to provide the srsRAN VM with a Virtual Function (VF) from the high-precision NIC card while keeping the Physical Function (PF) on the host for PTP4L grandmaster:

  1. Enable IOMMU in BIOS/UEFI (Manual Step)

  2. Configure GRUB parameters (Manual Step)

    # Edit GRUB configuration
sudo nano /etc/default/grub

# Add or modify GRUB_CMDLINE_LINUX_DEFAULT line:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=on iommu=pt"

# Update GRUB and reboot
sudo update-grub
sudo reboot

  3. Enable SR-IOV on the NIC (Automated by script)

  4. Create Virtual Functions (VFs) (Automated by script)

  5. Configure VF passthrough to srsRAN VM (Automated by script)

  6. Keep PF on host for PTP4L grandmaster (Automated by script)

  7. Configure VF for VM PCI passthrough (Manual Step)

# Configure SR-IOV and VF setup (Steps 3-6 automated)
sudo ./scripts/install/install_host_vf_setup.sh

The script automatically performs the following steps:

Step 3 - Enable SR-IOV on the NIC:

  • Automatically detects Intel E810 interfaces
  • Verifies ice driver is loaded
  • Checks system requirements for SR-IOV

Step 4 - Create Virtual Functions (VFs):

  • Creates 1 Virtual Function on the Intel E810 NIC
  • Resets existing VFs if present
  • Verifies VF creation success

Step 5 - Configure VF for VM passthrough:

  • Sets VF trust mode ON
  • Disables spoof checking for VF
  • Enables promiscuous mode on Physical Function
  • Sets specific MAC address for the VF: a2:b3:a6:4e:de:49
  • Configures ring buffer optimization (TX/RX: 4096 each)
  • Enables VF true promiscuous support

Step 6 - Keep PF on host for PTP4L grandmaster:

  • Physical Function remains on KVM host
  • PF will be used for PTP4L grandmaster service
  • VF PCI address is provided for OpenNebula VM passthrough configuration

Script Output:

  • VF PCI address for VM passthrough configuration
  • Environment variables for reference
  • Verification commands
  • Configuration summary

Reference: KVM PCI Passthrough Documentation

Note: Detailed configuration scripts are available in scripts/install/ directory

2.3 PTP Grandmaster Setup on KVM Host

Configure PTP grandmaster synchronization on the KVM host server. This PTP grandmaster will synchronize the RU Liteon connected through the Physical Function (PF) of the Intel E810 NIC, providing precise timing reference for the entire 5G network.

Set up PTP4L grandmaster service on the KVM host:

# Configure PTP4L grandmaster on KVM host with Intel E810 NIC
sudo ./scripts/install/install_host_ptp_grandmaster.sh

This script configures:

  • PTP4L Grandmaster: Runs on the PF (Physical Function) of the Intel E810 NIC connected to RU Liteon
  • phc2sys Service: Synchronizes system clock with NIC hardware clock
  • GNSS Synchronization: Uses integrated GNSS receiver for time reference
  • Hardware Timestamping: Enables precise timing on the host NIC
  • Systemd Services: Creates persistent PTP grandmaster and phc2sys services

2.4 KVM Host Performance Optimization

2.4.1 Performance Optimization Script

Use the automated performance optimization script:

# Optimize KVM host for low-latency 5G workloads
sudo ./scripts/performance/optimize_kvm_host.sh

This script automatically applies the following optimizations:

  • CPU Frequency Governor: Sets to performance mode across all cores
  • Kernel Parameters: Configures C-states, P-states, and debugging features for low latency
  • Real-time Kernel Detection: Checks for RT kernel installation and provides guidance
  • Runtime Optimizations: Creates systemd service for persistent performance tuning
  • DRM KMS Polling: Disables to reduce unnecessary interrupts

2.4.2 Real-time Kernel Installation

Install real-time kernel for optimal performance:

# Install RT kernel with PREEMPT_RT patch
sudo ./scripts/install/install_rt_kernel.sh

Choose installation method:

  • From source: Full compilation (30-60 minutes)
  • From packages: Fast installation using pre-built .deb files

2.4.3 KVM Host Performance Characteristics

The optimized KVM host provides:

  • CPU Pinning: Physical CPUs are allocated to VMs to minimize contention and ensure consistent performance. The srsRAN VM is assigned 32 physical CPUs to handle compute-intensive tasks.

  • IOMMU & PCI Passthrough:

    • IOMMU is enabled (intel_iommu=on) to support direct hardware access
    • Intel E810-C NIC is passed through to the srsRAN VM via PCI-PT for fronthaul communication
    • PTP grandmaster runs on host to provide < 20 ns synchronization accuracy to RU Liteon

  • Kernel Optimization: Real-time kernel with CONFIG_PREEMPT_RT enabled to prioritize time-critical tasks and reduce scheduling jitter

  • Network Buffer Tuning: Increased network buffer sizes via sysctl to prevent packet drops under high load

For detailed optimization information, see scripts/performance/README.md

3. RU LiteON Configuration and Verification

This section covers the configuration and verification of the RU LiteON radio unit. The RU must be properly configured and synchronized before proceeding with virtual machine instantiation.

3.1 Prerequisites and Connection

This guide covers the configuration of the RU LiteON radio unit. Before proceeding with the configuration steps, ensure that a connection to the RU LiteON has been established.

Prerequisites:

  • Physical network connectivity between KVM host and RU LiteON established
  • RU LiteON powered on and accessible
  • Management credentials for RU LiteON available

3.2 RU Configuration

For detailed RU LiteON configuration, the manufacturer's official command guide should be followed.

A quick configuration guide would be:

  • With the enable command we access privileged mode. Then with the configure command we can configure the different RU parameters.
  • The important parameters that we must change to match those configured in the core and srsRAN are: bandwidth, center-frequency, compression-bit, du-mac-address, jumboframe, phasecomp-mode, sync-source and c/u-plane-vlan.

Example configuration:

(config)# bandwidth 100000000
Old Bandwidth 20000000
New Bandwidth 100000000

(config)# compression-bit 9
Old Compression Bit = 8
New Compression Bit = 9

(config)# du-mac-address a2b3a64ede49
Old DU MAC Address = 001122334466
New DU MAC Address = a2b3a64ede49

(config)# jumboframe 1
Old jumboframe = 0x00000000
New jumboframe = 0x00000001

(config)# phasecomp-mode true
phase compensation mode : Enable

(config)# slot-id 1
Old slotid = 0x00000002
New slotid = 0x00000001

(config)# sync-source PTP
sync source : PTP
Active after reboot

(config)# c/u-plane-vlan
control and user Plane vlan = 564

(config)# center-frequency
Center Frequency = 3850020000

Configuration verification:

To verify that the configuration has been applied correctly:

  • Exit config mode with exit
  • View the current configuration with show running_config

Important notes:

  • The MAC address du-mac-address must match the one configured in the KVM host VF (a2:b3:a6:4e:de:49)
  • The sync-source PTP must be configured to synchronize with the KVM host PTP grandmaster
  • The frequency and bandwidth parameters must match the srsRAN configuration

3.3 PTP Synchronization Verification

After configuration, verify that the RU LiteON is properly synchronized with the PTP grandmaster running on the KVM host.

The RU should now be power-cycled. Once completed successfully, the RU state can be checked with:

# show oru-status
Sync State  : SYNCHRONIZED
RF State    : Ready
DPD         : Ready
DuConnected : notReady

Expected Status:

  • Sync State: Should show "SYNCHRONIZED" indicating proper PTP synchronization
  • RF State: Should show "Ready" indicating RF components are operational
  • DPD: Should show "Ready" indicating Digital Pre-Distortion is functional
  • DuConnected: Will show "notReady" until srsRAN CU/DU is configured and connected

Note: Do not proceed to VM instantiation until the RU shows proper synchronization status. The "DuConnected" status will remain "notReady" until srsRAN is configured and a UE attaches to the network.

4. Virtual Machine Instantiation

4.1 Virtual Machine Specifications

srsRAN Virtual Machine Requirements

  • CPU: 32 vCPUs
  • RAM: 32 GB
  • Storage: 60 GB
  • Operating System: Ubuntu 24.04 LTS (recommended)
  • Network: Multiple network interfaces for management and data plane
  • Hardware: VF (Virtual Function) from Intel E810 NIC via SR-IOV passthrough

open5GS Virtual Machine Recommendations

  • CPU: 8 vCPUs
  • RAM: 16 GB
  • Storage: 40 GB
  • Operating System: Ubuntu 24.04 LTS (recommended)
  • Network: Management and data plane interfaces

4.2 VM Instantiation Process

  1. Prerequisites: Ensure KVM host configuration is complete (Section 2) and RU LiteON verification is successful (Section 3)
  2. Create VM templates with the specified resource requirements
  3. Configure network templates for management and data networks
  4. Configure SR-IOV VF passthrough for srsRAN VM using the VF created in Section 2.2
  5. Instantiate VMs using OpenNebula Sunstone or CLI
  6. Verify VM deployment and connectivity
  7. Verify VF assignment to srsRAN VM

Important: The srsRAN VM requires the Virtual Function (VF) configured in Section 2.2. Do not instantiate the srsRAN VM until the KVM host SR-IOV configuration is complete and RU LiteON verification passes.

5. srsRAN Virtual Machine Configuration

5.1 Network Interface and PTP Synchronization

5.1.1 NIC Driver Installation

Install Intel iavf driver for the Virtual Function (VF) passed through to the VM:

# Install Intel iavf driver for VF
sudo ./scripts/install/install_iavf_driver.sh

Note: The Intel ice driver is not required in the VM since it only receives a Virtual Function (VF) via SR-IOV passthrough. The Physical Function (PF) remains on the KVM host with the ice driver for PTP4L grandmaster functionality. The VM needs the iavf (Intel Adaptive Virtual Function) driver to properly handle the VF.

The script automatically:

  • Checks and installs required dependencies (build-essential, kernel headers, gcc, make)
  • Extracts and compiles Intel iavf driver version 4.13.3 from source
  • Loads iavf kernel module and removes old versions if present
  • Verifies VF detection and configuration
  • Shows available network interfaces and Virtual Functionsnel module
  • Verify VF detection and configuration

Driver packages available in sources/drivers/ directory

5.1.2 PTP Synchronization Setup

Install LinuxPTP:

# Install LinuxPTP version 4.3-dirty from source
sudo ./scripts/install/install_linuxptp.sh

Configure ptp_kvm for VM clock synchronization:

# Set up ptp_kvm for VM system clock synchronization
sudo ./scripts/install/install_vm_sync.sh

The VM PTP setup script:

  • Loads the ptp_kvm kernel module which creates a virtual PTP device (/dev/ptp_kvm) in the VM
  • Configures phc2sys service to use the ptp_kvm device for VM system clock synchronization
  • Synchronizes VM system clock with the KVM host clock through the virtual PTP device
  • Creates and enables systemd service for automatic startup
  • Stops conflicting time synchronization services

Note: PTP4L grandmaster runs on the KVM host server (configured in Section 2.3), not in the VM. The VM uses the ptp_kvm kernel module combined with phc2sys to synchronize its system clock directly with the KVM host clock, which is itself synchronized with the NIC's hardware clock via PTP4L grandmaster.

ptp_kvm + phc2sys Architecture:

  • Host: Runs PTP4L grandmaster on PF and phc2sys to synchronize host system clock
  • VM: Uses ptp_kvm module to create virtual PTP device, then phc2sys synchronizes VM system clock using this device
  • Communication: Direct host-VM clock synchronization through virtual PTP device
  • Timing Accuracy: < 100 ns between host and VM clocks

Reference configuration files:

  • configs/ptp/services/phc2sys.service - VM clock synchronization service

5.2 Network Configuration

Configure network interfaces for:

  • Management network connectivity
  • Data plane connectivity with open5GS core (data plane N3 and control plane N2)
  • Fronthaul interface for RU communication (for DPDK case, use script scripts/performance/configure_dpdk_interface.sh)

DPDK Interface Configuration Script: The configure_dpdk_interface.sh script automates the configuration of DPDK drivers and network interfaces for high-performance packet processing. It:

  • Binds the iavf driver to the specified PCI device (0000:01:01.0)
  • Configures network interface settings (promiscuous mode, allmulticast, ring buffers)
  • Installs and loads igb_uio driver for DPDK compatibility
  • Verifies the configuration and provides system status information

5.3 srsRAN Installation and Configuration

5.3.1 Installation

Install srsRAN Project for Split 7.2 configuration:

# Install srsRAN Project with optional DPDK support
sudo ./scripts/install/install_srsran.sh

This script:

  • Checks and installs required dependencies for srsRAN Project
  • Prompts for optional DPDK support (for high-bandwidth scenarios)
  • Compiles srsRAN Project from source (srsRAN_Project-release_24_10_1)
  • Configures for Split 7.2 (no extra 3rd-party dependencies required)
  • Installs binaries to /usr/local and updates system PATH
  • Runs comprehensive tests during build process

Source packages available in sources/software/ directory

5.3.2 srsRAN VM Performance Optimization

Optimize the srsRAN VM for low-latency 5G workloads:

# Optimize srsRAN VM for real-time performance
sudo ./scripts/performance/optimize_vm_srsRAN.sh

This script:

  • Verifies real-time kernel (required for srsRAN timing)
  • Optimizes PTP services with CPU affinity and real-time priority
  • Applies VM-specific performance tuning for memory and network
  • Creates performance service for persistent optimizations

Critical Requirements:

  • Real-time kernel must be installed before running this script
  • PHC2SYS service must be configured and running (PTP4L grandmaster runs on host)

5.3.3 Configuration

Configure srsRAN gNB with appropriate parameters:

  • Radio frequency configuration
  • Network interfaces binding
  • Core network integration parameters

Reference configuration files in configs/srsran/ directory:

  • gnb_config_n77_20mhz_siso.cfg - Basic SISO & 20 MHz configuration
  • gnb_config_n77_100mhz_4x2.cfg - Advanced MIMO & 100 MHz configuration
  • gnb_config_dpdk_*.cfg - DPDK-enabled configurations

Note: These configuration files are thoroughly documented with detailed comments explaining each parameter and its purpose.

6. open5GS Virtual Machine Configuration

6.1 Network Configuration

Configure network interfaces for:

  • Management network access
  • Data plane connectivity with srsRAN CU/DU
  • External connectivity (if required)

6.2 open5GS Installation

Install open5GS 5G core network components using the automated installation script:

# Install open5GS 5G SA Core with automated configuration
sudo ./scripts/install/install_open5gs.sh

This script:

  • Installs MongoDB 8.0 for subscriber data management
  • Installs open5GS latest version with 5G SA Core functions only
  • Configures network parameters (N2 interface, PLMN, TAC)
  • Sets up WebUI for subscriber management with external access
  • Disables 4G/LTE functions (only 5G SA Core is enabled)
  • Configures firewall rules for UE connectivity
  • Starts and enables services in proper order

Interactive Configuration: The script will prompt for:

  • N2 Interface IP: AMF NGAP bind address (default: 127.0.0.5 for local testing)
  • PLMN Configuration: MCC/MNC (default: 999/70 for private networks)
  • TAC: Tracking Area Code (default: 1)
  • WebUI Access: IP and port for external management (default: 0.0.0.0:9999)

Installed Components (5G SA Only):

  • NRF (NF Repository Function)
  • AMF (Access and Mobility Management Function)
  • SMF (Session Management Function)
  • UPF (User Plane Function)
  • AUSF (Authentication Server Function)
  • UDM (Unified Data Management)
  • UDR (Unified Data Repository)
  • PCF (Policy Control Function)
  • NSSF (Network Slice Selection Function)
  • BSF (Binding Support Function)

For detailed installation information, refer to the official open5GS documentation:

6.3 open5GS Configuration for srsRAN Integration

The install_open5gs.sh script (section 6.2) automatically configures most open5GS parameters required for srsRAN integration. However, additional fine-tuning may be required for specific deployment scenarios.

6.3.1 Automated Configuration (via install_open5gs.sh)

The installation script automatically configures:

  • Core Network Functions: All 5G SA components (AMF, SMF, UPF, NRF, UDM, UDR, AUSF, PCF, NSSF, BSF)
  • Network Parameters: N2 interface IP, PLMN (MCC/MNC), TAC configuration
  • Service Integration: Proper service startup order and dependencies
  • Firewall Rules: NAT and routing for UE connectivity

6.3.2 Manual Configuration (if required)

For advanced scenarios or custom deployments, manual configuration may be needed:

AMF Configuration (/etc/open5gs/amf.yaml):

  • N2 interface binding (NGAP) - automatically configured by script
  • PLMN configuration - automatically configured by script
  • TAC (Tracking Area Code) - automatically configured by script
  • Security algorithms and parameters

SMF Configuration (/etc/open5gs/smf.yaml):

  • UPF association and selection
  • PDU session policies
  • QoS flow configuration

UPF Configuration (/etc/open5gs/upf.yaml):

  • Data plane interface configuration
  • Packet forwarding rules
  • QoS enforcement policies

Integration Parameters for srsRAN CU/DU:

  • Network interface bindings - N2 interface automatically configured
  • PLMN (Public Land Mobile Network) configuration - automatically configured
  • Security parameters (encryption/integrity algorithms)
  • QoS policies and traffic management

Note: Most configuration parameters are automatically set by the install_open5gs.sh script based on interactive prompts. Manual configuration is typically only needed for advanced customization or troubleshooting.

6.4 WebUI Access and Subscriber Management

The install_open5gs.sh script (section 6.2) automatically configures the open5GS WebUI to be accessible externally by binding it to 0.0.0.0:9999. This allows remote management of subscribers and network configuration.

6.4.1 WebUI Access

After running the installation script, the WebUI is accessible at:

http://<open5gs-vm-ip>:9999

Default Credentials:

  • Username: admin
  • Password: 1423

6.4.2 UE Registration

Use the WebUI to register User Equipment (UE) in the network:

  1. Access WebUI: Navigate to http://<open5gs-vm-ip>:9999
  2. Login: Use default credentials (admin/1423)
  3. Add Subscriber: Click "Subscriber" → "Create"
  4. Configure UE Parameters:
    • IMSI: International Mobile Subscriber Identity
    • K: Authentication key (128-bit)
    • OPc: Operator code (128-bit)
    • APN: Access Point Name (default: "internet")
    • QoS Profile: Quality of Service settings

6.4.3 Database Management

MongoDB 8.0 is automatically installed and configured by the script for:

  • Subscriber data management
  • Authentication credentials storage
  • Session management data
  • Network function registration data

Database Access (if manual management required):

# Connect to MongoDB
mongo --host localhost --port 27017

# Switch to open5gs database
use open5gs

# View subscribers
db.subscribers.find()

Note: The WebUI provides a user-friendly interface for all subscriber management tasks, eliminating the need for direct database manipulation in most scenarios.

7. Testing and Validation

7.1 System Integration Testing

Verify PTP synchronization accuracy

Check the srsRAN VM clock synchronization by monitoring the phc2sys.service created in section 5.1.2:

# Check VM PTP synchronization status
sudo systemctl status phc2sys.service
sudo journalctl -u phc2sys.service -f

The offset should be less than 500 ns for proper synchronization.

Note: The kvm host also runs its own phc2sys.service (configured in section 2.3) to synchronize the host system clock with the NIC hardware clock.

Test inter-VM connectivity

Verify connectivity between all VMs using ping:

# Test connectivity from srsRAN VM to open5GS VM
ping <open5gs_vm_ip>

# Test connectivity from open5GS VM to srsRAN VM
ping <srsran_vm_ip>

Validate srsRAN and open5GS integration

Check integration status by monitoring logs:

# Check srsRAN logs

# Check open5GS AMF logs in Open5GS VM
sudo journalctl -u open5gs-amfd.service -f

7.2 End-to-End Testing

  • RU Liteon connectivity testing
  • UE (User Equipment) attachment procedures
  • Data plane functionality verification
  • Some testing scripts and procedures available in scripts/testing/ directory*

8. Troubleshooting and Maintenance

8.1 Common Issues

PTP Synchronization Issues

Symptoms: RU shows "NOT SYNCHRONIZED" or high offset values

Check:

# Verify PTP4L grandmaster service on KVM host
sudo systemctl status ptp4l.service
sudo journalctl -u ptp4l.service -f

# Check host clock synchronization (on KVM host)
sudo systemctl status phc2sys.service
sudo journalctl -u phc2sys.service -f

# Check VM PTP synchronization (on VM)
sudo systemctl status phc2sys.service
sudo journalctl -u phc2sys.service -f

# Verify NIC hardware timestamping in KVM host
ethtool -T <interface_name>

SR-IOV VF Issues

Symptoms: VM cannot access VF or network performance issues

Check:

# Verify VF creation on KVM host
ip link show
lspci | grep -i ethernet

# Check VF configuration
ip link show <pf_interface>

# Verify iavf driver in VM
lsmod | grep iavf
dmesg | grep iavf

srsRAN Connection Issues

Symptoms: RU shows "DuConnected: notReady" or no UE attachment

Note: It is normal for "DuConnected: notReady" to appear when no mobile devices are attached to the network.

Check:

# Verify srsRAN service status
sudo systemctl status srsran
sudo journalctl -u srsran -f

# Check network connectivity to RU
ping <ru_ip_address>

# Verify configuration files
cat /etc/srsran/gnb.cfg

open5GS Core Issues

Symptoms: UE cannot register or no data connectivity

Check:

# Check all open5GS services
sudo systemctl status open5gs-*

# Monitor AMF logs
sudo journalctl -u open5gs-amfd.service -f

# Check UPF connectivity
sudo journalctl -u open5gs-upfd.service -f

# Verify subscriber database
mongo --eval "db.subscribers.find()"

VM Performance Issues

Symptoms: High latency or packet drops

Check:

# Verify CPU pinning and isolation
cat /proc/cmdline
taskset -cp <pid>

# Check real-time kernel
uname -r | grep rt

8.2 Monitoring and Logging

System Health Monitoring

# Monitor all critical services
sudo systemctl status ptp4l.service phc2sys.service srsran open5gs-*

# Check system resources
free -h
df -h
iostat 1 5

Log Collection

# Collect PTP logs
sudo journalctl -u ptp4l.service --since "1 hour ago" > ptp_logs.txt

# Collect srsRAN logs
sudo journalctl -u srsran --since "1 hour ago" > srsran_logs.txt

# Collect open5GS logs
sudo journalctl -u open5gs-* --since "1 hour ago" > open5gs_logs.txt

9. Future Automation

This manual deployment guide will remain as the reference for manual deployment procedures. A separate automation guide will be developed that leverages the procedures documented here to create automated deployment solutions using:

  • OneDeploy: Deployment orchestration
  • OneDeploy: OpenNebula automated deployment tool

The project includes pre-configured VM appliances in the automation/appliances/ directory that integrate srsRAN and open5GS components for automated deployment:

  • srsRAN Appliance: Pre-configured gNB with optimized performance
  • open5GS Appliance: 5G SA Core with automated subscriber management

These appliances provide ready-to-deploy templates compatible with OpenNebula and reduce deployment time significantly. The manual procedures documented in this guide will serve as the foundation for creating automated playbooks and infrastructure definitions in the dedicated automation guide, with particular focus on appliance-based deployment strategies.

10. References

11. Appendices

Appendix A: Configuration Files

All configuration files referenced in this guide are available in the configs/ directory of this project.

Appendix B: Installation and Performance Scripts

Installation and performance optimization scripts are available in the scripts/ directory:

Installation Scripts (scripts/install/):

  • install_rt_kernel.sh - Real-time kernel installation (source or packages)
  • install_ice_driver.sh - Intel ice driver v1.17.2 installation
  • install_linuxptp.sh - LinuxPTP v4.3-dirty installation
  • install_host_ptp_grandmaster.sh - PTP4L grandmaster setup on KVM host
  • install_host_vf_setup.sh - SR-IOV and VF configuration on KVM host
  • install_vm_sync.sh - ptp_kvm service setup for VM clock synchronization
  • install_srsran.sh - srsRAN Project installation with DPDK option

Performance Scripts (scripts/performance/):

  • optimize_kvm_host.sh - KVM host performance optimization
  • optimize_vm_srsRAN.sh - srsRAN VM performance optimization

For detailed usage information, see scripts/install/README.md and scripts/performance/README.md

Appendix C: Driver and Software Sources

All required drivers and software packages are available in the sources/ directory.