Logo slack_icon Join Slack
Network Switch Initial Setup — Netris Documentation

Get Started

  • Introduction to Netris
    • What is Netris
    • What Netris does
    • Who Netris is for
    • How an AI network differs from a traditional network
    • What to read next
  • Definitions
    • Access and roles
    • Fabric roles
    • Network constructs
    • Services
    • AI-specific
    • Infrastructure and operations
  • Netris Architecture
    • Core components (always present)
    • AI components (additive, when a GPU cluster is present)
    • Netris Controller
    • Netris Switch Agent
    • Netris SoftGate
    • Management Network Architecture
    • Security Design Principles
    • Shared Responsibility
    • What to read next

Try & Learn Netris

  • Lab Scenario: GPU-as-a-Service network with NVIDIA Spectrum-X architecture
    • Initialize the Netris controller
    • Start a simulation
    • Monitoring Dashboard
    • Topology
    • SSH to Switches
    • SSH to GPU servers
    • NHN (Netris host networking plugin)
    • Server Cluster Template
    • Server Cluster
    • Checking the connectivity
    • Cleanup the Controller
  • Learn Netris Operations with Kubernetes
    • Intro
    • Install Netris Operator
    • Deploy an Application with an On-Demand Netris Load Balancer
    • Using Netris Custom Resources
      • Introduction to Netris Custom Resources
      • L4LB Custom Resource
      • Importing Existing Resources from Netris Controller to Kubernetes
      • Reclaim Policy

Reference

  • Netris Supported Functionality & Platforms Matrix
    • Switch Fabric Management Functions
    • Host Networking
    • Monitoring & Telemetry
    • External Routing Functions
    • Cloud Networking Functions & Constructs
    • Overlay Network Functions
    • AI Specific Functions
    • Security
    • Netris Controller Administration
    • Management Interfaces
  • Netris and NOS versions compatibility matrix
  • Hardware Requirements
    • Netris Controller
    • Netris SoftGate HS (Multi-VPC version)
  • Supported Switch Hardware
    • Nvidia
    • Dell
    • EdgeCore
    • Arista

Installation

  • Controller Installation
    • Installing HA Netris Controller in Air-Gapped Environments
      • Why Air-Gapped Installation?
      • Why a High Availability (HA) Cluster?
      • Prerequisites
      • Obtain the Installation File
      • Steps to Install
        • 1. Preparing Each Node
        • 2. Install K3s on All Nodes
        • 3. Import Necessary Container Images
        • 4. Configure kube-vip for KubeAPI High Availability
        • 5. Add Helm Chart Packages to K3s
        • 6. Verify and Scale Core K3s Components
        • 7. Deploy Kube-VIP Cloud Controller
        • 8. Install Traefik Proxy
        • 9. Deploy North-South controller VIP (optional)
        • 10. Deploy the Netris Controller
        • 11. (Optional) Enable SSL with cert-manager
        • 13. Set Up the Local Netris Repository
        • 14. Validate Your Deployment
      • After Installation
      • How to consume local repository
    • Upgrading HA Netris Controller in Air-Gapped Environments
      • Obtain the Upgrade File
      • 1. Preparing Each Node
        • 1.1 Transfer the File to the Servers
        • 1.2 Extract the Tarball
        • 1.3 Navigate to the Installation Directory
      • 2. Steps to Upgrade Controller
        • 2.1 Import Necessary Container Images
        • 2.2 Add Helm Chart Packages Upgrades to K3s
        • 2.3 Database backup
        • 2.4 Upgrade Netris Controller
      • 3. Steps to Upgrade the Local Netris Repository
    • Maintenance Procedures
      • Node Maintenance Best Practices
        • Individual Node Maintenance (Recommended Approach)
        • Full Cluster Maintenance (When All Nodes Need Simultaneous Maintenance)
      • Verifying MariaDB Cluster Health
      • Maintenance Best Practices
    • MariaDB automatic backups: locate, verify, and restore
      • Locate the backup directory on each controller node
      • Copy the backup file to your home directory
      • Verify the backup file integrity
      • Verify MariaDB cluster readiness before restore
      • Copy the backup file into a MariaDB pod
      • Restore the backup
      • Take a manual backup
      • Summary
    • Helm Chart Installation
      • Requirements
      • Get Repo Info
      • Installing the Chart
      • Uninstalling the Chart
      • Chart Configuration
    • Zero Touch Provisioning (ZTP)
      • Overview
      • Prerequisites
        • 1. HA Controller with North-South VIP
        • 2. Configure Netris Local Repository
        • 3. Upload NOS images to the local repo
        • 4. Controller Management Address IP in Controller Settings
        • 5. IPAM Subnets in the Management VPC
        • 6. V-Net with DHCP Relay
        • 7. Inventory Profile Configuration
        • 8. Switch MAC Address in Inventory
        • 9. Configure Static Routes on the Controller Nodes
        • 10. Deploy the Netris ZTP DHCP Server
        • 11. Verify ZTP Readiness
      • How ZTP works (Process Summary)
      • Troubleshooting
  • Network Switch Initial Setup
    • Nvidia Cumulus v5.9+ Switch Initial Setup
    • Nvidia Cumulus v5 Switch Initial Setup
    • Nvidia Cumulus v3.7 Switch Initial Setup
    • EdgeCore SONiC Switch Initial Setup
    • Dell SONiC Switch Initial Setup
    • Ubuntu SwitchDev Switch Initial Setup
  • SoftGate HS
    • Overview
      • Services Provided by SoftGate
      • What SoftGate does not do
      • Hardware Requirements
    • Performance
    • Architecture Overview
      • SoftGate physical connectivity
      • SoftGate logical connectivity
      • SoftGate node Control Plane
      • SoftGate node Data Plane
      • SoftGate Roles and Flavors
    • Installation and Initial Deployment
      • Prerequisites
      • Installation Guide
    • Upstream/Border Routers (EBGP)
      • Configuring upstream BGP
      • BGP route exchange between SoftGates, upstream routers, and downstream switches
    • High Availability and Load Distribution
      • Netris VPC originated traffic routing
      • Internet originated traffic routing
      • Failover behavior
    • Observability and Operations
  • Netris Host Networking
    • Overview
    • How It Works
    • Before You Begin
      • System Requirements
      • Dependencies
      • Recommended NVIDIA Component Versions
        • Spectrum-X v2.1.2
      • Permissions
    • Installation Overview
    • Installation
    • Configuration
      • 1. Review the Configuration File
      • 2. Configure BlueField3/ConnectX7/ConnectX8 NICs
      • 3. Verify BlueField3/ConnectX7/ConnectX8 NICs Parameters
      • 4. Start the NHN daemon
    • Operating the NHN plugin
      • Starting NHN
      • Monitoring NHN
      • Stopping NHN
    • Using the bf3-config
      • Command-Line Options
      • Examples
    • Using the Verifier
      • Command-Line Options
      • Examples
    • Network Configuration Formats
      • Network Manager Auto-Detection
      • Netplan (systemd-networkd)
      • Ifupdown (Ubuntu interfaces)
  • Installing BlueField-3 DPUs
    • Concepts
    • Prerequisites
    • Overview
    • Creating the Resources
      • 1. Interfaces — physical-ifaces.yaml
      • 2. BFB — bfb.yaml
      • 3. DPUFlavor — hbn-dpuflavor.yaml
      • 4. DOCA HBN service — hbn-service.yaml
      • 5. Netris DPU agent service — netris-agent-service.yaml
      • 6. DPUDeployment — dpudeployment.yaml
    • Apply Order
    • Verify
    • Next Steps

Fabric Management

  • Netris Site
  • VPC
    • Overview
    • How a VPC’s VRF is placed on the fabric
    • Adding a new VPC
    • VPC child objects
    • System VPC and Default VPC
      • What the Default VPC is for
      • What the System VPC is for
    • Historical note: why System VPC and Default VPC are the same VPC today
  • Inventory
    • Adding Switches
    • Adding SoftGates
    • Adding Servers
    • Adding DPUs
  • Topology Manager
    • Adding Links
  • Inventory Profiles
    • Overview
    • SNMPv2 credentials
    • NetQ Settings
    • Syslog Settings
    • Fabric Settings
    • GPU Cluster Specific Settings
    • Custom Rules
    • ZTP Settings
  • IP Address Management (IPAM)
    • Allocations and Subnets
    • Subnet purpose and service dependencies
    • Add an Allocation
    • Add a Subnet
  • BGP
    • BGP Overview
    • Basic BGP
    • Advanced BGP
    • BGP Objects
      • IPv4 Prefix
      • IPv6 Prefix
      • Community
    • BGP route-maps
    • eBGP Importing Non-Default Routes into a VPC
      • How to Import Non-Default Prefixes
  • Network Interfaces
  • Link Aggregation (LAG)
    • Automatic LAG with EVPN Multi-homing
    • Custom LAG
  • Maintenance Mode
    • Overview
    • Maintenance Mode for Softgate - What’s happening behind the scenes?
    • Maintenance Mode for Switch - What’s happening behind the scenes?
  • Static Routing

Services

  • V-Net
    • Introduction
    • L2VPN V-Nets
      • DHCP and DHCP Relay
        • DHCP Option Sets
        • Netris DHCP
        • DHCP Relay
      • Link Aggregation and Multihoming
        • Ethernet VPN Multi-Homing (EVPN-MH)
        • MC-LAG
      • V-Net Fields explained
      • Advanced V-Net Fields explained
      • Multisite V-Nets
      • Link Aggregation and Multihoming
      • DHCP
      • Labels
    • L3VPN
      • Creating an L3VPN V-Net
      • Rules and limits
    • Verification Tools
      • UI Tools
  • Server Cluster
    • Introduction
    • Server Cluster Template
      • Server Cluster Template Examples:
        • Ethernet-only Fabric Example
        • Infiniband Fabric Example
        • NVLink (NVL72 or NVL144) Fabric Example
        • IPv6 Example
      • Template Fields Explained:
      • Adding a Server Cluster Template
      • Advanced Uses
        • Non-overlapping subnets
        • Specify gateway
    • Creating Server Cluster
      • Adding a Server Cluster
      • Shared Endpoints
        • Shared and dedicated endpoints across fabrics
        • Untagged VLAN on Shared Endpoints
      • Server Cluster Fields Explained:
  • NAT
    • Enabling NAT
    • Defining NAT rules
  • L4 Load Balancer (L4LB)
    • Enabling L4LB service
    • Consuming L4LB service
  • VPC Peering
    • Peering a BGP-only VPC
  • VPC Connect
    • Overview
  • Access Control Lists (ACL)
    • Where ACL Entries Are Installed
    • ACL Default Policy
    • ACL Entries / Rules
      • Description of ACL entry fields.
    • Checking for overlapping rules
    • ACL Processing Order

Netris Integrations

  • NVIDIA UFM (InfiniBand) Integration Plugin for Netris Controller
    • Overview
      • Key Benefits
    • Architecture
    • Prerequisites
    • UFM Configuration Requirements
    • Installation
    • Upgrading the UFM Integration
      • Multiple UFM Instances
    • Configuration Parameters
      • Netris Controller Configuration
      • NVIDIA UFM Configuration
      • Agent Configuration
    • Usage Guide
      • 1. Server Configuration in Netris
      • 2. Create a Server Cluster Template
      • 3. Create Server Clusters
      • 4. Verification
      • 5. Monitoring Integration Status
    • Replacing HCAs and Servers
    • Functional Workflow
      • PKey membership: dedicated and shared
    • InfiniBand Security (Recommended)
      • What is MKey Protection?
      • Impact on Netris Integration
      • Configuration Steps
        • 1. Set the MKey Value
        • 2. Enable MKey Protection Level
        • 3. Enable VSKey (Vendor Specific Key)
        • 4. Restart UFM Service
        • 5. Verification
    • Monitoring and Troubleshooting
      • Viewing Logs
      • Common Issues and Solutions
        • Connection Issues to Netris Controller or UFM
        • PKey Assignment Issues
        • SHARP Reservation Issues
        • Synchronization Delays
    • Version Compatibility
    • Getting Started Guide
      • Quick Setup Example
    • Additional Resources
  • NVIDIA NMX-C (NVLink) Integration Plugin for Netris Controller
    • Overview
      • Key Benefits
    • Architecture
      • High-Level Workflow
    • Version Compatibility
    • Prerequisites
    • Netris NVLink Plugin Installation
      • Installing the Netris NVLink plugin
        • Step 1: Configuring the NMX-C plugin credentials
        • Step 2: Configuring NMX Controller connection
        • Step 3: Applying the K8S deployment
      • Loading GPU Inventory
        • Step 1: Install the nvlink-loader
        • Step 2: Load the GPU inventory into the Netris Controller
        • Step 3: Verify the imported GPU inventory
    • Using the Netris NVLink plugin
    • Verification
    • Maintenance and Deprovisioning
    • Additional Resources
  • Kubernetes Integration
    • Install Netris Operator
      • Helm Chart Method
      • Regular Manifest Method
    • Using Type ‘LoadBalancer’
    • Using Netris Custom Resources
      • Introduction to Netris Custom Resources
      • L4LB Custom Resource
      • V-Net Custom Resource
      • BGP Custom Resource
      • Importing existing resources from Netris Controller to Kubernetes
      • Reclaim Policy
    • Calico CNI Integration
      • Disabling Netris-Calico Integration
  • Netris-CloudStack Integration
    • High-Level Concept of Integration
      • How It Works
      • Challenges Addressed
      • Benefits
      • Use Cases
    • Compute and Network Architecture
      • Diagram Overview
      • Network Flow
    • Prerequisites
      • Step-by-Step Configuration Instructions for the Netris Controller
      • IPAM Setup
        • Create an Allocation
        • Create Subnets
      • Inventory Setup
        • Adding Servers
        • Creating Servers’ Links
        • Optimize BGP Overlay for Hypervisor
        • Adding Subnets for CloudStack Cluster
        • Enabling Internet Connectivity for ACS Servers
        • Enabling Access to CloudStack Management GUI
        • Creating CloudStack Networks
    • Server Configuration and Software Installation
      • Configuring Network on CloudStack Management Server (Server 1)
        • Understanding the Network Layout
        • Netplan Configuration
      • Install Netris-CloudStack Agent on Hypervisor Servers
        • Bringing Up NICs Before Installation
        • Pre-Installation Steps (For Deployments Without OOB)
        • Key Functions of the Netris-CloudStack Agent
        • Installation Steps
        • Example Successful Output of One-Liner Script
        • Verification Steps
        • Checking Network Connectivity
        • Finalizing the Network Setup
        • Keeping the Temporary OOB VNet for Emergency Access
        • Managing Additional NICs on the Server
      • Install CloudStack Management Service
        • Installation Steps
    • Configuring CloudStack for Netris Integration
      • Enabling the Netris Plugin in CloudStack
      • Initializing CloudStack Setup
        • Steps to Initialize CloudStack
    • Using CloudStack with Netris Isolation Method
      • Creating a VPC
      • Creating a Network Tier
      • Port Forwarding
        • Seamless Integration Between CloudStack and Netris
        • Step-by-Step Process for Configuring Port Forwarding in CloudStack with Netris
        • Key Notes:
      • Configuring Static NAT in CloudStack
        • How Static NAT Works with CloudStack and Netris
        • Steps to Configure Static NAT:
        • How CloudStack and Netris Work Together
        • Final Notes
  • Terraform: Netris provider
    • Install Terraform
    • Create a directory for Terraform files
    • Configure a provider
    • Prepare an infrastructure plan
    • Create resources
    • Delete resources
  • EVPN on Host
    • Overview
      • Supported Operating Systems
    • Use Cases
      • Managed Kubernetes
      • Other Uses
    • How It Works
      • Bare Metal GPU Nodes
      • Shared Nodes
      • Underlay Configuration
      • Packet Flow
      • Management Plane
    • Prerequisites
      • 1. Management V-Net
      • 2. Server Objects
      • 3. Server Networking Configuration
      • 4. Netris Controller
    • Installation
      • Install the EVPN-on-Host agent
      • Enable Underlay
      • Verification
    • Provisioning Tenants
      • Verification
  • BlueField-3 DPU Support in Netris
    • Overview
    • What Is a DPU
    • Prerequisites and Assumptions
    • Use Cases
    • Configuring a Server with a DPU
    • VF Interface Naming
    • Connecting DPU VFs to a VNet
    • Monitoring

Monitoring & Observability

  • Graph Boards
    • Graph Boards
    • API Logs
    • Dashboard
  • Topology & Wiring Validation
    • Switch-to-switch and Switch-to-SoftGate
    • Switch-to-Server
  • NVIDIA NetQ Integration
    • Topology Blueprint Activation
  • Netris Healthchecks
    • RX/TX drops and errors threshold calculation method
  • Looking Glass

Tutorials

  • Getting Started with Switch-Fabric Manager & VPC

Miscellaneous

  • Accounts
    • Users
    • Tenants
      • Admin Tenant vs Guest Tenant
    • Permission Groups
    • User Roles
  • Custom Configuration Snippets
    • Overview
    • Arista EOS
    • NVIDIA Cumulus Linux (NVUE)
  • Labels
    • How labels work
    • Rules and limits
    • When to use labels

Release Notes

  • Release Notes
    • Netris 4.16.0 Release Notes (Sep/02/2026)
      • What’s new in Netris 4.16.0?
      • Features
      • Enhancements
      • Bug fixes
      • Security fixes
      • References
    • Netris 4.15.0 Release Notes (Aug/18/2026)
      • What’s new in Netris 4.15.0?
      • Features
      • Enhancements
      • Bug fixes
      • Security fixes
      • References
    • Netris 4.14.0 Release Notes (Jul/31/2026)
      • What’s new in Netris 4.14.0?
      • Features
      • Enhancements
      • Bug fixes
      • Security fixes
      • References
    • Netris 4.13.0 Release Notes (Jul/15/2026)
      • What’s new in Netris 4.13.0?
      • Features
      • Enhancements
      • Bug fixes
      • Security fixes
      • References
    • Netris 4.12.0 Release Notes (Jul/2/2026)
      • What’s new in Netris 4.12.0?
      • Features
      • Enhancements
      • Bug fixes
      • Security fixes
      • References
    • Netris 4.11.0 Release Notes (Jun/15/2026)
      • What’s new in Netris 4.11.0?
      • Features
      • Enhancements
      • Bug fixes
      • Security fixes
      • References
    • Netris 4.10.0 Release Notes (May/19/2026)
      • What’s new in Netris 4.10.0?
      • Features
      • Enhancements
      • Bug fixes
      • Security fixes
      • References
    • Netris 4.9.0 Release Notes (Apr/30/2026)
      • What’s new in Netris 4.9.0?
      • Features
      • Bug fixes and improvements
      • References
    • Netris 4.8.0 Release Notes (Apr/15/2026)
      • What’s new in Netris 4.8.0?
      • Features
      • Bug fixes
      • References
    • Netris 4.7.0 Release Notes (Mar/11/2026)
      • What’s new in Netris 4.7.0?
      • Features
      • Bug fixes
      • References
    • Netris 4.6.1 Release Notes (Feb/27/2026)
      • What’s new in Netris 4.6.1?
      • Features
      • Enhancements
      • Bug fixes
      • Security fixes
      • References
    • Netris 4.6.0 Release Notes (Feb/10/2026)
      • What’s new in Netris 4.6.0?
      • Features
      • Enhancements
      • Bug fixes
      • References
    • Netris 4.5.3 Release Notes (August/27/2025)
      • What’s new in Netris 4.5.3?
      • Features
      • Bug fixes
      • References
    • Netris 4.4.0 Release Notes (May/15/2025)
      • What’s new in Netris 4.4.0?
      • Features
      • Enhancements
      • References
    • Netris 4.3.0 Release Notes (July/31/2024)
      • What’s new in Netris 4.3.0?
      • Features
      • Bug fixes
      • References
    • Netris 4.2.0 Release Notes (May/1/2024)
      • What’s new in Netris 4.2.0?
      • Features
      • Bug fixes
      • References
    • Netris 4.1.1 Release Notes (Jan/31/2024)
      • What’s new in Netris 4.1.1?
      • Features
      • Bug fixes
      • References
    • Netris 4.0.0 Release Notes (Jul/25/2023)
      • What’s new in Netris 4.0.0?
      • Features
      • Bug fixes
      • References
    • Netris 3.5.0 Release Notes (Oct/24/2023)
      • What’s new in Netris 3.5.0?
      • Features
      • Bug fixes
      • References
    • Netris 3.4.4 Release Notes (Sep/29/2023)
      • What’s new in Netris 3.4.4?
      • Features
      • Bug fixes
      • References
    • Netris 3.4.3 Release Notes (Nov/09/2023)
      • What’s new in Netris 3.4.3?
      • Bug fixes
      • References
    • Netris 3.4.0 Release Notes (Mar/06/2023)
      • What’s new in Netris 3.4.0?
      • New Platform Integration
      • Features
      • Bug fixes
      • References
    • Netris 3.3.0 Release Notes (Dec/05/2022)
      • What’s new in Netris 3.3.0?
      • Features
      • Bug fixes
      • Various improvements and bug fixes
      • References
    • Netris 3.2.1 Release Notes (Oct/05/2022)
      • What’s new in Netris 3.2.1?
      • Bug fixes
      • References
    • Netris 3.2.0 Release Notes (Sep/22/2022)
      • What’s new in Netris 3.2.0?
      • Features
      • Bug fixes
      • References
    • Netris 3.1.0 Release Notes (Aug/1/2022)
      • What’s new in Netris 3.1.0?
      • Features
      • Bug fixes
      • References
    • Netris 3.0.0 Release Notes (Oct/6/2021)
      • What’s new in Netris 3.0.0?
      • References
    • Netris 2.9.0 Release Notes (Mar/5/2021)
      • What’s new in Netris 2.9.0?
      • References
    • Netris 2.8.0 Release Notes (May/13/2020)
      • What’s new in Netris 2.8.0?
      • Features
      • Bug fixes
      • References
Netris docs
  • Network Switch Initial Setup
  • Edit on GitHub
Previous Next

Network Switch Initial Setup

  • Nvidia Cumulus v5.9+ Switch Initial Setup
  • Nvidia Cumulus v5 Switch Initial Setup
  • Nvidia Cumulus v3.7 Switch Initial Setup
  • EdgeCore SONiC Switch Initial Setup
  • Dell SONiC Switch Initial Setup
  • Ubuntu SwitchDev Switch Initial Setup
Previous Next

© Copyright 2022, Netris.

Built with Sphinx using a theme provided by Read the Docs.
Alternate Versions v: 4.16
Free theme by Read the Docs.