wiki:Workshops/SigComm2022/OpticalTutorial

Version 13 (modified by tingjunchen, 2 years ago) ( diff )

—

Site Navigation

  1. COSMOS Testbed Overview
    1. Concepts
    1. Testbed Workflow
    1. Availability and Resource Status
    1. Events and Conferences
  6. Getting Started
    1. Make an Account
    2. Create and Configure SSH Keys
    3. Make a Reservation
    4. Log in to your Reservation
    5. Control Resources with OMF
    6. Run a Hello World Experiment
    7. Get Help and Support
  7. User Guide
    1. Policies
    1. Account Creation
    1. Scheduling and Reservations
    1. Disk Images
    1. Frequently Asked Questions
    1. Resource Control with OMF
    1. Remote Access
    1. Chrome Remote Desktop Setup Page
    1. Measurement & Result Collection
    1. Storage
    1. Support
    1. Contributing to the Wiki
  20. Tutorials
    1. SDR and Wireless
    2. Optical Networking
    3. Wired Networking
    4. Edge Computing
    5. 4G/5G Systems
    6. Orchestration Platforms
  21. Architecture
    1. Data Flow
    1. Deployment Map
    1. Domains
    1. Naming Convention
    1. Networks
    1. Optical
  28. Resources, Services and APIs
    1. RF Control
    2. SDR Control
    3. Compute Control
    4. Network Control
    5. Optical Control
  29. Datasets
  30. Hardware Info
    1. Cameras
    1. Compute
    1. Network
    1. Nodes
    1. Optical
    1. RF Subsystems
    1. Antennas
    1. Full-Duplex Radio
    1. RF Front End
    1. Software Defined Radios (SDR)
  41. RF Policies & Compliance
    1. Outdoor Radio Frequency Allocation
    2. Program Experiment License
    3. Spectrum Monitoring
    4. Emergency Stop Procedures
    5. Network and Platform Security

Setting an Optical Network on COSMOS Test-bed

This wiki page contains the tutorial of setting up an optical network on the COSMOS Test-bed.

Authors:

Zehao Wang, Duke University <zehao.w[at]duke.edu>
Tingjun Chen, Duke University <tingjun.chen[at]duke.edu>

Description

Cosmos test-bed provides a possibility to create and use optical networks of various topologies. An example of how an optical network could be configured and used is provided. A simple experiment on switching of optical paths is described.

Compute Nodes and ToR switch interfaces used

  • Interfaces on sw-da-co1: 
    Eth 1/1/1:1 (vlan 1066) srv1-co1 data1
Eth 1/1/66  (vlan 1066) 10g dwdm tunable (Calient 5.7.3)
  • Interfaces on sw-da-lg1: 
    Eth 1/1/1:1 (vlan 1033) srv1-lg1 data1
Eth 1/1/1:2 (vlan 1034) srv2-lg1 data1
Eth 1/1/33  (vlan 1033) 10g dwdm tunable (Calient 5.7.5)
Eth 1/1/34  (vlan 1034) 10g dwdm tunable (Calient 5.7.6)

Experiment Topology and Resources Required

Figure 1. Topology setup of the experiment.

This experiment demonstrates optical switching between the short (1-hop) path, between srv1-co1 and srv1-lg1, and the long (2-hop) path, between srv1-co1 and srv2-lg1. It represents changing of the light path in C-RAN when a "Client" wants to dynamically change its base-band processing location between a nearby "Edge Cloud" and a further away "Central Cloud".

This experiment uses the following hardware resources and configurations:

  • 3 compute servers and 4 ROADM units in the bed.cosmos-lab.org domain: 
    srv1-co1.bed, srv1-lg1.bed, srv2-lg1.bed
rdm1-co1 (10.104.1.1), rdm2-co1 (10.104.1.2), rdm1-lg1 (10.104.2.1), rdm2-lg1 (10.104.2.2)
  • Interfaces on ToR switch sw-da-co1: 
    Eth 1/1/1:1 (vlan 1066) srv1-co1 data1
Eth 1/1/66  (vlan 1066) 10g dwdm tunable (connected to Calient 5.7.3)
  • Interfaces on ToR switch sw-da-lg1: 
    Eth 1/1/1:1 (vlan 1033) srv1-lg1 data1
Eth 1/1/1:2 (vlan 1034) srv2-lg1 data1
Eth 1/1/33  (vlan 1033) 10g dwdm tunable (connected to Calient 5.7.5)
Eth 1/1/34  (vlan 1034) 10g dwdm tunable (connected to Calient 5.7.6)

Setting Up Optical Topology

Setting up an optical topology consists in connecting ROADMs accordingly to the needs of experiment. ROADMs need to be connected correctly to each other and to ToR switch. All interconnections are realized by Calient S320 Switch.

Fig.2 Physical Optical Interconnections.

Calient Switch S320 has ports physically connected to ROADMs, ToR, Prof. Zussman’s lab and 3 loopback connections. All ports are designated in the Fig.2 Physical Optical Interconnections.

In order to support the Experiment_1 logical connections and to realize emulate real distance between Edge Cloud an Central Cloud, one could choose to pass the optical signal through the loop of 22 km passing by 32 Avenue of the Americas, NYC; and organize Client port Connections as Next:

  • For the comb source:
ID Ports Connection Devices Connection Comment
1 5.8.5→1.1.1comb_source_1 to ROADM1-CO1.port_1 Forming background traffic into ROADM1-CO1 add port 1
2 5.8.8→1.4.1comb_source_2 to ROADM2-CO1.port_1 Forming background traffic into ROADM2-CO1 add port 1
4 5.7.3↔1.1.1transceiver_3 with ROADM1-CO1.port_2 Connecting transceiver3 with ROADM1-CO1 add/drop port 1
5 1.1.8→5.5.1ROADM1-CO1.line_port to fiber_spool_1 Connecting ROADM1-CO1 line out to the 10 km fiber spool
6 5.5.2→1.1.8fiber_spool_2 to ROADM1-CO1.line_port Connecting the 10 km fiber spool to ROADM1-CO1 line in
7 5.5.1→1.4.8fiber_spool_1 to ROADM1-LG1.line_port Connecting the 10 km fiber spool to ROADM1-LG1 line in
8 1.4.8→5.5.2ROADM1-LG1.line_port to fiber_spool_2 Connecting ROADM1-LG1 line out to the 10 km fiber spool
9 2.1.8→4.6.1ROADM2-LG1.line_port to dark_fiber_1 Connecting ROADM2-LG1 line out to the 34 km dark fiber
10 4.6.2→2.1.8dark_fiber_2 to ROADM2-LG1.line_port Connecting the 34 km dark fiber to ROADM2-LG1 line in
11 4.6.1→2.4.8dark_fiber_1 to ROADM2-CO1.line_port Connecting the 34 km dark fiber to ROADM2-CO1 line in
12 2.4.8→4.6.2ROADM2-CO1.line_port to dark_fiber_2 Connecting ROADM2-CO1 line out to the 34 km dark fiber
  • For the short route:
ID Ports Connection Devices Connection Comment
1 2.4.1↔5.7.5 ROADM1-LG1.port_2 with transceiver_5 Connecting ROADM1-LG1 add/drop port 2 to transceiver5 to emulate the connection to the edge cloud
  • For the long route:
ID Ports Connection Devices Connection Comment
1 2.4.1↔5.7.6 ROADM2-CO1.port_2 with transceiver_6 Connecting ROADM2-CO1 add/drop port 2 to transceiver6 to emulate the connection to the center cloud

ROADMs Configuration

All of these configurations can be performed by Python scripts developed to work with the COSMOS test-bed. The Python commands send NETCONF commands to the ROADM.

Setting “Snake” Connection

Correct ROADM operation requires Line In port of a ROADM to always receive a light. That is why there is a dedicated transceiver (tengigabitethernet 1/33 on ToR) that sends light through all ROADMs by passing through loop-back connection on Calient S320 (port 5.5.1) and redirecting back, so the light is received on the same transceiver. This kind of connection is called “Snake”.

In order maintain this “Snake” for “Experiment_1” next connections form Table 1 must be in place: 1,3,5,6,8,9.

tengigabitethernet 1/33/1 on ToR configuration

Snake Interface (to passe through all ROADMs in loop): 60 (DWDM Channel C60) 1529,55 nm 196,00 Thz with frequency range [195.95,196.05] Thz

MUX/DEMUX configuration

  • ROADM 4:

DEMUX IN/OUT port: 5101/5204 MUX IN/OUT port: 4104/4201

  • ROADM 1:

DEMUX IN/OUT port: 5101/5201 MUX IN/OUT port: 4101/4201

  • ROADM 2:

DEMUX IN/OUT port: 5101/5201 MUX IN/OUT port: 4101/4201

  • ROADM 3:

DEMUX IN/OUT port: 5101/5204 MUX IN/OUT port: 4104/4201

ALS Disable Sequence (for 60 seconds)

  1. ROADM 4 booster,
  2. ROADM 2 booster,
  3. ROADM 3 booster,
  4. ROADM 1 booster,

Setting “Experiment_1” Connections

Configuring ToR1↔ToR2 Connection 1

  • ROADM 4:
  1. Enable MUX port 4102 “From ToR 1”
  2. Add Connection “Exp1-FromTor1” with Input/ Output Port 4102/4201 with bandwidth [192.95;193.05] (python add_connection.py 10.104.1.4 1 10 in-service false 4102 4201 192950 193050 5 Exp1-FromTor1)
  3. Enable DEMUX port 5202 “Towards ToR 1”
  4. Add Connection “Exp1-TorwardTor1” with I/O Port 5101/5202 (python add_connection.py 10.104.1.4 2 10 in-service false 5101 5202 192950 193050 5 Exp1-TorwardTor1)
  • ROADM 1:
  1. Enable MUX port 4102 “From ToR 2”
  2. Add Connection “From ToR 2” with I/O Port 4102/4201 with bandwidth [192.95;193.05]
  3. Enable DEMUX port 5202 “Towards ToR 2”
  4. Add Connection “Towards ToR 2” with I/O Port 5101/5202

Configuting ToR1↔ToR3 Connection 2

  • ROADM 4 (Same As For Connection 1):
  1. Enable MUX port 4102 “From ToR 1”
  2. Add Connection “From ToR 1” with I/O Port 4102/4201 with bandwidth [192.95;193.05]
  3. Enable DEMUX port 5202 “Towards ToR 1”
  4. Add Connection “Towards ToR 1” with I/O Port 5101/5202 with bandwidth [192.95;193.05]
  • ROADM 1 <Not Same!>:
  1. Enable MUX port 4101 “Through Port” (enabled for Snake)
  2. Add Connection “Through In” with I/O Port 4101/4201 with bandwidth [192.95;193.05]
  3. Enable DEMUX port 5201 “Through Port” (enabled for Snake)
  4. Add Connection “Through Out” with I/O Port 5101/5201 with bandwidth [192.95;193.05]
  • ROADM 2 (Same As For ROADM1):
  1. Enable MUX port 4101 “Through Port” (enabled for Snake)
  2. Add Connection “Through In” with I/O Port 4101/4201 with bandwidth [192.95;193.05]
  3. Enable DEMUX port 5201 “Through Port” (enabled for Snake)
  4. Add Connection “Through Out” with I/O Port 5101/5201 with bandwidth [192.95;193.05]
  • ROADM 3 (Same As For ROADM4):
  1. Enable MUX port 4102 “From ToR 3”
  2. Add Connection “From ToR 3” with I/O Port 4102/4201 with bandwidth [192.95;193.05]
  3. Enable DEMUX port 5202 “Towards ToR 3”
  4. Add Connection “Towards ToR 3” with I/O Port 5101/5202 with bandwidth [192.95;193.05]

Network Interfaces Configuration for Experiment_1

Setting Up ToR switch with 3 logical ToR switches

  1. Preparing the interfaces to be set as VLAN switch ports: 
sw-tor-lg1#configure
sw-tor-lg1(conf)#interface twentyFiveGigE 1/1/1
sw-tor-lg1(conf-if-tf-1/1/1)#switchport
sw-tor-lg1(conf-if-tf-1/1/1)#no shutdown
sw-tor-lg1(conf-if-tf-1/1/1)#exit
sw-tor-lg1(conf)#interface twentyFiveGigE 1/1/2
sw-tor-lg1(conf-if-tf-1/1/2)#switchport
sw-tor-lg1(conf-if-tf-1/1/2)#no shutdown
sw-tor-lg1(conf-if-tf-1/1/2)#exit
sw-tor-lg1(conf)#interface twentyFiveGigE 1/1/3
sw-tor-lg1(conf-if-tf-1/1/3)#switchport
sw-tor-lg1(conf-if-tf-1/1/3)#no shutdown
sw-tor-lg1(conf-if-tf-1/1/3)#exit
sw-tor-lg1(conf)#interface tengigabitethernet 1/31/1
sw-tor-lg1(conf-if-te-1/31/1)#switchport
sw-tor-lg1(conf-if-te-1/31/1)#no shutdown
sw-tor-lg1(conf-if-te-1/31/1)#exit
sw-tor-lg1(conf)#interface tengigabitethernet 1/29/1
sw-tor-lg1(conf-if-te-1/29/1)#switchport
sw-tor-lg1(conf-if-te-1/29/1)#no shutdown
sw-tor-lg1(conf-if-te-1/29/1)#exit
sw-tor-lg1(conf)#interface tengigabitethernet 1/32/1
sw-tor-lg1(conf-if-te-1/32/1)#switchport
sw-tor-lg1(conf-if-te-1/32/1)#no shutdown
sw-tor-lg1(conf-if-te-1/32/1)#exit
  1. Assigning interfaces to VLANs 
sw-tor-lg1#configure
sw-tor-lg1(conf)#interface vlan 121
sw-tor-lg1(conf-if-vl-121)#untagged twentyFiveGigE 1/1/1
sw-tor-lg1(conf-if-vl-121)#untagged tengigabitethernet 1/32/1
sw-tor-lg1(conf-if-vl-121)#exit
sw-tor-lg1(conf)#interface vlan 122
sw-tor-lg1(conf-if-vl-122)#untagged twentyFiveGigE 1/1/2
sw-tor-lg1(conf-if-vl-122)#untagged tengigabitethernet 1/29/1
sw-tor-lg1(conf-if-vl-122)#exit
sw-tor-lg1(conf)#interface vlan 123
sw-tor-lg1(conf-if-vl-123)#untagged twentyFiveGigE 1/1/3
sw-tor-lg1(conf-if-vl-123)#untagged tengigabitethernet 1/31/1
sw-tor-lg1(conf-if-vl-123)#exit
sw-tor-lg1(conf)#exit
  1. Assigning a wavelength to transceivers: 
sw-tor-lg1#configure
sw-tor-lg1(conf)#interface tengigabitethernet 1/32/1
sw-tor-lg1(conf-if-te-1/32/1)#wavelength 1553.3
sw-tor-lg1(conf-if-te-1/32/1)#exit
sw-tor-lg1(conf)#interface tengigabitethernet 1/29/1
sw-tor-lg1(conf-if-te-1/29/1)#wavelength 1553.3
sw-tor-lg1(conf-if-te-1/29/1)#exit
sw-tor-lg1(conf)#interface tengigabitethernet 1/32/1
sw-tor-lg1(conf-if-te-1/32/1)#wavelength 1553.3
sw-tor-lg1(conf-if-te-1/32/1)#exit
sw-tor-lg1(conf)#exit
  1. Verify VLANs: 
sw-tor-lg1#show vlan

Codes: * - Default VLAN, G - GVRP VLANs, R - Remote Port Mirroring VLANs, P - Primary, C - Community, I - Isolated
       O - Openflow, Vx - Vxlan
Q: U - Untagged, T - Tagged
   x - Dot1x untagged, X - Dot1x tagged
   o - OpenFlow untagged, O - OpenFlow tagged
   G - GVRP tagged, M - Vlan-stack
   i - Internal untagged, I - Internal tagged, v - VLT untagged, V - VLT tagged

    NUM    Status    Description                     Q Ports
    121    Active                                    U Te 1/32/1
                                                     U Tf 1/1/1
    122    Active                                    U Te 1/29/1
                                                     U Tf 1/1/2
    123    Active                                    U Te 1/31/1
                                                     U Tf 1/1/3

Configuring Compute Nodes (Servers srv1..3-lg1)

  1. Install net-tools: 
sudo apt install net-tools
  1. Configure interfaces eo1 and assign IP addresses: 
native@srv1-lg1:~$ sudo ifconfig eno1 192.168.1.1 netmask 255.255.255.0
native@srv2-lg1:~$ sudo ifconfig eno1 192.168.1.2 netmask 255.255.255.0
native@srv3-lg1:~$ sudo ifconfig eno1 192.168.1.3 netmask 255.255.255.0

Perform Experiment_1

  1. Establish Connection ToR1↔ToR2.
  2. Try Ping from Srv1 to Srv2: 
native@srv1-lg1:~$ ping 192.168.1.3
PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data.
64 bytes from 192.168.1.3: icmp_seq=1 ttl=64 time=0.460 ms
64 bytes from 192.168.1.3: icmp_seq=2 ttl=64 time=0.423 ms
  1. Establish Connection ToR1↔ToR2.
  2. Try Ping from Srv1 to Srv3.

Attachments (1)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.