| 1 | [[Include(WikiToC)]] |
| 2 | |
| 3 | == Setting up a Virtual / Simulated Optical Network using the Mininet-Optical Software Emulator == |
| 4 | |
| 5 | This wiki page is intended for participants attending COSMOS workshop in SIGCOMM 2022. This contains a tutorial for setting up a virtual/simulated optical network using Mininet-Optical. [https://mininet-optical.org Mininet-Optical] was developed as part of the COSMOS and COSM-IC projects and |
| 6 | adds optical network emulation and simulation capabilities to |
| 7 | [https://mininet.org Mininet] to support software emulation of optical and packet-optical networks, |
| 8 | including modeling of optical transmission effects and impairments. |
| 9 | |
| 10 | This is a Mininet-Optical version of the tutorial at [wiki:Workshops/SigComm2022/OpticalTutorial] and is intended to show how an experiment designed for COSMOS' optical network testbed (hardware) may be adapted for use in a software emulation environment running on Linux (for example in a VM on a laptop or other hardware server), and how Mininet-Optical may be used to design experiments that will later be run on the COSMOS hardware testbed. |
| 11 | |
| 12 | The Mininet-Optical version of this tutorial implements the same network topology and the same experiment as the hardware testbed tutorial, but there are differences between the software and hardware environments. |
| 13 | |
| 14 | Please refer to [wiki:Workshops/SigComm2022/OpticalTutorial] for comparison with the hardware environment. |
| 15 | |
| 16 | ---- |
| 17 | = Description = |
| 18 | The COSMOS testbed enables creation and use of optical networks of various topologies. Similarly, Mininet-Optical enables creation of virtual optical networks using software emulation. An example of how an optical network could be configured and used is provided. A simple experiment on switching of optical paths is described. |
| 19 | |
| 20 | ---- |
| 21 | |
| 22 | = Compute Nodes and ToR switch interfaces used = |
| 23 | |
| 24 | * Each compute node has 1 Ethernet interface: |
| 25 | {{{ |
| 26 | srv1-co1-eth0 |
| 27 | srv1-lg1-eth0 |
| 28 | srv2-lg1-eth0 |
| 29 | }}} |
| 30 | |
| 31 | * There are three ToR Ethernet interfaces: |
| 32 | {{{ |
| 33 | sw-da-co1-eth1 |
| 34 | sw-da-lg1-eth2 |
| 35 | sw-da-lg1-eth3 |
| 36 | }}} |
| 37 | |
| 38 | * There are three WDM transceivers, with separate output and input ports: |
| 39 | {{{ |
| 40 | sw-da-co1-wdm320/321 |
| 41 | sw-da-lg1-wdm290/291 |
| 42 | sw-da-lg1-wdm310/311 |
| 43 | }}} |
| 44 | |
| 45 | * and two COMB Sources, with Line-out ports: |
| 46 | {{{ |
| 47 | comb1-wdm4201 |
| 48 | comb2-wdm4201 |
| 49 | }}} |
| 50 | |
| 51 | |
| 52 | ---- |
| 53 | |
| 54 | = Experiment_1 Context = |
| 55 | || [[Image(sigcomm2022tutorial_topology.png, width=500px)]] || |
| 56 | Fig.1 Logical Topology of Experiment_1 |
| 57 | |
| 58 | This experiment demonstrates optical switching between the short (1-hop) path, between {{{srv1-co1}}} and {{{srv1-lg1}}}, and the long (2-hop) path, between {{{srv1-co1}}} and {{{srv2-lg1}}}. It represents changing of the light path in C-RAN when a "Client" wants to dynamically change its base-band processing location between a nearby "Edge Cloud" and a further away "Central Cloud". |
| 59 | |
| 60 | To simulate the topology in hardware, the network component names are kept the same. |
| 61 | |
| 62 | Experiment includes 3 servers: |
| 63 | {{{ |
| 64 | srv1-co1 (192.168.1.1/24) |
| 65 | srv1-lg1 (192.168.1.2/24) |
| 66 | srv2-lg1 (192.168.1.3/24)] |
| 67 | }}} |
| 68 | |
| 69 | Experiment includes 4 ROADMs: |
| 70 | {{{ |
| 71 | rdm1-co1 (localhost:1834) |
| 72 | rdm1-lg1 (localhost:1832) |
| 73 | rdm2-lg1 (localhost:1833) |
| 74 | rdm2-co1 (localhost:1844) |
| 75 | }}} |
| 76 | |
| 77 | |
| 78 | 2 ToR Switches are connected to the 3 servers on the following interfaces: |
| 79 | |
| 80 | {{{ |
| 81 | sw-da-co1-eth1 <--> srv1-co1-eth0 |
| 82 | sw-da-lg1-eth2 <--> srv1-lg1-eth0 |
| 83 | sw-da-lg1-eth3 <--> srv2-lg1-eth0 |
| 84 | }}} |
| 85 | |
| 86 | |
| 87 | 3 Ethernet interfaces and 3 WDM transceivers will be connected within the ToR switch: |
| 88 | |
| 89 | {{{ |
| 90 | sw-da-co1-eth1 ; sw-da-co1-wdm320/321 (output/input) |
| 91 | sw-da-lg1-eth2 ; sw-da-lg1-wdm290/291 (output/input) |
| 92 | sw-da-lg1-eth3 ; sw-da-lg1-wdm310/311 (output/input) |
| 93 | }}} |
| 94 | |
| 95 | |
| 96 | We are assigning the following wavelength to the transceivers: |
| 97 | |
| 98 | {{{ |
| 99 | 1543.60 nm, 194.35 THz with bandwidth ~[194.30;194.40] THz |
| 100 | This corresponds to channel 61 in Mininet-Optical's as well as COSMOS' default channel grid. |
| 101 | }}} |
| 102 | |
| 103 | We are assigning the following bands of wavelength to the comb sources to simulate background traffic: |
| 104 | |
| 105 | {{{ |
| 106 | Band 1: [1560.50,1564.10] nm ~ [191.80,192.25] THz. This corresponds to channels 10-19 in Mininet-Optical's and COSMOS' default channel grid. |
| 107 | Band 2: [1548.40,1552.00] nm ~ [193.30,193.75] THz. This corresponds to channels 40-49 in Mininet-Optical's and COSMOS' default channel grid. |
| 108 | Band 3: [1532.60,1536.10] nm ~ [195.30,195.75] THz. This corresponds to channels 80-89 in Mininet-Optical's and COSMOS' default channel grid. |
| 109 | }}} |
| 110 | |
| 111 | |
| 112 | ---- |
| 113 | = Installing Mininet-Optical disk image on COSMOS' node = |
| 114 | |
| 115 | Before starting this tutorial, you will need to install Mininet-Optical in the console. We have already created a disk image with a pre-installed version of Mininet-Optical with necessary codes. If you don't have access to the COSMOS' console already, you can follow the sign-up and installation instructions at [wiki:Workshops/SigComm2022/SignupInstructions] |
| 116 | |
| 117 | To install the Mininet-Optical disk image, follow the below steps: |
| 118 | |
| 119 | 1. Remove any installed disk images on the node |
| 120 | {{{#!shell-session |
| 121 | user@console:~$ omf tell -a reset -t <node> |
| 122 | }}} |
| 123 | Here, <node> is the name of the appropriate node (for example, node1-2.sb2.orbit-lab.org) |
| 124 | 2. Install the Mininet-Optical OS image file. |
| 125 | {{{#!shell-session |
| 126 | user@console:~$ omf load -i sigcomm22-mnoptical-tutorial.ndz -t <node> |
| 127 | }}} |
| 128 | 3. Turn the node on |
| 129 | {{{#!shell-session |
| 130 | user@console:~$ omf tell -a on -t <node> |
| 131 | }}} |
| 132 | 3. Ping the node to check if disk image has been installed correctly |
| 133 | {{{#!shell-session |
| 134 | user@console:~$ ping <node> |
| 135 | }}} |
| 136 | 4. Log in to the node, and fetch the latest version of Mininet-Optical |
| 137 | {{{#!shell-session |
| 138 | $ ssh root@<node> |
| 139 | root@node# cd mininet-optical |
| 140 | root@node:~/mininet-optical# git fetch |
| 141 | root@node:~/mininet-optical# git checkout cosmos-tutorial |
| 142 | root@node:~/mininet-optical# git pull --rebase |
| 143 | root@node:~/mininet-optical# make install certs |
| 144 | }}} |
| 145 | 5. Run a sample script to check Mininet-Optical has been installed correctly. |
| 146 | {{{#!shell-session |
| 147 | root@node:~/mininet-optical# sudo PYTHONPATH=. python3 ./examples/simplelink.py |
| 148 | }}} |
| 149 | |
| 150 | The tutorial topology and a sample configuration script should be found in `~/mininet-optical/examples/sigcommtutorial.py` and `~/mininet-optical/examples/config-sigcommtutorial.sh`. |
| 151 | |
| 152 | If you want to test out Mininet-Optical on your computer after the tutorial, we have provided necessary resources in the appendix. |
| 153 | |
| 154 | ---- |
| 155 | |
| 156 | = Setting Up the Optical Topology = |
| 157 | |
| 158 | In the COSMOS optical testbed, all devices are connected to a Calient S320 space switch. This switch serves as a programmable patch panel that allows any port to be connected to any other port, enabling realization of arbitrary topologies with fast reconnection between experiments. |
| 159 | |
| 160 | It is possible to create a virtual space switch/programmable patch panel in Mininet-Optical to emulate the COSMOS optical testbed itself, but for this tutorial we will implement the topology using Mininet-Optical's topology API. |
| 161 | |
| 162 | The Mininet-Optical emulated network is created using a Python script, [https://github.com/Mininet-Optical/mininet-optical/blob/cosmos-tutorial/mnoptical/examples/sigcommtutorial.py examples/sigcommtutorial.py]. Take a look at it now to see how the topology is implemented. |
| 163 | |
| 164 | The topology itself is created using Mininet's high-level topology template API. Specifically, we create a subclass of class `Topo` and override the `build()` method: |
| 165 | |
| 166 | |
| 167 | {{{#!python |
| 168 | class TutorialTopo( Topo ): |
| 169 | ... |
| 170 | def build( self ): |
| 171 | }}} |
| 172 | |
| 173 | |
| 174 | ROADMs, Comb Sources and ToR switches are added using `addSwitch()` calls: |
| 175 | |
| 176 | {{{#!python |
| 177 | # ROADMs |
| 178 | NC = NetconfPortBase |
| 179 | rdm1co1 = self.addSwitch('rdm1co1', cls=LROADM, netconfPort=NC+4) |
| 180 | ... |
| 181 | # ToR switches |
| 182 | swda_co1 = self.addSwitch('swda-co1', cls=Terminal, transceivers=[('32', -1.5*dBm)]) |
| 183 | ... |
| 184 | #Comb Sources |
| 185 | comb1 = self.addSwitch('comb1', cls=CombSource, power=comb1_power) |
| 186 | ... |
| 187 | }}} |
| 188 | |
| 189 | Servers are added using `addHost()` calls: |
| 190 | |
| 191 | {{{#!python |
| 192 | # Servers |
| 193 | srv1_co1 = self.addHost('srv1-co1') |
| 194 | ... |
| 195 | }}} |
| 196 | |
| 197 | In Mininet, ports are created by specifying port numbers when we add links. |
| 198 | (This is due to the underlying link emulation which uses Linux virtual Ethernet (`veth`) |
| 199 | pairs.) |
| 200 | |
| 201 | Because of this, we need to specify the correct port numbers when we create the links. |
| 202 | |
| 203 | The base port numbers for a Lumentum ROADM20 are specified at the top of the file: |
| 204 | |
| 205 | {{{#!python |
| 206 | # Lumentum Roadm20 Port numbering |
| 207 | LINEIN, LINEOUT = 5101, 4201 |
| 208 | ADD, DROP = 4100, 5200 |
| 209 | }}} |
| 210 | |
| 211 | The server and ToR port numbers are as specified above. These port numbers are exactly the same as hardware experiment. |
| 212 | |
| 213 | Boost and Pre-amp amplifiers associated with ROADMS are added accordingly in the inter-ROADM connections. WDM fiber links are unidirectional and are added using `wdmLink()` calls: |
| 214 | |
| 215 | {{{#!python3 |
| 216 | # Inter-ROADM links |
| 217 | # We put 32km of fiber between rdm2lg1 and rdm2co1 |
| 218 | # Default fiber length is 1m if not specified |
| 219 | |
| 220 | # Sub-millisecond delays won't be accurate (due to scheduler timing |
| 221 | # granularity and running in a VM) but this will add observable |
| 222 | # propagation delay for the longer links. |
| 223 | |
| 224 | self.wdmLink(rdm1co1, rdm1lg1, LINEOUT, LINEIN, |
| 225 | spans=[0.0*m, |
| 226 | ('boost', {'target_gain': 18.0*dB, 'wdg_id': 'wdg1'}), |
| 227 | 10.0*km, |
| 228 | ('preamp', {'target_gain': 18.0*dB, 'wdg_id': 'wdg1'})], |
| 229 | delay='33us') |
| 230 | |
| 231 | self.wdmLink(rdm1lg1, rdm1co1, LINEOUT, LINEIN, |
| 232 | spans=[0.0*m, |
| 233 | ('boost', {'target_gain': 18.0*dB, 'wdg_id': 'linear'}), |
| 234 | 10.0*km, |
| 235 | ('preamp', {'target_gain': 18.0*dB, 'wdg_id': 'linear'})], |
| 236 | delay='33us') |
| 237 | |
| 238 | # Configuring Optical links between rdm1-lg1 and rdm2-lg1 |
| 239 | self.wdmLink(rdm1lg1, rdm2lg1, DROP+11, ADD+11) # Forward pass-through |
| 240 | self.wdmLink(rdm2lg1, rdm1lg1, DROP+11, ADD+11) # Backward pass-through |
| 241 | |
| 242 | # Configuring Optical Links between rdm2-lg1 and rdm2-co1 |
| 243 | self.wdmLink(rdm2lg1, rdm2co1, LINEOUT, LINEIN, |
| 244 | spans=[0.0*m, |
| 245 | ('boost', {'target_gain': 18.0*dB, 'wdg_id': 'wdg1'}), |
| 246 | 32.0*km, |
| 247 | ('preamp', {'target_gain': 18.0*dB, 'wdg_id': 'wdg1'})], |
| 248 | delay='106us') # Forward Link |
| 249 | self.wdmLink(rdm2co1, rdm2lg1, LINEOUT, LINEIN, |
| 250 | spans=[0.0*m, |
| 251 | ('boost', {'target_gain': 18.0*dB, 'wdg_id': 'wdg2'}), |
| 252 | 32.0*km, |
| 253 | ('preamp', {'target_gain': 18.0*dB, 'wdg_id': 'wdg2'})], |
| 254 | delay='106us') # Backward Link |
| 255 | |
| 256 | # ROADM add/drop 2 <-> ToR transceiver links |
| 257 | self.wdmLink(swda_co1, rdm1co1, 320, ADD+2) #Forward Link |
| 258 | self.wdmLink(rdm1co1, swda_co1, DROP+2, 321) #Backward Link |
| 259 | ... |
| 260 | # Configuring Optical Links between Comb Source and Tor ROADMS |
| 261 | self.wdmLink(comb1, rdm1co1, CombSource.LINEOUT, ADD+1) |
| 262 | ... |
| 263 | # Configuring Ethernet connections between Servers and Tor switches |
| 264 | self.addLink(server1, swda_co1, port1=0, port2=1) |
| 265 | |
| 266 | }}} |
| 267 | |
| 268 | Lastly, the topology and network objects are instantiated and everything is started up (and shut down) in the `__main__` section of the network setup script: |
| 269 | |
| 270 | {{{#!python |
| 271 | if __name__ == '__main__': |
| 272 | ... |
| 273 | topo = TutorialTopo() |
| 274 | net = Mininet( topo=topo, controller=None ) |
| 275 | restServer = RestServer( net ) |
| 276 | net.start() |
| 277 | restServer.start() |
| 278 | netconfServer = NetconfServer( |
| 279 | net, username=username, password=password, sslkeyfile=sslkeyfile ) |
| 280 | netconfServer.start() |
| 281 | ... |
| 282 | if 'test' in argv: |
| 283 | test(net) |
| 284 | else: |
| 285 | info(TutorialTopo.__doc__+'\n') |
| 286 | CLI(net) |
| 287 | netconfServer.stop() |
| 288 | restServer.stop() |
| 289 | net.stop() |
| 290 | info( 'Done.\n') |
| 291 | }}} |
| 292 | |
| 293 | = Creating the Mininet-Optical Network = |
| 294 | |
| 295 | All of these commands should be run in a terminal window for the VM or server where Mininet-Optical is installed. |
| 296 | |
| 297 | 1. We will run Mininet-Optical from the top directory of the source tree: |
| 298 | |
| 299 | {{{#!shell-session |
| 300 | root@node:# cd ~/mininet-optical |
| 301 | }}} |
| 302 | |
| 303 | 2. We should check to make sure that the tutorial scripts are present: |
| 304 | |
| 305 | {{{#!shell-session |
| 306 | root@node:~/mininet-optical# ls examples/sigcommtutorial.py |
| 307 | root@node:~/mininet-optical# ls examples/config-sigcommtutorial.sh |
| 308 | }}} |
| 309 | |
| 310 | 3. If they are missing, we can fetch and install the appropriate branch: |
| 311 | |
| 312 | {{{#!shell-session |
| 313 | root@node:~/mininet-optical# git fetch |
| 314 | root@node:~/mininet-optical# git checkout cosmos-tutorial |
| 315 | root@node:~/mininet-optical# make install |
| 316 | }}} |
| 317 | |
| 318 | 4. Since the NETCONF agents for the ROADMs use SSL, we first have to generate a set of (self-signed) SSL certificates for them to use: |
| 319 | |
| 320 | {{{#!shell-session |
| 321 | root@node:~/mininet-optical# make certs |
| 322 | }}} |
| 323 | |
| 324 | 5. Now we should be able to run the tutorial script to create the emulated network: |
| 325 | |
| 326 | {{{#!shell-session |
| 327 | root@node:~/mininet-optical# sudo PYTHONPATH=. examples/cosmostutorial.py |
| 328 | }}} |
| 329 | |
| 330 | 6. This should start up Mininet-Optical, create the tutorial network, and start the CLI: |
| 331 | |
| 332 | {{{#!shell-session |
| 333 | SIGCOMM22 mini-tutorial topology |
| 334 | |
| 335 | |
| 336 | comb1 -> rdm1co1 <--10km--> rdm1lg1 || rdm2lg1 <--34km--> rdm2co1 <- comb2 |
| 337 | | | | |
| 338 | swda_co1 swda_lg1--------------------------| |
| 339 | | | | |
| 340 | srv1-co1 srv1-lg1 srv2-lg1 |
| 341 | |
| 342 | This is for the SIGCOMM22 mini-tutorial at: |
| 343 | https://wiki.cosmos-lab.org/wiki/Workshops/SigComm2022/MininetOptical |
| 344 | |
| 345 | *** Starting CLI: |
| 346 | mininet-optical> |
| 347 | }}} |
| 348 | |
| 349 | Mininet-Optical CLI commands may be entered at the `mininet-optical>` prompt. |
| 350 | |
| 351 | = Controller Configuration = |
| 352 | |
| 353 | Now that we have the topology setup for the network, we want to configure the lightpaths and pass packets using [https://datatracker.ietf.org/group/netconf/about/ NETCONF], which is network management protocol for software defined networks. We have this configuration in the file [https://github.com/Mininet-Optical/mininet-optical/blob/cosmos-tutorial/mnoptical/examples/config-sigcommtutorial.py examples/config-sigcommtutorial.py], which we require to run from another terminal window. |
| 354 | |
| 355 | (Note: if you want to use the X11 features of Mininet, such as the `xterm` or `plot` commands, you may need to use `sudo HOME=~` rather than just `sudo`.) |
| 356 | <Should we add instructions to setup x11?> |
| 357 | |
| 358 | If you don't have one open already, open up another terminal window with the below instructions: |
| 359 | |
| 360 | === [WIP] Opening another terminal window in Orbit === |
| 361 | |
| 362 | 1. SSH into |
| 363 | |
| 364 | {{{ |
| 365 | #!shell-session |
| 366 | user@console:~$ cd ~/mininet-optical |
| 367 | }}} |
| 368 | |
| 369 | 2. Connect to the VM where Mininet-Optical is installed |
| 370 | |
| 371 | {{{ |
| 372 | #!shell-session |
| 373 | user@console:~$ cd ~/mininet-optical |
| 374 | }}} |
| 375 | |
| 376 | 3. Enter source directory |
| 377 | |
| 378 | {{{#!shell-session |
| 379 | user@console:~$ cd ~/mininet-optical |
| 380 | }}} |
| 381 | |
| 382 | Unless specified otherwise, all of the configuration commands below should |
| 383 | be entered in this second window at the shell prompt. |
| 384 | All of these configurations can be performed by Python scripts developed to work with the COSMOS test-bed. The Python commands send NETCONF commands to the ROADM. |
| 385 | |
| 386 | === Setting “Snake” Connection === |
| 387 | |
| 388 | (This is not currently part of the Mininet-Optical software emulated configuration.) |
| 389 | |
| 390 | === Running the configuration script === |
| 391 | |
| 392 | Run the script with the following command |
| 393 | {{{#!shell-session |
| 394 | root@node:~/mininet-optical# bash -x ./examples/config-sigcommtutorial.py |
| 395 | }}} |
| 396 | |
| 397 | This executes the configuration file which establishes the ground connections first and turns on the transceivers. We walk through the scipt below to deep dive into what's happening. |
| 398 | |
| 399 | Mininet-Optical's `Terminal` is the equivalent of the ToR switch |
| 400 | which contains Ethernet interfaces as well as WDM transceivers. |
| 401 | |
| 402 | Instead of using a Cisco-style CLI to configure it, we use its |
| 403 | default REST API. |
| 404 | |
| 405 | As noted above, 194.35 THz corresponds to channel C61 on Mininet-Optical's default 50GHz channel grid. C1's middle frequency is 191350 GHz, so C61 is at 191350 + 60*50 = 194350 GHz. |
| 406 | |
| 407 | 1. Connect Ethernet interfaces to Transceivers and set channel |
| 408 | |
| 409 | {{{#!bash |
| 410 | curl "$swda_co1/connect?node=swda-co1ðPort=1&wdmPort=320&wdmInPort=321&channel=61" |
| 411 | curl "$swda_lg1/connect?node=swda-lg1ðPort=2&wdmPort=290&wdmInPort=291&channel=61" |
| 412 | curl "$swda_lg1/connect?node=swda-lg1ðPort=3&wdmPort=310&wdmInPort=311&channel=61" |
| 413 | }}} |
| 414 | |
| 415 | 2. Turn on all transceivers and comb sources |
| 416 | |
| 417 | {{{#!bash |
| 418 | curl "$swda_co1/turn_on?node=swda-co1" |
| 419 | curl "$swda_lg1/turn_on?node=swda-lg1" |
| 420 | curl "$comb1/turn_on?node=comb1" |
| 421 | curl "$comb2/turn_on?node=comb2" |
| 422 | }}} |
| 423 | |
| 424 | 3. Configure Ethernet interfaces and assign IP addresses: |
| 425 | |
| 426 | {{{#!bash |
| 427 | ... |
| 428 | m=~/mininet/util/m |
| 429 | ... |
| 430 | $m srv1-co1 ifconfig srv1-co1-eth0 192.168.1.1/24 |
| 431 | $m srv1-lg1 ifconfig srv1-lg1-eth0 192.168.1.2/24 |
| 432 | $m srv2-lg1 ifconfig srv2-lg1-eth0 192.168.1.3/24 |
| 433 | }}} |
| 434 | |
| 435 | Note that `~/mininet/util/m` can be used to 'log in' to any one of the servers much as you would with `ssh` (but it actually spawns a shell in the appropriate `cgroup`/network namespace.) |
| 436 | |
| 437 | ---- |
| 438 | |
| 439 | === Performing base test configuration |
| 440 | |
| 441 | Once you execute the script, the script will ask for a prompt to perform a base test. This test pings 'srv-co1' to 'srv-lg1'; and pings 'srv-co1' to 'srv2-lg1' without establishing any lightpath connections between ROADMS. Press `Return` key to perform the base test. |
| 442 | |
| 443 | {{{#!shell-session |
| 444 | *** Base test before configuration |
| 445 | press return to test base configuration> |
| 446 | ... |
| 447 | *** srv1-co1 pinging srv1-lg1 |
| 448 | PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. |
| 449 | 64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=0.932 ms |
| 450 | 64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=0.145 ms |
| 451 | 64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=0.153 ms |
| 452 | |
| 453 | --- 192.168.1.2 ping statistics --- |
| 454 | 3 packets transmitted, 3 received, 0% packet loss, time 2089ms |
| 455 | rtt min/avg/max/mdev = 0.145/0.410/0.932/0.369 ms |
| 456 | ... |
| 457 | *** srv1-co1 pinging srv2-lg1 |
| 458 | PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data. |
| 459 | From 192.168.1.1 icmp_seq=1 Destination Host Unreachable |
| 460 | From 192.168.1.1 icmp_seq=2 Destination Host Unreachable |
| 461 | From 192.168.1.1 icmp_seq=3 Destination Host Unreachable |
| 462 | |
| 463 | --- 192.168.1.3 ping statistics --- |
| 464 | 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2192ms |
| 465 | }}} |
| 466 | |
| 467 | As expected, none of the servers are able to ping each other because no lightpath connections have been established as of yet. |
| 468 | |
| 469 | == Configuring ROADMs |
| 470 | |
| 471 | ROADMs in Mininet-Optical may be configured via several mechanisms. An internal Python API may be used for configuration within the script that creates the network. More realistically, two external SDN/RPC control interfaces are provided: a simple REST interface and a more realistic NETCONF interface which is partially compatible with the NETCONF interface of the hardware Lumentum ROADM20. |
| 472 | |
| 473 | For this tutorial, we are using NETCONF interface to configure the ROADMs to closely match the interface used in hardware experiment. |
| 474 | |
| 475 | === [WIP] MUX/DEMUX configuration === |
| 476 | |
| 477 | As a reminder, here are the ROADM port numbers: |
| 478 | |
| 479 | * ROADM 4: |
| 480 | DEMUX IN/OUT (ADD1/LINEOUT) port: 5101/5201 |
| 481 | MUX IN/OUT (LINEIN/DROP1)port: 4101/4201 |
| 482 | * ROADM 1: |
| 483 | DEMUX IN/OUT (ADD1/LINEOUT) port: 5101/5201 |
| 484 | MUX IN/OUT (LINEIN/DROP1) port: 4101/4201 |
| 485 | * ROADM 2: |
| 486 | DEMUX IN/OUT (ADD1/LINEOUT) port: 5101/5201 |
| 487 | MUX IN/OUT (LINEIN/DROP1) port: 4101/4201 |
| 488 | * ROADM 3: |
| 489 | DEMUX IN/OUT (ADD1/LINEOUT) port: 5101/5201 |
| 490 | MUX IN/OUT (LINEIN/DROP1) port: 4101/4201 |
| 491 | |
| 492 | Note that the servers are connected to ADD2/DROP2 while DROP1/ADD1 are used |
| 493 | as passthrough ports between ROADM 1 and ROADM 2. |
| 494 | |
| 495 | ''Also note that these port numbers are the same as the hardware |
| 496 | tutorial page.'' |
| 497 | |
| 498 | == Network Interfaces Configuration for Experiment_1 (short-hop) == |
| 499 | |
| 500 | In Experiment 1, we are choosing to pass the optical signal through 1 hop (via a pair of 10km fiber spools). This requires us to establish a connection between rdm1-co1 and rdm1-lg1. |
| 501 | Once you perform the base test as described above, the script will prompt you to press `Return` key to perform the test for configuration 1. Before trying the configuration, let's dive into what connections this script will install: |
| 502 | |
| 503 | ==== Configuring srv1-co1<=>srv1-lg1 Connection 1 on Mininet-Optical using NETCONF ==== |
| 504 | |
| 505 | * **rdm1-co1**: |
| 506 | 1. Enable MUX port 4102 “From sw-da-co1” |
| 507 | 2. Add Connection “Exp1_From_sw-da-co1” with Input/ Output Port 4102/4201 with bandwidth [194.30;194.40] |
| 508 | 3. Enable DEMUX port 5202 “Toward sw-da-co1” |
| 509 | 4. Add Connection “Exp1_Toward_sw-da-co1” with I/O Port 5101/5202 |
| 510 | |
| 511 | The NETCONF servers for `rdm-co1`[r4] and `rdm-lg1`[r1] are listening on `localhost` at ports 1834 and 1831, respectively. |
| 512 | We will use the `examples/nc_add_connection.py` script to configure connections using NETCONF. |
| 513 | |
| 514 | {{{#!bash |
| 515 | testdir=$(dirname $0) |
| 516 | addc=$testdir/nc_add_connection.py |
| 517 | r4=localhost:1834 |
| 518 | r1=localhost:1831 |
| 519 | }}} |
| 520 | |
| 521 | Note that MUX/ADD/LINEOUT is module 1 and DEMUX/DROP/LINEIN is module 2. We are calling all of our connections connection 10. The interfaces are `in-service` and `blocked` is `false`. The 5 is an attenuation setting which may currently be ignored in Mininet-Optical. |
| 522 | |
| 523 | {{{#!bash |
| 524 | $addc $r4 1 10 in-service false 4102 4201 194300 194400 5 Exp1_From_sw-da-co1 |
| 525 | $addc $r4 2 10 in-service false 5101 5202 194300 194400 5 Exp1-Toward_swda_co1 |
| 526 | }}} |
| 527 | |
| 528 | * **rdm1-lg1**: |
| 529 | 5. Enable MUX port 4102 “From sw-da-lg1” |
| 530 | 6. Add Connection “Exp1_From_sw-da-lg1” with I/O Port 4102/4201 with bandwidth [192.95;193.05] |
| 531 | 7. Enable DEMUX port 5202 “Towards sw-da-lg1” |
| 532 | 8. Add Connection “Exp1_Towards_sw-da-lg1” with I/O Port 5101/5202 |
| 533 | |
| 534 | {{{#!bash |
| 535 | $addc $r1 1 10 in-service false 4102 4201 194300 194400 5 Exp1_From_sw-da-lg1 |
| 536 | $addc $r1 2 10 in-service false 5101 5202 194300 194400 5 Exp1_Toward_sw-da-lg1 |
| 537 | }}} |
| 538 | |
| 539 | ---- |
| 540 | |
| 541 | ==== Performing the experiment and results |
| 542 | |
| 543 | Now you can try installing the above lightpaths by yourself. As prompted by the terminal, press `Return` to install the configuration and perform the test. This will establish the connections between ROADMs as described above, which you can also view with the lines printed. |
| 544 | |
| 545 | |
| 546 | {{{#!shell-session |
| 547 | ... |
| 548 | *** Test configuration 1 |
| 549 | press return to configure and test configuration 1> |
| 550 | ... |
| 551 | *** Installing ROADM configuration 1 for srv1-co1<-->srv1-lg1 (NETCONF) |
| 552 | ... |
| 553 | *** srv1-co1 pinging srv1-lg1 |
| 554 | PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. |
| 555 | 64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=0.724 ms |
| 556 | 64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=0.164 ms |
| 557 | 64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=0.130 ms |
| 558 | |
| 559 | --- 192.168.1.2 ping statistics --- |
| 560 | 3 packets transmitted, 3 received, 0% packet loss, time 2130ms |
| 561 | rtt min/avg/max/mdev = 0.130/0.339/0.724/0.272 ms |
| 562 | |
| 563 | ... |
| 564 | |
| 565 | *** srv1-co1 pinging srv2-lg1 |
| 566 | PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data. |
| 567 | From 192.168.1.1 icmp_seq=1 Destination Host Unreachable |
| 568 | From 192.168.1.1 icmp_seq=2 Destination Host Unreachable |
| 569 | From 192.168.1.1 icmp_seq=3 Destination Host Unreachable |
| 570 | |
| 571 | --- 192.168.1.3 ping statistics --- |
| 572 | 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2192ms |
| 573 | }}} |
| 574 | |
| 575 | As expected, srv1-co1 and srv1-lg1 are able to ping each other because we established the ROADM rules for short-hop configuration. Consequently, srv1-co1 and srv2-lg1 are not able to ping each other because no such connection is established yet. |
| 576 | |
| 577 | Note that the average ping time for srv1-co1 to srv1-lg1 is 0.339 ms. |
| 578 | |
| 579 | ''Note that the Mininet-Optical ROADM dataplane is currently modeled using OvS switching in the Linux kernel, so each hop will add some delay that would not be seen on hardware. Process scheduling, OS and VM overhead, etc. can create additional delays in a software emulator.'' |
| 580 | |
| 581 | == Network Interfaces Configuration for Experiment_2 (long-hop) |
| 582 | |
| 583 | In Experiment 2, we are choosing to pass the optical signal through 2 hops (via a pair of 10km fiber spools with the 32km Manhattan dark fiber). This requires us to establish a connection between srv1-co1 and srv2-lg1. Once you perform the experiment 1 as described above, the script will prompt you to press Return key to perform the test for configuration 2. Before trying the configuration, let's dive into what connections this script will install: |
| 584 | |
| 585 | ==== [WIP] Configuring srv1-co1<=>srv2-lg1 Connection 2 on Mininet-Optical using NETCONF ==== |
| 586 | |
| 587 | * rdm1-co1 (Same As For Experiment 1): |
| 588 | 1. Enable MUX port 4102 “From ToR 1” |
| 589 | 2. Add Connection “From ToR 1” with I/O Port 4102/4201 with bandwidth [192.95;193.05] |
| 590 | 3. Enable DEMUX port 5202 “Towards ToR 1” |
| 591 | 4. Add Connection “Towards ToR 1” with I/O Port 5101/5202 with bandwidth [192.95;193.05] |
| 592 | {{{ |
| 593 | examples/nc_add_connection.py localhost:1834 1 10 in-service false 4102 4201 192950 193050 5 Exp1-FromTor1 |
| 594 | examples/nc_add_connection.py localhost:1834 2 10 in-service false 5101 5202 192950 193050 5 Exp1-TorwardTor1 |
| 595 | }}} |
| 596 | |
| 597 | * ROADM 1 <Not Same!>: |
| 598 | 5. Enable MUX port 4101 “Through Port” (enabled for Snake) |
| 599 | 6. Add Connection “Through In” with I/O Port 4101/4201 with bandwidth [192.95;193.05] |
| 600 | 7. Enable DEMUX port 5201 “Through Port” (enabled for Snake) |
| 601 | 8. Add Connection “Through Out” with I/O Port 5101/5201 with bandwidth [192.95;193.05] |
| 602 | |
| 603 | This time we pass channel 34 through ROADM 1: |
| 604 | {{{ |
| 605 | examples/nc_add_connection.py localhost:1831 1 10 in-service false 4101 4201 192950 193050 5 Exp1-EastR1 |
| 606 | examples/nc_add_connection.py localhost:1831 2 10 in-service false 5101 5201 192950 193050 5 Exp1-WestR1 |
| 607 | }}} |
| 608 | |
| 609 | * ROADM 2 (Same As For ROADM1): |
| 610 | 9. Enable MUX port 4101 “Through Port” (enabled for Snake) |
| 611 | 10. Add Connection “Through In” with I/O Port 4101/4201 with bandwidth [192.95;193.05] |
| 612 | 11. Enable DEMUX port 5201 “Through Port” (enabled for Snake) |
| 613 | 12. Add Connection “Through Out” with I/O Port 5101/5201 with bandwidth [192.95;193.05] |
| 614 | |
| 615 | And pass through ROADM2: |
| 616 | {{{ |
| 617 | examples/nc_add_connection.py localhost:1832 1 10 in-service false 4101 4201 192950 193050 5 Exp1-WestR2 |
| 618 | examples/nc_add_connection.py localhost:1832 2 10 in-service false 5101 5201 192950 193050 5 Exp1-EastR2 |
| 619 | }}} |
| 620 | |
| 621 | * ROADM 3 (Same As For ROADM4): |
| 622 | 13. Enable MUX port 4102 “From ToR 3” |
| 623 | 14. Add Connection “From ToR 3” with I/O Port 4102/4201 with bandwidth [192.95;193.05] |
| 624 | 15. Enable DEMUX port 5202 “Towards ToR 3” |
| 625 | 16. Add Connection “Towards ToR 3” with I/O Port 5101/5202 with bandwidth [192.95;193.05] |
| 626 | |
| 627 | And we drop at ROADM3: |
| 628 | {{{ |
| 629 | examples/nc_add_connection.py localhost:1833 1 10 in-service false 4102 4201 192950 193050 5 Exp1-FromTor2 |
| 630 | examples/nc_add_connection.py localhost:1833 2 10 in-service false 5101 5202 192950 193050 5 Exp1-TorwardTor2 |
| 631 | }}} |
| 632 | |
| 633 | Note: Observe the slightly longer RTT to `server3`, reflecting the increased propagation time across two 22km fibers to reach the "Central Cloud" data center. |
| 634 | |
| 635 | ---- |
| 636 | |
| 637 | = Shutting down Mininet-Optical = |
| 638 | |
| 639 | To exit Mininet-Optical, type `exit` or press control-D at the `mininet-optical>` prompt. |
| 640 | |
| 641 | = Acknowledgments = |
| 642 | |
| 643 | This tutorial is based on the hardware tutorial at [wiki:Workshops/SigComm2022/OpticalTutorial] by T. Chen et al.. |
| 644 | |
| 645 | <<<credits>>> |
| 646 | |
| 647 | = References = |
| 648 | |
| 649 | COSMOS Optical Hardware Testbed Tutorial: [wiki:Workshops/SigComm2022/OpticalTutorial] |
| 650 | |
| 651 | Mininet-Optical Tutorial network script: |
| 652 | https://github.com/Mininet-Optical/mininet-optical/blob/cosmos-tutorial/mnoptical/examples/sigcommtutorial.py |
| 653 | |
| 654 | Mininet-Optical Tutorial configuration script: |
| 655 | https://github.com/Mininet-Optical/mininet-optical/blob/cosmos-tutorial/mnoptical/examples/config-sigcommtutorial.sh |
| 656 | |
| 657 | Mininet-Optical documentation: https://mininet-optical.org |
| 658 | |
| 659 | Mininet-Optical code: https://github.com/mininet-optical/mininet-optical |
| 660 | |
| 661 | (Packet) Mininet web site: https://mininet.org |
| 662 | |
| 663 | |