Changes between Version 2 and Version 3 of Architecture/Network


Ignore:
Timestamp:
Oct 14, 2019, 6:36:05 PM (5 years ago)
Author:
msherman
Comment:

—

Legend:

Unmodified
Added
Removed
Modified

  • Architecture/Network

    v2 v3  
    11[[TOC(hardware*,depth=3)]]
    2 == Networking ==
     2
     3== Network Architecture ==
    34There exist three primary networks that are accessible by experimenters in COSMOS. Device connections to these networks are determined by the number of interfaces and their maximum bandwidth. Every device has at least a connection to the [#ControlPlane Control Plane]. Then depending on the number of additional interfaces, the device may have a connection to either one or both of the [#DataPlane Data Planes]. Please see the device specific pages for more detailed information.
    45
    … …  
    89=== Data Plane ===
    910There exist two parallel data plane networks that are available to experimenters. The data plane networks are interconnected with Dell Z9100-ON switches providing up to 100G connectivity between devices. The switches are built on the Broadcom Tomahawk chipset and support Openflow 1.3 as well as ONIE. The data planes are meant for latency and bandwidth sensitive experiment traffic where performance is intended to be deterministic.
     11
     12=== Isolation and infrastructure
     13
     14We enforce a split between "Control", "Data", and "Infrastructure" networks. Control and Infrastructure share common physical hardware (Switches, cables, etc.), using VLANs to provide isolation. Together these provide services and user access to the testbed.
     15
     16The Data networks operate on physically separate hardware, to preserve repeatability of experiments, and to prevent experiments from interfering with testbed services.
     17
     18L2 Networks do not span sites, with access being done via L3 tunnels, or via public internet.
     19
     20The Instrumentation network is special, in that it is accessible to experiments, but is managed by infrastructure services. This provides, for example, optical or RF devices that need direct L2 access to multiple domains. Access to this network is granted on a case by case basis to experiments. This is also the only L2 network that spans sites, via L2 Tunnel.
     21
     22Consoles give users access from outside and communicate with services on the DMZ network, and access to the testbed devices over control.
     23
     24=== Network Roles
     25Each Domain has a control network, that is statically configured, as well as two configurable data plane networks. Imaging and other configuration tasks is done via the control network, while experiment and radio traffic is passed on the data networks.
     26
     27In addition, each "site" has some infrastructure networks.
     28
     29* CM: used for power control of nodes, not directly accessible to users, instead is managed by services
     30* Inside: Private network for service traffic, VM Storage traffic, etc.
     31* DMZ: User accessible network for service traffic
     32* MGMT: Private network for control of network devices, VM Cluster synchronization, etc.
     33
     34
     35=== Logical Sites ===
     36Currently, there are two logically distinct sites.
     37* The NYC COSMOS Deployment
     38 * bed.cosmos, including exterior radios and optical underlay
     39 * sb2.cosmos
     40* WINLAB, 671 Rt 1 South, North Brunswick, NJ
     41 * sb1.cosmos
     42 * grid.orbit
     43 * outdoor.orbit
     44 * sb1 - sb10.orbit
     45
     46=== Vlan Assignment
     47
     48{{{
     49#!Markdown
     50
     51| vlan id | subnet        | name           | location |
     52|---------|---------------|----------------|----------|
     53| 101     | 10.101.0.0/16 | mgmt_win       | winlab   |
     54| 102     | 10.102.0.0/16 | inside_win     | winlab   |
     55| 103     | 10.103.0.0/16 | cm_win         | winlab   |
     56| 104     | 10.104.0.0/16 | dmz_win        | winlab   |
     57| 105     | 10.105.0.0/16 | mgmt_nyc       | nyc      |
     58| 106     | 10.106.0.0/16 | inside_nyc     | nyc      |
     59| 107     | 10.107.0.0/16 | cm_nyc         | nyc      |
     60| 108     | 10.108.0.0/16 | dmz_nyc        | nyc      |
     61| 109     | 10.109.0.0/16 | instrument     | both     |
     62| 110     | 10.110.0.0/16 | bed_ctrl       | nyc      |
     63| 111     | 10.110.0.0/16 | bed_data1      | nyc      |
     64| 112     | 10.110.0.0/16 | bed_data2      | nyc      |
     65| 113     | 10.110.0.0/16 | sb1_ctrl       | winlab   |
     66| 114     | 10.110.0.0/16 | sb1_data1      | winlab   |
     67| 115     | 10.110.0.0/16 | sb1_data2      | winlab   |
     68| 116     | 10.110.0.0/16 | sb2_ctrl       | nyc      |
     69| 117     | 10.110.0.0/16 | sb2_data1      | nyc      |
     70| 118     | 10.110.0.0/16 | sb2_data2      | nyc      |
     71
     72}}}
     73
     74
     75
     76
     77[[Image(COSMOS Net Arch.png, 600px, right)]]
     78