Changes between Version 14 and Version 15 of Tutorials/Wireless/mmwaveRFSoC


Ignore:
Timestamp:
Oct 18, 2021, 4:52:51 PM (3 years ago)
Author:
skrimpon
Comment:

—

Legend:

Unmodified
Added
Removed
Modified

  • Tutorials/Wireless/mmwaveRFSoC

    v14 v15  
    22
    33
    4 = Using COSMOS SandBox1 =
     4= Millimeter Wave Experimentation in COSMOS SB1 with Xilinx RFSoC =
    55
    66{{{#!box information
    77Upgrade in Progress
    88}}}
    9 Last updated: October 12, 2020
     9Authors:
     10- Panagiotis Skrimponis, New York University: ps3857[at]nyu.edu
     11- Prasanthi Maddala, Rutgers University
    1012
    11 === Description ===
    12 A pair of Sivers EVK06002s, evaluation kits for Sivers IMA TRX BF/01, are deployed on COSMOS SandBox1. TRX BF/01 is a 16+16 IEEE802.11ad Beamforming Transceiver with a complete Radio front-end with 57-66 GHz mmWave frequency range. This tutorial demonstrates how  to transmit and receive a signal in the 60GHz mmWave spectrum using these Sivers front-ends.
     13Last updated: October 18, 2021
    1314
    14 Current mmWave setup in COSMOS SB1 looks as shown in the figure below.
     15=== Summary ===
     16=== Publications ===
     17For more information about the integration of the advanced programmable and open-source millimeter wave array systems in COSMOS, please read:
     181. T. Chen, P. Maddala, P. Skrimponis, J. Kolodziejski, X. Gu, A. Paidimarri, S. Rangan, G. Zussman, and I. Seskar, "Programmable and open-access millimeter-wave radios in the PAWR COSMOS testbed," in Proc. ACM Mobicom'21 Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization (WiNTECH'21) (to appear), 2021. [https://wimnet.ee.columbia.edu/wp-content/uploads/2021/08/COSMOS_mmWave_WiNTECH2021.pdf Download]
     191. D. Raychaudhuri, I. Seskar, G. Zussman, T. Korakis, D. Kilper, T. Chen, J. Kolodziejski, M. Sherman, Z. Kostic, X. Gu, H. Krishnaswamy, S. Maheshwari, P. Skrimponis, and C. Gutterman, "Challenge: COSMOS: A city-scale programmable testbed for experimentation with advanced wireless," in Proc. ACM Mobicom'20, 2020." ​[https://dl.acm.org/doi/10.1145/3372224.3380891 ACM] [https://wimnet.ee.columbia.edu/wp-content/uploads/2020/02/MobiCom2020_COSMOS.pdf Download] [https://wimnet.ee.columbia.edu/wp-content/uploads/2020/09/mobicom2020_cosmos_slides.pdf Presentation] [https://youtu.be/t2XxgExefns Long Video] [https://youtu.be/BIMubLpxyAc Short Video]
     201. P. Skrimponis, P. Maddala, J. Kolodziejski, I. Seskar, and S. Rangan, "60 GHz Beam Tracking Testbed," Brooklyn 6G Summit, 2021. [https://docs.google.com/presentation/d/1VHWGFwFMLlsBRYP4Um7cTyLeSYhuDDcT/edit?usp=sharing&ouid=104982133251171242392&rtpof=true&sd=true Presentation] [https://youtu.be/-tmH6C33Cl0 Video]
     21Please cite the above papers if you use the hardware. Please email Panagiotis Skrimponis (ps3857[at]nyu.edu) if you have any questions.
    1522
    16 || [[Image(MISO_tutorial.jpg, 800px)]] ||
     23== Experiment Setup ==
    1724
    18 In the benchtop 60GHz setup of SandBox1, either USRP X310 or RFSoC ZCU111 can be used for baseband processing. RF Switch boxes as shown in the figure are used to select between X310 and RFSoC. RF switch commands are explained at wiki:/Resources/Services/RFSwitch. To select ZCU111 as baseband processing units, all the switches in the RF switch boxes are configured to port 2.
    19 
    20 This experiment uses the Xilinx ZynqMP Ultrascale+ RFSoC ZCU111 board for the baseband processing. Sivers control software on srv3, srv4 talk to the mmWave front-ends over USB links. Host GNURadio applications on srv3, srv4, send/receive data samples to/from the RFSoC over 1G Ethernet link.
    21  
    22 === Prerequisites ===
    23 In order to access the test bed, create a reservation and have it approved by the reservation service. Access to the resources is granted after the reservation is confirmed. Please follow the process shown on the COSMOS [wiki:GettingStarted getting started] page to get started.
    24 
    25 === Resources required ===
    26 - Servers
    27  - srv1-in1
    28  - srv1-in2
    29 -
    30 
    31 2 servers srv3-lg1, srv4-lg1, 2 Xilinx RFSoCs rfdev3-3, rfdev3-4 and both the Sivers platforms rfdev3-5, rfdev3-6 on COSMOS SB1.
    32 === Tutorial Setup ===
    33 
    34 Follow the steps below to gain access to the [wiki:Architecture/Domains/cosmos_sb1 sandbox 1 console] and set up nodes with appropriate images.
    35  1. If you don't have one already, sign up for a [https://www.cosmos-lab.org/portal-2/ COSMOS account]
    36  1. [wiki:/GettingStarted#MakeaReservation Create a resource reservation] on sandbox 1
    37  1. [wiki:/GettingStarted#LogintoyourReservation Login] into sandbox 1 console (console.sb1.cosmos-lab.org) with two SSH sessions.
    38  1. Make sure all the nodes and devices of this reservation are turned off:
    39 {{{#!shell
    40 omf tell -a offh -t system:topo:allres
     25=== COSMOS SB1 Nodes ===
     26Follow the steps below to gain access to COSMOS [wiki:Architecture/Domains/cosmos_sb1 sb1] and set up nodes with appropriate image:
     271. If you don't have a COSMOS account already, you need to [https://www.cosmos-lab.org/portal-2/ sign up].
     281. [wiki:/GettingStarted#MakeaReservation Create a resource reservation] on [wiki:Architecture/Domains/cosmos_sb1 sb1].
     291. [wiki:/GettingStarted#LogintoyourReservation Login] into [wiki:Architecture/Domains/cosmos_sb1 sb1] `console.sb1.cosmos-lab.org` using SSH.
     30{{{#!shell-session
     31not_a_user@laptop:~$ ssh -Y your_username@console.sb1.cosmos-lab.org
    4132}}}
    42  1. The image rfsoc_sivers_sb1.ndz is [baseline_uhd.ndz] with Sivers control software and the Xilinx XSCT software installed
    43   Load rfsoc_sivers_sb1.ndz on srv3,srv4. 
    44 {{{#!shell
    45 omf load -i rfsoc_sivers_sb1.ndz -t srv3-lg1,srv4-lg1
     331. Make sure all the nodes and devices of this reservation are turned off:
     34{{{#!shell-session
     35your_username@console:~$ omf tell -a offh -t system:topo:allres
    4636}}}
    47  1. Turn all the required resources on and check the status
    48 {{{#!shell
    49 omf tell -a on -t srv3-lg1,srv4-lg1,rfdev3-3,rfdev3-4,rfdev3-5,rfdev3-6
     371. Load the {{{rfsoc_sivers_sb1.ndz}}} on `srv1-in1` and `srv1-in2`.
     38{{{#!shell-session
     39your_username@console:~$ omf load -i rfsoc_sivers_sb1.ndz -t srv1-in1,srv1-in2
    5040}}}
    51  1. Check that the nodes are turned on
    52 {{{#!shell
    53 omf stat -t system:topo:allres
     411. Power on all the required resources.
     42{{{#!shell-session
     43your_username@console:~$ omf tell -a on -t srv1-in1,srv1-in2,rfdev3-in1,rfdev3-in2,sdr2-in1,sdr2-in2
    5444}}}
    55  1. Make sure the RF switch boxes are configured to use RFSoCs (all switches set to port 2)
     451. Check the status of the nodes and devices of [wiki:Architecture/Domains/cosmos_sb1 sb1].
    5646{{{#!shell-session
    57 root@console:~# curl am1.cosmos-lab.org:5054/rf_switch/status?name=rfsw1.sb1.cosmos-lab.org,rfsw2.sb1.cosmos-lab.org
     47your_username@console:~$ omf stat -t system:topo:allres
     48}}}
     491. Configure the RF switches to connect RFSoC with Sivers array
     50{{{#!shell-session
     51your_username@console:~$ curl "am1.cosmos-lab.org:5054/rf_switch/set?name=rfsw1.sb1.cosmos-lab.org,rfsw2.sb1.cosmos-lab.org&switch=1,2,3,4&port=2"
     52}}}
     531. Make sure the RF switches are configured correctly (all switches set to port 2)
     54{{{#!shell-session
     55your_username@console:~# curl am1.cosmos-lab.org:5054/rf_switch/status?name=rfsw1.sb1.cosmos-lab.org,rfsw2.sb1.cosmos-lab.org
    5856
    5957<response status="OK">
    … …  
    7270</response>
    7371}}}
    74  1. If some or all the ports are set to "1", set them to port "2"
    75 {{{#!shell
    76 curl "am1.cosmos-lab.org:5054/rf_switch/set?name=rfsw1.sb1.cosmos-lab.org,rfsw2.sb1.cosmos-lab.org&switch=1,2,3,4&port=2"
    77 }}}
    78  1. ssh to the nodes, use option -Y for using GUI.
    7972
    80 === Experiment Execution ===
     73=== RFSoC Setup ===
     74Every time we power cycle the FPGAs, we need to download the firmware (i.e., linux image, rootfs, bistream) using the Xilinx XSCT tool over JTAG. Please note the this process will take around 5-10'.
    8175
    82 ==== Configure the 10G interface ====
    83 The IP addresses for Ethernet Port 1(10G) on the RFSoCs rfdev3-3 and rfdev3-4 are hard-coded to `10.39.6.3` and `10.39.6.4` respectively. To access them from `srv3-lg1` or `srv4-lg1`, configure the network interface `enp1s0` of `srv3-lg1` to `10.39.1.3` and `srv4-lg1` to `10.39.1.4`, as follows
    8476{{{#!shell-session
    85 root@srv3-lg1:~# ip link set enp1s0 mtu 9000 up
    86 root@srv3-lg1:~# ip addr add 10.39.1.3/16 dev enp1s0
    87 root@srv3-lg1:~# ip addr show dev enp1s0
    88 4: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    89     link/ether f4:52:14:83:b9:30 brd ff:ff:ff:ff:ff:ff
    90     inet 10.39.1.3/16 brd 10.39.255.255 scope global enp1s0
    91        valid_lft forever preferred_lft forever
    92     inet6 fe80::f652:14ff:fe83:b930/64 scope link
    93        valid_lft forever preferred_lft forever
     77your_username@console:~$ ssh -Y root@srv1-in1
     78root@srv1-in1:~$ cd mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in1/
     79root@srv1-in1:~/mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in1$ xsct download_firmware.tcl
    9480}}}
    9581
    96 
    97 ==== Prepare RFSoC boards ====
    98 We need to download the FPGA firmware, the linux image and the rootfs over the JTAG. We use the Xilinx XSCT tool to do that. Note that this process will take around 5-10'.
    99 
    100 * Configure the FPGA firmware of the `rfdev3-3` from the `srv3-lg1` as follows:
    10182{{{#!shell-session
    102 root@srv3-lg1:~# cd rfsoc/firmware/sb1_rfdev3_3/
    103 root@srv3-lg1:~/rfsoc/firmware/sb1_rfdev3_3# xsct download_firmware.tcl
     83your_username@console:~$ ssh -Y root@srv1-in2
     84root@srv1-in2:~$ cd mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in2/
     85root@srv1-in2:~/mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in2$ xsct download_firmware.tcl
    10486}}}
    10587
    106 * Configure the FPGA firmware of the `rfdev3-4` from the `srv4-lg1` as follows:
     88=== Network Interface Setup ===
     89Using the information available in [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb1  COSMOS WiKi] we need to configure the ethernet interfaces.
     90[[BR]]
     91||||||||= First corner (1-1) =||
     92||  ''Device''  ||  ''Control Network''  ||  ''Data Network 1''  ||  ''Data Network 2''  ||
     93|| Support Server  ||  10.37.1.3 (srv1-in1)  ||  10.38.1.3 (srv1a-in1)  ||  10.39.1.3 (srv1b-in1)  ||
     94|| RFSoC Device  ||  10.37.6.3 (sdr2-in1)  ||  10.38.6.3 (sdr2-in1a)  ||  10.39.6.3 (sdr2-in1b)  ||
    10795{{{#!shell-session
    108 root@srv4-lg1:~# cd rfsoc/firmware/sb1_rfdev3_4/
    109 root@srv4-lg1:~/rfsoc/firmware/sb1_rfdev3_4# xsct download_firmware.tcl
     96root@srv1-in1:~# ip link set enp1s0 mtu 9000 up
     97root@srv1-in1:~# ip link set enp3s0 mtu 9000 up
     98root@srv1-in1:~# ip addr add 10.38.1.3/16 dev enp1s0
     99root@srv1-in1:~# ip addr add 10.39.1.3/16 dev enp3s0
     100}}}
     101||||||||= Second corner (20-20) =||
     102||  ''Device''  ||  ''Control Network''  ||  ''Data Network 1''  ||  ''Data Network 2''  ||
     103|| Support Server  ||  10.37.1.4 (srv1-in2)  ||  10.38.1.4 (srv1a-in2)  ||  10.39.1.4 (srv1b-in2) ||
     104|| RFSoC Device  ||  10.37.6.4 (sdr2-in2)  ||  10.38.6.4 (sdr2-in2a)  ||  10.39.6.4 (sdr2-in2b)  ||
     105{{{#!shell-session
     106root@srv1-in2:~# ip link set enp1s0 mtu 9000 up
     107root@srv1-in2:~# ip link set enp3s0 mtu 9000 up
     108root@srv1-in2:~# ip addr add 10.38.1.4/16 dev enp1s0
     109root@srv1-in2:~# ip addr add 10.39.1.4/16 dev enp3s0
    110110}}}
    111111
    112 * You can connect to FPGA over a serial monitor as follows:
     112=== Software Update ===
     113Update the software:
    113114{{{#!shell-session
    114 root@srv3-lg1:~# minicom -D /dev/ttyUSB5
    115 root@srv4-lg1:~# minicom -D /dev/ttyUSB1
     115root@srv1-in1:~$ cd mmwsdr
     116root@srv1-in1:~/mmwsdr$ git pull
    116117}}}
    117118
    118 ==== Prepare Sivers 60GHz front-ends ====
    119 * To demonstrate the experiment here, we use Sivers front-end SN0240 as transmitter and SN0243 as receiver
     119{{{#!shell-session
     120root@srv1-in2:~$ cd mmwsdr
     121root@srv1-in2:~/mmwsdr$ git pull
     122}}}
    120123
    121 ||  SN0240 as transmitter                    ||  SN0243 as receiver                       ||
    122 ||  `root@srv4-lg1:~/ederenv# ./start_mb1.sh  --gui SN0240` || `root@srv3-lg1:~/ederenv# ./start_mb1.sh  --gui SN0243` ||
    123 ||  Click "Enable Tx" and "LO leakage Cal"   ||  Click "Enable Rx"                         ||
    124 ||  [[Image(Sivers_TX_SN0240.jpg, 600px)]]      ||  [[Image(Sivers_RX_SN0243.jpg, 600px)]]      ||
    125 === Execution ===
    126 ==== Run the experiment ====
    127 * Start the transmitter application on the TX node(srv4-lg1). Run the tx_waveforms python script in the examples folder:
    128 {{{#!shell-session
    129 root@srv4-lg1:~/rfsoc/examples# ./tx_waveforms
    130 }}}
    131 ||  Tx Waveform   ||
    132 ||  [[Image(rfsoc_tx.png, 700px)]]  ||
     124== Demos ==
     125In the following subsections, you can find detailed descriptions for each demo.
    133126
    134 * Start the receiver application on the RX node(srv3-lg1). Run the rx_waveforms python script in the examples folder:
    135 {{{#!shell-session
    136 root@srv3-lg1:~/rfsoc/examples# ./rx_waveforms
    137 }}}
    138 ||  Rx Waveform  ||
    139 ||  [[Image(rfsoc_rx.png, 700px)]]  ||
     127=== Basic ===
     128=== Calibration ===
     129=== Channel Sounder ===
     130=== Array Pattern ===
     131=== Beam Tracking Measurements ===