[[Include(WikiToC)]] = Millimeter Wave Experimentation in COSMOS SB1 with Xilinx RFSoC = {{{#!box information Upgrade in Progress }}} Authors: - Panagiotis Skrimponis, New York University: ps3857[at]nyu.edu - Prasanthi Maddala, Rutgers University Last updated: October 18, 2021 === Summary === === Publications === For more information about the integration of the advanced programmable and open-source millimeter wave array systems in COSMOS, please read: 1. T. Chen, P. Maddala, P. Skrimponis, J. Kolodziejski, X. Gu, A. Paidimarri, S. Rangan, G. Zussman, and I. Seskar, "Programmable and open-access millimeter-wave radios in the PAWR COSMOS testbed," in Proc. ACM Mobicom'21 Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization (WiNTECH'21) (to appear), 2021. [https://wimnet.ee.columbia.edu/wp-content/uploads/2021/08/COSMOS_mmWave_WiNTECH2021.pdf Download] 1. D. Raychaudhuri, I. Seskar, G. Zussman, T. Korakis, D. Kilper, T. Chen, J. Kolodziejski, M. Sherman, Z. Kostic, X. Gu, H. Krishnaswamy, S. Maheshwari, P. Skrimponis, and C. Gutterman, "Challenge: COSMOS: A city-scale programmable testbed for experimentation with advanced wireless," in Proc. ACM Mobicom'20, 2020." ​[https://dl.acm.org/doi/10.1145/3372224.3380891 ACM] [https://wimnet.ee.columbia.edu/wp-content/uploads/2020/02/MobiCom2020_COSMOS.pdf Download] [https://wimnet.ee.columbia.edu/wp-content/uploads/2020/09/mobicom2020_cosmos_slides.pdf Presentation] [https://youtu.be/t2XxgExefns Long Video] [https://youtu.be/BIMubLpxyAc Short Video] 1. P. Skrimponis, P. Maddala, J. Kolodziejski, I. Seskar, and S. Rangan, "60 GHz Beam Tracking Testbed," Brooklyn 6G Summit, 2021. [https://docs.google.com/presentation/d/1VHWGFwFMLlsBRYP4Um7cTyLeSYhuDDcT/edit?usp=sharing&ouid=104982133251171242392&rtpof=true&sd=true Presentation] [https://youtu.be/-tmH6C33Cl0 Video] Please cite the above papers if you use the hardware. Please email Panagiotis Skrimponis (ps3857[at]nyu.edu) if you have any questions. == Experiment Setup == === COSMOS SB1 Nodes === Follow the steps below to gain access to COSMOS [wiki:Architecture/Domains/cosmos_sb1 sb1] and set up nodes with appropriate image: 1. If you don't have a COSMOS account already, you need to [https://www.cosmos-lab.org/portal-2/ sign up]. 1. [wiki:/GettingStarted#MakeaReservation Create a resource reservation] on [wiki:Architecture/Domains/cosmos_sb1 sb1]. 1. [wiki:/GettingStarted#LogintoyourReservation Login] into [wiki:Architecture/Domains/cosmos_sb1 sb1] `console.sb1.cosmos-lab.org` using SSH. {{{#!shell-session not_a_user@laptop:~$ ssh -Y your_username@console.sb1.cosmos-lab.org }}} 1. Make sure all the nodes and devices of this reservation are turned off: {{{#!shell-session your_username@console:~$ omf tell -a offh -t system:topo:allres }}} 1. Load the {{{rfsoc_sivers_sb1.ndz}}} on `srv1-in1` and `srv1-in2`. {{{#!shell-session your_username@console:~$ omf load -i rfsoc_sivers_sb1.ndz -t srv1-in1,srv1-in2 }}} 1. Power on all the required resources. {{{#!shell-session your_username@console:~$ omf tell -a on -t srv1-in1,srv1-in2,rfdev3-in1,rfdev3-in2,sdr2-in1,sdr2-in2 }}} 1. Check the status of the nodes and devices of [wiki:Architecture/Domains/cosmos_sb1 sb1]. {{{#!shell-session your_username@console:~$ omf stat -t system:topo:allres }}} 1. Configure the RF switches to connect RFSoC with Sivers array {{{#!shell-session your_username@console:~$ curl "am1.cosmos-lab.org:5054/rf_switch/set?name=rfsw1.sb1.cosmos-lab.org,rfsw2.sb1.cosmos-lab.org&switch=1,2,3,4&port=2" }}} 1. Make sure the RF switches are configured correctly (all switches set to port 2) {{{#!shell-session your_username@console:~# curl am1.cosmos-lab.org:5054/rf_switch/status?name=rfsw1.sb1.cosmos-lab.org,rfsw2.sb1.cosmos-lab.org }}} === RFSoC Setup === Every time we power cycle the FPGAs, we need to download the firmware (i.e., linux image, rootfs, bistream) using the Xilinx XSCT tool over JTAG. Please note the this process will take around 5-10'. {{{#!shell-session your_username@console:~$ ssh -Y root@srv1-in1 root@srv1-in1:~$ cd mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in1/ root@srv1-in1:~/mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in1$ xsct download_firmware.tcl }}} {{{#!shell-session your_username@console:~$ ssh -Y root@srv1-in2 root@srv1-in2:~$ cd mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in2/ root@srv1-in2:~/mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in2$ xsct download_firmware.tcl }}} === Network Interface Setup === Using the information available in [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb1 COSMOS WiKi] we need to configure the ethernet interfaces. [[BR]] ||||||||= First corner (1-1) =|| || ''Device'' || ''Control Network'' || ''Data Network 1'' || ''Data Network 2'' || || Support Server || 10.37.1.3 (srv1-in1) || 10.38.1.3 (srv1a-in1) || 10.39.1.3 (srv1b-in1) || || RFSoC Device || 10.37.6.3 (sdr2-in1) || 10.38.6.3 (sdr2-in1a) || 10.39.6.3 (sdr2-in1b) || {{{#!shell-session root@srv1-in1:~# ip link set enp1s0 mtu 9000 up root@srv1-in1:~# ip link set enp3s0 mtu 9000 up root@srv1-in1:~# ip addr add 10.38.1.3/16 dev enp1s0 root@srv1-in1:~# ip addr add 10.39.1.3/16 dev enp3s0 }}} ||||||||= Second corner (20-20) =|| || ''Device'' || ''Control Network'' || ''Data Network 1'' || ''Data Network 2'' || || Support Server || 10.37.1.4 (srv1-in2) || 10.38.1.4 (srv1a-in2) || 10.39.1.4 (srv1b-in2) || || RFSoC Device || 10.37.6.4 (sdr2-in2) || 10.38.6.4 (sdr2-in2a) || 10.39.6.4 (sdr2-in2b) || {{{#!shell-session root@srv1-in2:~# ip link set enp1s0 mtu 9000 up root@srv1-in2:~# ip link set enp3s0 mtu 9000 up root@srv1-in2:~# ip addr add 10.38.1.4/16 dev enp1s0 root@srv1-in2:~# ip addr add 10.39.1.4/16 dev enp3s0 }}} === Environment Setup === To control the Sivers using a local connection we need to setup the FTDI drivers for every new ssh connection. {{{#!shell-session root@srv1-in1:~$ source ~/mmwsdr/host/scripts/sivers_ftdi.sh }}} === Software Update === Update the software: {{{#!shell-session root@srv1-in1:~$ cd mmwsdr root@srv1-in1:~/mmwsdr$ git pull }}} {{{#!shell-session root@srv1-in2:~$ cd mmwsdr root@srv1-in2:~/mmwsdr$ git pull }}} == Demos == In the following subsections, you can find detailed descriptions for each demo. === Basic === - '''video.py''' In this script we show the control of the SDR movement. Each SDR is mounted on top of an XY-Table that allows for independent movement in the horizontal plane and rotation along the vertical axis. To control the XY-Table we use the HTTP API detailed in the following [https://wiki.cosmos-lab.org/wiki/Resources/Services/XYTable page]. To visualize this motion we open a live stream from a camera using OpenCV. {{{#!shell-session root@srv1-in1:~/mmwsdr/host/demos/basic$ python video.py --node srv1-in1 }}} - '''onenode.py:''' In this demo we control a single SDR node. We create an SDR object and an XY-Table object using the `mmwsdr` library. The SDR object configures and controls a Xilinx RFSoC ZCU111 eval board and a Sivers IMA transceiver board. A user can provide arguments to the script, such as the carrier frequency, the COSMOS node id and the transceiver mode. The script by default starts a local connection with a carrier frequency at 60.48 GHz in receive mode. 1. Start the transmitter in `srv1-in1`: {{{#!shell-session root@srv1-in1:~/mmwsdr/host/demos/basic$ python onenode.py --freq 60.48e9 --node srv1-in1 --mode tx }}} 2. Start the receive in `srv1-in2`: {{{#!shell-session root@srv1-in2:~/mmwsdr/host/demos/basic$ python onenode.py --freq 60.48e9 --node srv1-in2 --mode rx }}} 3. Observe the received signal in frequency-domain: - '''ederarray.py:''' The Python drivers of the Sivers library require Python 2. One way to remove this dependence is to remotely control the Sivers array with an HTTP server. The script 'ederarray.py' instantiates an object to control the Sivers array and starts the HTTP server to listen to control commands {{{#!shell-session root@srv1-in1:~/mmwsdr/host/mmwsdr/array$ python ederarray.py -u SN0240 }}} {{{#!shell-session root@srv1-in2:~/mmwsdr/host/mmwsdr/array$ python ederarray.py -u SN0243 }}} {{{#!comment The FTDI drivers of the Sivers array require the use of Python 2. To alleviate this we create an HTTP server }}} - '''twonode.py:''' In this demo we control both the SDR nodes using a single script. We create an SDR object and an XY-Table object for each node using the `mmwsdr` library. The SDR object configures and controls a Xilinx RFSoC ZCU111 eval board and a Sivers IMA transceiver board. Shown below are the commands to start an HTTP server on srv1-in1 and to run twonode.py script on srv1-in2. twonode.py script in this case connects directly to its local Sivers front-end SN0240 and it connects to SN0243 via the HTTP server running on srv1-in2. 1. Start the remote Eder server on srv1-in1 : {{{#!shell-session root@srv1-in1:~/mmwsdr/host/mmwsdr/array$ python ederarray.py -u SN0240 }}} 2. Start the experiment on the local server srv1-in2: {{{#!shell-session root@srv1-in2:~/mmwsdr/host/demos/basic$ python twonode.py --freq 60.48e9 --mode tx --node srv1-in2 }}} === Channel Sounder === In this demo, we show a frequency-domain channel sounder at 60 GHz. We generate N_FFT symbols in frequency domain. We generate a wide-band sequence filling [sc_min, sc_max] sub-carries with random 4-QAM symbols. We use IFFT to get the time-domain TX sequence. We transmit the data from SDR1 with cyclic repeat. We receive 100 frames of N_FFT data from SDR2. The user can select to save or process the data. When we process the data In this section we demonstrated a frequency-domain channel sounder. We generate a tx sequence using `mmwave.utils.waveform.wideband` function. === Calibration === In this demo, we use the calibration techniques by Sivers. The performance of the calibration can be further improved with the IQ calibration techniques described in the following papers, 1. A. Dhananjay, K. Zheng, M. Mezzavilla, L. Iotti, D. Shasha, and S. Rangan, "Pi-Radio v1: Calibration techniques to enable fully-digital beamforming at 60 GHz," Computer Networks, Volume 196, 2021, 108220, ISSN 1389-1286, https://doi.org/10.1016/j.comnet.2021.108220. 2. A. Dhananjay, K. Zheng, J. Haarla, L. Iotti, M. Mezzavilla, D. Shasha, and S. Rangan, "Calibrating a 4-channel Fully-Digital 60 GHz SDR," In Proc. of the 14th International Workshop on Wireless Network Testbeds, Experimental evaluation & Characterization (WiNTECH'20). Association for Computing Machinery, New York, NY, USA, 40–47. https://doi.org/10.1145/3411276.3412195 Originally this techniques target a 4-channel fully-digital array. However, the algorithms are versatile and can target the phased arrays as well. ==== Calibrate RX IQ imbalance ==== {{{#!shell-session root@srv1-in1:~/mmwsdr/host/mmwsdr/array$ python ederarray.py --unit SN0240 }}} ==== Calibrate TX IQ imbalance ==== {{{#!shell-session root@srv1-in1:~/mmwsdr/host/mmwsdr/array$ python ederarray.py --unit SN0240 }}} === Array Pattern === {{{#!shell-session root@srv1-in1:~/mmwsdr/host/mmwsdr/array$ python ederarray.py --unit SN0240 }}} {{{#!shell-session root@srv1-in2:~/mmwsdr/host/demos/basic$ python twonode.py --freq 60.48e9 --mode tx }}} === Beam Tracking ===