[[Include(WikiToC)]] = Millimeter Wave Experimentation in COSMOS SB1 with Xilinx RFSoC = {{{#!box information Upgrade in Progress }}} Authors: - Panagiotis Skrimponis, New York University: ps3857[at]nyu.edu - Prasanthi Maddala, Rutgers University Last updated: October 18, 2021 === Summary === === Publications === For more information about the integration of the advanced programmable and open-source millimeter wave array systems in COSMOS, please read: 1. T. Chen, P. Maddala, P. Skrimponis, J. Kolodziejski, X. Gu, A. Paidimarri, S. Rangan, G. Zussman, and I. Seskar, "Programmable and open-access millimeter-wave radios in the PAWR COSMOS testbed," in Proc. ACM Mobicom'21 Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization (WiNTECH'21) (to appear), 2021. [https://wimnet.ee.columbia.edu/wp-content/uploads/2021/08/COSMOS_mmWave_WiNTECH2021.pdf Download] 1. D. Raychaudhuri, I. Seskar, G. Zussman, T. Korakis, D. Kilper, T. Chen, J. Kolodziejski, M. Sherman, Z. Kostic, X. Gu, H. Krishnaswamy, S. Maheshwari, P. Skrimponis, and C. Gutterman, "Challenge: COSMOS: A city-scale programmable testbed for experimentation with advanced wireless," in Proc. ACM Mobicom'20, 2020." ​[https://dl.acm.org/doi/10.1145/3372224.3380891 ACM] [https://wimnet.ee.columbia.edu/wp-content/uploads/2020/02/MobiCom2020_COSMOS.pdf Download] [https://wimnet.ee.columbia.edu/wp-content/uploads/2020/09/mobicom2020_cosmos_slides.pdf Presentation] [https://youtu.be/t2XxgExefns Long Video] [https://youtu.be/BIMubLpxyAc Short Video] 1. P. Skrimponis, P. Maddala, J. Kolodziejski, I. Seskar, and S. Rangan, "60 GHz Beam Tracking Testbed," Brooklyn 6G Summit, 2021. [https://docs.google.com/presentation/d/1VHWGFwFMLlsBRYP4Um7cTyLeSYhuDDcT/edit?usp=sharing&ouid=104982133251171242392&rtpof=true&sd=true Presentation] [https://youtu.be/-tmH6C33Cl0 Video] Please cite the above papers if you use the hardware. Please email Panagiotis Skrimponis (ps3857[at]nyu.edu) if you have any questions. == Experiment Setup == === COSMOS SB1 Nodes === Follow the steps below to gain access to COSMOS [wiki:Architecture/Domains/cosmos_sb1 sb1] and set up nodes with appropriate image: 1. If you don't have a COSMOS account already, you need to [https://www.cosmos-lab.org/portal-2/ sign up]. 1. [wiki:/GettingStarted#MakeaReservation Create a resource reservation] on [wiki:Architecture/Domains/cosmos_sb1 sb1]. 1. [wiki:/GettingStarted#LogintoyourReservation Login] into [wiki:Architecture/Domains/cosmos_sb1 sb1] `console.sb1.cosmos-lab.org` using SSH. {{{#!shell-session not_a_user@laptop:~$ ssh -Y your_username@console.sb1.cosmos-lab.org }}} 1. Make sure all the nodes and devices of this reservation are turned off: {{{#!shell-session your_username@console:~$ omf tell -a offh -t system:topo:allres }}} 1. Load the {{{rfsoc_sivers_sb1.ndz}}} on `srv1-in1` and `srv1-in2`. {{{#!shell-session your_username@console:~$ omf load -i rfsoc_sivers_sb1.ndz -t srv1-in1,srv1-in2 }}} 1. Power on all the required resources. {{{#!shell-session your_username@console:~$ omf tell -a on -t srv1-in1,srv1-in2,rfdev3-in1,rfdev3-in2,sdr2-in1,sdr2-in2 }}} 1. Check the status of the nodes and devices of [wiki:Architecture/Domains/cosmos_sb1 sb1]. {{{#!shell-session your_username@console:~$ omf stat -t system:topo:allres }}} 1. Configure the RF switches to connect RFSoC with Sivers array {{{#!shell-session your_username@console:~$ curl "am1.cosmos-lab.org:5054/rf_switch/set?name=rfsw1.sb1.cosmos-lab.org,rfsw2.sb1.cosmos-lab.org&switch=1,2,3,4&port=2" }}} 1. Make sure the RF switches are configured correctly (all switches set to port 2) {{{#!shell-session your_username@console:~# curl am1.cosmos-lab.org:5054/rf_switch/status?name=rfsw1.sb1.cosmos-lab.org,rfsw2.sb1.cosmos-lab.org }}} === RFSoC Setup === Every time we power cycle the FPGAs, we need to download the firmware (i.e., linux image, rootfs, bistream) using the Xilinx XSCT tool over JTAG. Please note the this process will take around 5-10'. {{{#!shell-session your_username@console:~$ ssh -Y root@srv1-in1 root@srv1-in1:~$ cd mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in1/ root@srv1-in1:~/mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in1$ xsct download_firmware.tcl }}} {{{#!shell-session your_username@console:~$ ssh -Y root@srv1-in2 root@srv1-in2:~$ cd mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in2/ root@srv1-in2:~/mmwsdr/fpga/nonrt-ch1/jtag/sb1_sdr2_in2$ xsct download_firmware.tcl }}} === Network Interface Setup === Using the information available in [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb1 COSMOS WiKi] we need to configure the ethernet interfaces. [[BR]] ||||||||= First corner (1-1) =|| || ''Device'' || ''Control Network'' || ''Data Network 1'' || ''Data Network 2'' || || Support Server || 10.37.1.3 (srv1-in1) || 10.38.1.3 (srv1a-in1) || 10.39.1.3 (srv1b-in1) || || RFSoC Device || 10.37.6.3 (sdr2-in1) || 10.38.6.3 (sdr2-in1a) || 10.39.6.3 (sdr2-in1b) || {{{#!shell-session root@srv1-in1:~# ip link set enp1s0 mtu 9000 up root@srv1-in1:~# ip link set enp3s0 mtu 9000 up root@srv1-in1:~# ip addr add 10.38.1.3/16 dev enp1s0 root@srv1-in1:~# ip addr add 10.39.1.3/16 dev enp3s0 }}} ||||||||= Second corner (20-20) =|| || ''Device'' || ''Control Network'' || ''Data Network 1'' || ''Data Network 2'' || || Support Server || 10.37.1.4 (srv1-in2) || 10.38.1.4 (srv1a-in2) || 10.39.1.4 (srv1b-in2) || || RFSoC Device || 10.37.6.4 (sdr2-in2) || 10.38.6.4 (sdr2-in2a) || 10.39.6.4 (sdr2-in2b) || {{{#!shell-session root@srv1-in2:~# ip link set enp1s0 mtu 9000 up root@srv1-in2:~# ip link set enp3s0 mtu 9000 up root@srv1-in2:~# ip addr add 10.38.1.4/16 dev enp1s0 root@srv1-in2:~# ip addr add 10.39.1.4/16 dev enp3s0 }}} === Software Update === Update the software: {{{#!shell-session root@srv1-in1:~$ cd mmwsdr root@srv1-in1:~/mmwsdr$ git pull }}} {{{#!shell-session root@srv1-in2:~$ cd mmwsdr root@srv1-in2:~/mmwsdr$ git pull }}} === Environment Setup === To control the Sivers using a local connection we need to setup the FTDI drivers for every new ssh connection. {{{#!shell-session root@srv1-in1:~$ source ~/mmwsdr/host/scripts/sivers_ftdi.sh }}} == Demos == In the following subsections, you can find detailed descriptions for each demo. === Basic === - '''onenode.py:''' In this demo we control a single SDR. The script creates an SDR object that controls the Xilinx RFSoC FPGA eval board and Sivers IMA. A user can provide arguments to the script, such as the carrier frequency, the COSMOS node id and the transceiver mode. The script by default starts a local connection at srv1-in1 with a carrier frequency at 60.48 GHz in receive mode. {{{#!shell-session root@srv1-in1:~/mmwsdr/host/demos/basic$ source onenode.py --freq 60.48e9 --node srv1-in1 --mode rx }}} - '''ederarray.py:''' The Python drivers of the Sivers library requires Python 2. One way to remove this dependence is to remotely control the Sivers array with an HTTP server. We can start the HTTP server as follows, {{{#!shell-session root@srv1-in1:~/mmwsdr$ python ./host/mmwsdr/array/ederarray.py -u SN0240 }}} {{{#!shell-session root@srv1-in2:~/mmwsdr$ python ./host/mmwsdr/array/ederarray.py -u SN0243 }}} === Calibration === === Channel Sounder === === Array Pattern === === Beam Tracking Measurements ===