[[Include(WikiToC)]] = Savannah: Efficient mmWave Baseband Processing with Minimal and Heterogeneous Resources = === Description === In this tutorial, we demonstrate Savannah, a framework for efficient mmWave baseband processing with minimal and heterogeneous resources. * The codebase, README and installation instructions for Savannah software can be found in attachments called {{{Savannah.zip}}} under [https://docs.renew-wireless.org/license/ RENEW license]. * The COSMOS team contributes to: * Adding the UHD support for the Agora software under the [https://github.com/EttusResearch/uhd/blob/master/LICENSE.md UHD license]; * Updating DSP pipeline (Equalization; Demodulation); * Integrating ACC100 BBDEV acceleration for Decoding. We thank the RENEW team for their support and help throughout the process. The following papers describes the integration of the IBM 28 GHz PAAMs with Savannah in the COSMOS testbed. We would appreciate it if you cite this paper when publishing results obtained using the PAAMs deployed in COSMOS. * Z. Qi, Z. Gao, C. Tung, and T. Chen, "Programmable Millimeter-Wave MIMO Radios with Real-Time Baseband Processing". in Proc. ACM MobiCom'23 Workshop on Wireless Network Testbeds, Experimental evaluation & Characterization (WiNTECH '23), 2023 * Z. Qi*, C. Tung*, A. Kalia, and T. Chen, "Savannah: Efficient mmWave Baseband Processing with Minimal and Heterogeneous Resources". in Proc. 30th Annual International Conference on Mobile Computing and Networking (MobiCom'24), 2024 Authors: \\ Zhenzhou (Tom) Qi, Duke University \\ Chung-Hsuan Tung, Duke University \\ Zhihui Gao, Duke University \\ Tingjun Chen, Duke University Last updated: Sep. 11, 2024 === Prerequisites === In order to access COSMOS-SB2, create a reservation in COSMOS testbed and have it approved by the reservation service. Access to the resources is granted after the reservation is confirmed. Please follow the process shown on the COSMOS getting [https://wiki.cosmos-lab.org/wiki/GettingStarted started page] to get started. === Resources Required === * 2 USRP N310 SDRs ({{{sdr1-s1-lg1}}} and {{{sdr1-md1}}} in SB2) * 2 IBM 28GHz PAAMs ({{{rfdev2-1}}} and {{{rfdev2-2}}} in SB2 ) * 1 Server ({{{srv1-lg1}}}) The current hardware connection in SB2 as shown below * {{{sdr1-s1-lg1}}} RF2 TX/RX -- {{{rfdev2-1}}} IC0/TX/H, {{{sdr1-s1-lg1}}} RF2 RX2 -- {{{rfdev2-1}}} IC1/RX/H * {{{sdr1-s1-lg1}}} RF3 TX/RX -- {{{rfdev2-1}}} IC0/TX/V, {{{sdr1-s1-lg1}}} RF3 RX2 -- {{{rfdev2-1}}} IC1/RX/V * {{{sdr1-md1}}} RF2 TX/RX -- {{{rfdev2-2}}} IC0/TX/H, {{{sdr1-md1}}} RF2 RX2 -- {{{rfdev2-2}}} IC1/RX/H * {{{sdr1-md1}}} RF3 TX/RX -- {{{rfdev2-2}}} IC0/TX/V, {{{sdr1-md1}}} RF2 RX2 -- {{{rfdev2-2}}} IC1/RX/V || [[Image(cosmos_paam_202406.png, 400px)]] || The required software components used in this demo are already loaded to the {{{.ndz}}} node image, the node image includes: * Ubuntu 20.04, UHD 4.1.0 * PAAM Control to initialize the PAAM boards to integrate with USRPs. * PAAM CFO calibration. * Savannah: a framework for efficient mmWave baseband processing with minimal and heterogeneous resources. Please refer to {{{Savannah.zip}}} under attachments for more information. === Tutorial Setup === Follow the steps below to gain access to the sandbox console and set up nodes with appropriate images. 1. If you don't have one already, sign up for a [https://www.cosmos-lab.org/portal-2/ COSMOS account] 1. [wiki:/GettingStarted#MakeaReservation Create a resource reservation] on COSMOS SB2 1. [Documentation/Short/Login Login] into sandbox console {{{console.sb2.cosmos-lab.org}}}) with **four** SSH sessions. 1. In **terminal 1**, make sure all the nodes and devices used in the experiment are turned off. Use the following command for SB2 {{{#!shell omf tell -a offh -t sdr1-s1-lg1,sdr1-md1,rfdev2-1,rfdev2-2,srv1-lg1 }}} 1. Load {{{To_be_updated}}} on the server. {{{#!shell omf load -i To_be_updated -t srv1-lg1 -r 0 }}} 1. Turn all the required resources on and check the status of all the resources. Use the following commands for SB2. {{{#!shell omf tell -a on -t sdr1-s1-lg1,sdr1-md1,rfdev2-1,rfdev2-2,srv1-lg1 }}} {{{#!shell omf stat -t all }}} 1. {{{ssh}}} to the server with option -Y for using GUI for our live demo. {{{#!shell ssh -Y root@srv1-lg1 }}} 1. In **all the ssh sessions** for {{{srv1-lg1}}}, use the following to source the correct environment. {{{#!shell source /opt/intel/oneapi/setvars.sh --force --config="/opt/intel/oneapi/renew-config.txt" }}} ==== Find and prepare USRPs ==== * Upon logging into the server, set up the 10G data interfaces DATA1, DATA2 in **terminal 1**. {{{#!shell ifconfig DATA1 10.117.1.1 netmask 255.255.0.0 mtu 9000 up ifconfig DATA2 10.118.1.1 netmask 255.255.0.0 mtu 9000 up }}} {{{#!shell sudo sysctl -w net.core.rmem_max=536870912 sudo sysctl -w net.core.wmem_max=536870912 }}} After running the above commands, you should see that the data interfaces have the appropriate IP addresses assigned. {{{#!shell root@srv1-lg1:~# ifconfig DATA1 DATA1: flags=4163 mtu 1500 inet6 fe80::1e34:daff:fe42:d4c prefixlen 64 scopeid 0x20 ether 1c:34:da:42:0d:4c txqueuelen 1000 (Ethernet) RX packets 21092 bytes 1881634 (1.8 MB) RX errors 0 dropped 19183 overruns 0 frame 0 TX packets 686 bytes 204975 (204.9 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 root@srv1-lg1:~# ifconfig DATA2 DATA2: flags=4163 mtu 1500 inet6 fe80::1e34:daff:fe42:d4d prefixlen 64 scopeid 0x20 ether 1c:34:da:42:0d:4d txqueuelen 1000 (Ethernet) RX packets 21091 bytes 1881530 (1.8 MB) RX errors 0 dropped 19184 overruns 0 frame 0 TX packets 690 bytes 226549 (226.5 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 }}} * Run {{{uhd_find_devices}}} to make sure that both USRP N310s can be reached: {{{#!shell [INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100; UHD_4.1.0.HEAD-0-g25d617ca -------------------------------------------------- -- UHD Device 3 -------------------------------------------------- Device Address: serial: 315A35A addr: 10.117.2.1 claimed: False fpga: XG mgmt_addr: 10.116.2.1 mgmt_addr: 10.117.2.1 product: n310 type: n3xx -------------------------------------------------- -- UHD Device 4 -------------------------------------------------- Device Address: serial: 3176DF7 addr: 10.118.3.1 claimed: False fpga: XG mgmt_addr: 10.116.3.1 mgmt_addr: 10.118.3.1 product: n310 type: n3xx }}} ==== Configure IBM 28GHz PAAM ==== COSMOS uses the IBM 28GHz PAAM API to configure and initialize the PAAM boards. Under the console in **terminal 2**, please run the following 4 commands: {{{#!shell @console:~$ curl "http://am1.orbit-lab.org:5054/array_mgmt/configure?dev_name=rfdev2-1.sb2.cosmos-lab.org&ics=0&num_elements=16&txrx=tx&pol=h&theta=0&phi=0" @console:~$ curl "http://am1.orbit-lab.org:5054/array_mgmt/configure?dev_name=rfdev2-1.sb2.cosmos-lab.org&ics=0&num_elements=16&txrx=tx&pol=v&theta=0&phi=0" @console:~$ curl "http://am1.orbit-lab.org:5054/array_mgmt/configure?dev_name=rfdev2-2.sb2.cosmos-lab.org&ics=1&num_elements=16&txrx=rx&pol=h&theta=0&phi=0" @console:~$ curl "http://am1.orbit-lab.org:5054/array_mgmt/configure?dev_name=rfdev2-2.sb2.cosmos-lab.org&ics=1&num_elements=16&txrx=rx&pol=v&theta=0&phi=0" }}} An example message will print out to after each command to indicate that the PAAM board is being successfully configured: {{{#!shell }}} === Experiment Execution - DPDK BBDev Tests === ==== ACC100 Unittests ==== **Notice**: COSMOS testbed currently does not have ACC100 card installed in the servers, but we provide unit test code and testcases for your reference if you have installed the card on your own system. In {{{Savannah.zip}}}, you will find a README file called {{{BBDev_testcase.md}}}. In this file, you will find detailed instructions on the ACC100 system requirements, initialization, configuration, how to run default validation test and how to build your own testcase. We have provided example testcase w.r.t different MCS in the {{{mcs_enc_dec_pair}}} folder. Inside this folder, you will find encoding and decoding testcases for different MCSs specifically for 5G mmWave FR2 Numeorology-3 100MHz; A program called `test_bbdev_perf.c` based on DPDK's bbdev program; and expected encoding and decoding test results. Please feel free to contact the author if you have any questions related to how to use ACC100 card. === Experiment Execution === ==== General information ==== **Notice** For every terminal you opened up, please run: {{{#!shell source /opt/intel/oneapi/setvars.sh --force --config="/opt/intel/oneapi/renew-config.txt" }}} We have already built and installed all the necessary packages for running Savannah, please direct to the {{{~/Savannah/agora-mmwave/}}} folder. There are different modes we are able to run and the corresponding configurations are shown in the following. To configure to different modes, please direct to `/build`, and use `cmake .. -D=