[[Include(WikiToC)]] = IBM 28GHz PAAM Basics = === Description === In this tutorial, we demonstrate the basic use of the [https://wiki.cosmos-lab.org/wiki/Hardware/SubSystems/IBM#IBM28-GHzPhasedArrayBoardForCOSMOS IBM 28 GHz phased array antenna modules (PAAMs)] with USRP N310 software-defined radios (SDRs) in the [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb2 COSMOS Sandboxes (sb1, sb2)]. The following paper describes the integration of the IBM 28 GHz PAAMs (beta-version) with USRP SDRs in the COSMOS testbed. We would appreciate it if you cite this paper when publishing results obtained using the PAAMs deployed in COSMOS. * T. Chen, P. Maddala, P. Skrimponis, J. Kolodziejski, X. Gu, A. Paidimarri, S. Rangan, G. Zussman, and I. Seskar, “Programmable and open-access millimeter-wave radios in the PAWR COSMOS testbed,” in Proc. ACM MobiCom’21 Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization (WiNTECH’21), 2021. * X. Gu, A. Paidimarri, B. Sadhu, C. Baks, S. Lukashov, M. Yeck, Y. Kwark, T. Chen, G. Zussman, I. Seskar, and A. Valdes-Garcia, "Development of a compact 28-GHz software-defined phased array for a city-scale wireless research testbed," in Proc. IEEE International Microwave Symposium (IMS’21), 2021. Author: Tingjun Chen, Duke University (tingjun.chen [at] duke [dot] edu) Last updated: Apr. 10, 2022 === Acknowledgements === This work is collaboration with IBM. We thank Xiaoxiong Gu, Arun Paidimarri, Bodhisatwa Sadhu, and Alberto Valdes-Garcia for their contributions and support. More detailed information about the IBM 28 GHz PAAM can be found in the following references: * B. Sadhu, Y. Tousi, J. Hallin, S. Sahl, S. K. Reynolds, O. Renstrom, K. Sjogren, O. Haapalahti, N. Mazor, B. Bokinge, G. Weibull, H. Bengtsson, A. Carlinger, E. Westesson, J. Thillberg, L. Rexberg, M. Yeck, X. Gu, M. Ferriss, D. Liu, D. Friedman, A. Valdes-Garcia, "A 28-GHz 32-element TRX phased-array IC with concurrent dual-polarized operation and orthogonal phase and gain control for 5G communications," IEEE Journal of Solid-State Circuits, vol. 52, no. 12, pp. 3373-3391, 2017. === Prerequisites === In order to access a COSMOS sandbox, create a reservation and have it approved by the reservation service. Access to the resources is granted after the reservation is confirmed. Please follow the process shown on the COSMOS getting [https://wiki.cosmos-lab.org/wiki/GettingStarted started page] to get started. === Resources Required === In this tutorial we will use the following hardware resources, which are also shown in the figure below. * 2 USRP N310 SDRs ({{{sdr1-in1}}} and {{{sdr1-in2}}} in SB1, {{{sdr1-s1-lg1}}} and {{{sdr1-md1}}} in SB2) * 2 IBM 28GHz PAAMs ({{{rfdev4-in1}}} and {{{rfdev4-in2}}} in SB1, {{{rfdev2-1}}} and {{{rfdev2-2}}} in SB2 ) * 1 Server ({{{srv1-lg1}}}) The current hardware connection in SB1 as shown in this [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb1#RFPathConfigurationsformmWaveDevelopmentPlatforms diagram] * {{{sdr1-in1}}} RF2 TX/RX -- {{{rfdev4-in1}}} all ICs/TX/H, {{{sdr1-in1}}} RF2 RX2 -- {{{rfdev4-in1}}} all ICs/RX/H * {{{sdr1-in1}}} RF3 TX/RX -- {{{rfdev4-in1}}} all ICs/TX/V, {{{sdr1-in1}}} RF3 RX2 -- {{{rfdev4-in1}}} all ICs/RX/V * {{{sdr1-in2}}} RF2 TX/RX -- {{{rfdev4-in2}}} all ICs/TX/H, {{{sdr1-in2}}} RF2 RX2 -- {{{rfdev4-in2}}} all ICs/RX/H * {{{sdr1-in2}}} RF3 TX/RX -- {{{rfdev4-in2}}} all ICs/TX/V, {{{sdr1-in2}}} RF2 RX2 -- {{{rfdev4-in2}}} all ICs/RX/V The current hardware connection in SB2: * {{{sdr1-s1-lg1}}} RF2 TX/RX -- {{{rfdev2-1}}} IC1/TX/H, {{{sdr1-s1-lg1}}} RF2 RX2 -- {{{rfdev2-1}}} IC2/RX/H * {{{sdr1-md1}}} RF2 TX/RX -- {{{rfdev2-2}}} IC1/TX/H, {{{sdr1-md1}}} RF2 RX2 -- {{{rfdev2-2}}} IC2/RX/H || [[Image(mmwavePaamBasicsSetup.png, 600px)]] || Please note that combined IC IO are being used in SB1, and split IC IO in SB2. === Tutorial Setup === Follow the steps below to gain access to the sandbox console and set up nodes with appropriate images. 1. If you don't have one already, sign up for a [https://www.cosmos-lab.org/portal-2/ COSMOS account] 1. [wiki:/GettingStarted#MakeaReservation Create a resource reservation] on COSMOS SB1 or SB2 1. [Documentation/Short/Login Login] into sandbox console ({{{console.sb1.cosmos-lab.org}}} or {{{console.sb2.cosmos-lab.org}}}) with two SSH sessions. 1. Make sure all the nodes and devices used in the experiment are turned off. Use the first command for SB1 and the second command for SB2 (note the difference in the device names) {{{#!shell omf tell -a offh -t sdr1-in1,sdr1-in2,rfdev4-in1,rfdev4-in2,srv1-lg1 }}} {{{#!shell omf tell -a offh -t sdr1-s1-lg1,sdr1-md1,rfdev2-1,rfdev2-2,srv1-lg1 }}} 1. Use the {{{paam28GHz-tutorial-cosmos.ndz}}} node image with Ubuntu 18.04, UHD 3.15, gnuradio 3.8, and a grc example used in this tutorial. Load {{{paam28GHz-tutorial-cosmos.ndz}}} on the server. {{{#!shell omf load -i paam28GHz-tutorial-cosmos.ndz -t srv1-lg1 }}} 1. Turn all the required resources on and check the status of all the resources. Use the following commands for SB1 and SB2. {{{#!shell omf tell -a on -t sdr1-in1,sdr1-in2,rfdev4-in1,rfdev4-in2,srv1-lg1 }}} {{{#!shell omf tell -a on -t sdr1-s1-lg1,sdr1-md1,rfdev2-1,rfdev2-2,srv1-lg1 }}} {{{#!shell omf stat -t all }}} 1. {{{ssh}}} to the server with option -Y for using GUI with gnuradio. {{{#!shell ssh -Y root@srv1-lg1 }}} === Experiment Execution === ==== Find and prepare USRPs ==== Updated eth_config.sh script: {{{#!shell ipaddr_2=$(ip -o -s -4 addr show CTRL | awk '{print $4}' | cut -d/ -f1 | cut -d. --fields 2) ipaddr_3_4=$(ip -o -s -4 addr show CTRL | awk '{print $4}' | cut -d/ -f1 | cut -d. --fields 3,4) echo $ipaddr_3_4 ifconfig DATA1 10.$(($ipaddr_2 + 1)).$ipaddr_3_4 netmask 255.255.0.0 mtu 9000 up ifconfig DATA2 10.$(($ipaddr_2 + 2)).$ipaddr_3_4 netmask 255.255.0.0 mtu 9000 up sysctl -w net.core.rmem_max=62500000 sysctl -w net.core.wmem_max=62500000 }}} * Upon logging into the server, run eth_config.sh script. This sets up the 10G data interfaces eno1, eno2. After running the script, you should see that the data interfaces have the appropriate IP addresses assigned, as per the tables for [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb1#IPAddressAssignment SB1] and [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb2#IPAddressAssignment SB2]. {{{#!td SB1 {{{#!shell root@srv1-lg1:~# ./eth_config.sh root@srv1-lg1:~# ifconfig eno1 eno1: flags=4163 mtu 9000 inet 10.38.1.1 netmask 255.255.0.0 broadcast 10.38.255.255 inet6 fe80::1e34:daff:fe42:c3c prefixlen 64 scopeid 0x20 ether 1c:34:da:42:0c:3c txqueuelen 1000 (Ethernet) RX packets 712 bytes 74814 (74.8 KB) RX errors 0 dropped 595 overruns 0 frame 0 TX packets 42 bytes 9132 (9.1 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 root@srv1-lg1:~# ifconfig eno2 eno2: flags=4163 mtu 9000 inet 10.39.1.1 netmask 255.255.0.0 broadcast 10.39.255.255 inet6 fe80::1e34:daff:fe42:c3d prefixlen 64 scopeid 0x20 ether 1c:34:da:42:0c:3d txqueuelen 1000 (Ethernet) RX packets 643 bytes 51548 (51.5 KB) RX errors 0 dropped 599 overruns 0 frame 0 TX packets 42 bytes 9132 (9.1 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 }}} }}} {{{#!td SB2 {{{#!shell root@srv1-lg1:~# ./eth_config.sh root@srv1-lg1:~# ifconfig eno1 eno1: flags=4163 mtu 9000 inet 10.117.1.1 netmask 255.255.0.0 broadcast 10.117.255.255 inet6 fe80::1e34:daff:fe42:d4c prefixlen 64 scopeid 0x20 ether 1c:34:da:42:0d:4c txqueuelen 1000 (Ethernet) RX packets 254 bytes 20924 (20.9 KB) RX errors 0 dropped 185 overruns 0 frame 0 TX packets 25 bytes 4642 (4.6 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 root@srv1-lg1:~# ifconfig eno2 eno2: flags=4163 mtu 9000 inet 10.118.1.1 netmask 255.255.0.0 broadcast 10.118.255.255 inet6 fe80::1e34:daff:fe42:d4d prefixlen 64 scopeid 0x20 ether 1c:34:da:42:0d:4d txqueuelen 1000 (Ethernet) RX packets 218 bytes 22274 (22.2 KB) RX errors 0 dropped 186 overruns 0 frame 0 TX packets 25 bytes 4642 (4.6 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 }}} }}} * Run {{{und_find_devices}}} to make sure that both USRP N310s can be reached: {{{#!td SB1 {{{#!shell root@srv1-lg1:~# uhd_find_devices [INFO] [UHD] linux; GNU C++ version 7.5.0; Boost_106501; UHD_3.15.0.0-release -------------------------------------------------- -- UHD Device 0 -------------------------------------------------- Device Address: serial: 3176DF5 addr: 10.38.2.1 claimed: False mgmt_addr: 10.37.2.1 mgmt_addr: 10.38.2.1 mgmt_addr: 10.39.2.1 product: n310 type: n3xx -------------------------------------------------- -- UHD Device 1 -------------------------------------------------- Device Address: serial: 3196937 addr: 10.38.3.1 claimed: False mgmt_addr: 10.37.3.1 mgmt_addr: 10.38.3.1 mgmt_addr: 10.39.3.1 product: n310 type: n3xx }}} }}} {{{#!td SB2 {{{#!shell root@srv1-lg1:~# uhd_find_devices [INFO] [UHD] linux; GNU C++ version 7.5.0; Boost_106501; UHD_3.15.0.HEAD-0-gaea0e2de -------------------------------------------------- -- UHD Device 0 -------------------------------------------------- Device Address: serial: 315A35A addr: 10.117.2.1 claimed: False mgmt_addr: 10.116.2.1 mgmt_addr: 10.117.2.1 mgmt_addr: 10.118.2.1 product: n310 type: n3xx -------------------------------------------------- -- UHD Device 1 -------------------------------------------------- Device Address: serial: 3176DF7 addr: 10.117.3.1 claimed: False mgmt_addr: 10.116.3.1 mgmt_addr: 10.117.3.1 mgmt_addr: 10.118.3.1 product: n310 type: n3xx }}} }}} ==== Configure IBM 28GHz PAAM ==== COSMOS uses a RESTful [https://wiki.cosmos-lab.org/wiki/Resources/Services/ArrayMgmt service for IBM PAAM management] The service can be used for * Dynamic array management - where the user connects to the antenna using {{{connect}}} command, dynamically steers the antenna during the experiment using {{{steer}}} command, and disconnects once the experiment is done. * Static array management - where the user can connect, steer and disconnect using a single command, {{{configure}}} Details and examples for the above are provided at the [https://wiki.cosmos-lab.org/wiki/Resources/Services/ArrayMgmt array management page]. For this experiment, we use static array management commands as shown below. First, start PAAM #1 (rfdev4-in1) in RX mode with V-polarization using 16 antenna elements on all the ICs, and configure the RX beamforming direction to be in the broadside (0,0). Check the current consumption on 2v7_0,1,2,3 to make sure all the ICs have been successfully initialized. {{{#!shell root@console:~# curl "http://am1.orbit-lab.org:5054/array_mgmt/configure?dev_name=rfdev4-in1.sb1.cosmos-lab.org&ics=all&num_elements=16&txrx=rx&pol=v&theta=0&phi=0" }}} {{{#!shell }}} Similarly, start PAAM #2 (rfdev4-in2) in TX mode with V-polarization using 16 antenna elements on all the ICs, and configure the TX beamforming direction to be in the broadside (0,0). Check the current consumption to make sure the ICs have been successfully initialized. {{{#!shell root@console:~# curl "http://am1.orbit-lab.org:5054/array_mgmt/configure?dev_name=rfdev4-in2.sb1.cosmos-lab.org&ics=all&num_elements=16&txrx=tx&pol=v&theta=0&phi=0" }}} {{{#!shell }}} If you are on SB2, the commands are:\\ PAAM #1 (rfdev2-1.sb2.cosmos-lab.org) {{{#!shell root@console:~# curl "http://am1.orbit-lab.org:5054/array_mgmt/configure?dev_name=rfdev2-1.sb2.cosmos-lab.org&ics=1&num_elements=4&txrx=tx&pol=h&theta=0&phi=0" }}} PAAM #2 (rfdev2-2.sb2.cosmos-lab.org) {{{#!shell root@console:~# curl "http://am1.orbit-lab.org:5054/array_mgmt/configure?dev_name=rfdev2-2.sb2.cosmos-lab.org&ics=2&num_elements=4&txrx=rx&pol=h&theta=0&phi=0" }}} ==== Run the experiment ==== * On srv1-lg1, start gnuradio companion and open the example experiment that establishes a single tone transmission. Use the file that is appropriate for the domain (SB1 or SB2). {{{#!shell root@srv1-lg1:~# gnuradio-companion example_paam_tone_sb1.grc }}} * For running the experiment on SB1, configure the USRP sink (TX) with sdr1-in2 ("mgmt_addr=10.37.6.2,addr=10.39.6.2") and the USRP source (RX) with sdr1-in1("mgmt_addr=10.37.6.1,addr=10.39.6.1"). For SB2, configure the USRP sink (TX) with sdr1-s1-lg1 ("mgmt_addr=10.116.2.1,addr=10.117.2.1") and the USRP source (RX) with sdr1-md1 ("mgmt_addr=10.116.3.1,addr=10.117.3.1"). Set the carrier frequency to 3GHz (3e9) and the subdev to be "B:1" (RF3) (For SB2, use "B:0", RF2) on both TX and RX. In this example flowgraph, the sampling rate and the tone frequency are set to be 2.5MHz (2.5e6) and 1MHz (1e6), respectively. || [[Image(mmwavePaamBasicsFlowgraph.png, 600px)]] || ==== Observe the results ==== Figure below shows the frequency response of the received tone at 1MHz offset. || [[Image(mmwavePaamBasicsFreq_n_4.png, 600px)]] || ==== Finish the experiments ==== When the experiments are completed, make sure to clean up and turn off the PAAMs. For SB1:\\ {{{#!shell root@console:~# curl "http://am1.orbit-lab.org:5054/array_mgmt/cleanup?dev_name=rfdev4-in1.sb1.cosmos-lab.org" root@console:~# omf tell -a offh -t rfdev4-in1.sb1.cosmos-lab.org root@console:~# curl "http://am1.orbit-lab.org:5054/array_mgmt/cleanup?dev_name=rfdev4-in2.sb1.cosmos-lab.org" root@console:~# omf tell -a offh -t rfdev4-in2.sb1.cosmos-lab.org }}} \\ For SB2:\\ {{{#!shell root@console:~# curl "http://am1.orbit-lab.org:5054/array_mgmt/cleanup?dev_name=rfdev2-1.sb2.cosmos-lab.org" root@console:~# omf tell -a offh -t rfdev2-1.sb2.cosmos-lab.org root@console:~# curl "http://am1.orbit-lab.org:5054/array_mgmt/cleanup?dev_name=rfdev2-2.sb2.cosmos-lab.org" root@console:~# omf tell -a offh -t rfdev2-2.sb2.cosmos-lab.org }}}