[[Include(WikiToC)]] = IBM 28GHz PAAM Basics = === Description === In this tutorial, we demonstrate the basic use of the [https://wiki.cosmos-lab.org/wiki/Hardware/SubSystems/IBM#IBM28-GHzPhasedArrayBoardForCOSMOS IBM 28 GHz phased array antenna modules (PAAMs)] with USRP N310 software-defined radios (SDRs) in the [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb2 COSMOS Sandboxes (sb1, sb2)]. The following paper describes the integration of the IBM 28 GHz PAAMs (beta-version) with USRP SDRs in the COSMOS testbed. We would appreciate it if you cite this paper when publishing results obtained using the PAAMs deployed in COSMOS. * X. Gu, A. Paidimarri, B. Sadhu, C. Baks, S. Lukashov, M. Yeck, Y. Kwark, T. Chen, G. Zussman, I. Seskar, and A. Valdes-Garcia, "Development of a compact 28-GHz software-defined phased array for a city-scale wireless research testbed," in Proc. IEEE International Microwave Symposium (IMS’21) (to appear), 2021. Author: Tingjun Chen, Duke University / Columbia University (tingjun.chen [at] duke [dot] edu) Last updated: Apr. 11, 2021 === Acknowledgements === This work is collaboration with IBM. We thank Xiaoxiong Gu, Arun Paidimarri, Bodhisatwa Sadhu, and Alberto Valdes-Garcia for their contributions and support. More detailed information about the IBM 28 GHz PAAM can be found in the following references: * B. Sadhu, Y. Tousi, J. Hallin, S. Sahl, S. K. Reynolds, O. Renstrom, K. Sjogren, O. Haapalahti, N. Mazor, B. Bokinge, G. Weibull, H. Bengtsson, A. Carlinger, E. Westesson, J. Thillberg, L. Rexberg, M. Yeck, X. Gu, M. Ferriss, D. Liu, D. Friedman, A. Valdes-Garcia, "A 28-GHz 32-element TRX phased-array IC with concurrent dual-polarized operation and orthogonal phase and gain control for 5G communications," IEEE Journal of Solid-State Circuits, vol. 52, no. 12, pp. 3373-3391, 2017. === Prerequisites === In order to access a COSMOS sandbox, create a reservation and have it approved by the reservation service. Access to the resources is granted after the reservation is confirmed. Please follow the process shown on the COSMOS getting [https://wiki.cosmos-lab.org/wiki/GettingStarted started page] to get started. === Resources Required === In this tutorial we will use the following hardware resources in sb2, which are also shown in the figure below. * 2 USRP N310 SDRs ({{{sdr1-in1}}} and {{{sdr1-in2}}} in SB1, {{{sdr1-s1-lg1}}} and {{{sdr1-md1}}} in SB2) * 2 IBM 28GHz PAAMs ({{{rfdev4-in1}}} and {{{rfdev4-in2}}} in SB1, {{{rfdev2-1}}} and {{{rfdev2-2}}} in SB2 ) * 1 Server ({{{srv1-lg1}}}) The current hardware connection in SB1 as shown in this [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb1#RFPathConfigurationsformmWaveDevelopmentPlatforms diagram] * {{{sdr1-in1}}} RF2 TX/RX -- {{{rfdev4-in1}}} IC0/TX/H, {{{sdr1-in1}}} RF2 RX2 -- {{{rfdev4-in1}}} IC0/RX/H * {{{sdr1-in1}}} RF3 TX/RX -- {{{rfdev4-in1}}} IC0/TX/V, {{{sdr1-in1}}} RF3 RX2 -- {{{rfdev4-in1}}} IC0/RX/V * {{{sdr1-in2}}} RF2 TX/RX -- {{{rfdev4-in2}}} IC0/TX/H, {{{sdr1-in2}}} RF2 RX2 -- {{{rfdev4-in2}}} IC0/RX/H * {{{sdr1-in2}}} RF3 TX/RX -- {{{rfdev4-in2}}} IC0/TX/V, {{{sdr1-in2}}} RF2 RX2 -- {{{rfdev4-in2}}} IC0/RX/V The current hardware connection in SB2: * {{{sdr1-s1-lg1}}} RF2 TX/RX -- {{{rfdev2-1}}} IC1/TX/H, {{{sdr1-s1-lg1}}} RF2 RX2 -- {{{rfdev2-1}}} IC2/RX/H * {{{sdr1-md1}}} RF2 TX/RX -- {{{rfdev2-2}}} IC1/TX/H, {{{sdr1-md1}}} RF2 RX2 -- {{{rfdev2-2}}} IC2/RX/H || [[Image(mmwavePaamBasicsSetup.png, 600px)]] || === Tutorial Setup === Follow the steps below to gain access to the sandbox console and set up nodes with appropriate images. 1. If you don't have one already, sign up for a [https://www.cosmos-lab.org/portal-2/ COSMOS account] 1. [wiki:/GettingStarted#MakeaReservation Create a resource reservation] on COSMOS SB1 or SB2 1. [Documentation/Short/Login Login] into sandbox console ({{{console.sb1.cosmos-lab.org}}} or {{{console.sb2.cosmos-lab.org}}}) with two SSH sessions. 1. Make sure all the nodes and devices used in the experiment are turned off. Use the first command for SB1 and the second command for SB2 (note the difference in the device names) {{{#!shell omf tell -a offh -t sdr1-in1,sdr1-in2,rfdev4-in1,rfdev4-in2,srv1-lg1 }}} {{{#!shell omf tell -a offh -t sdr1-s1-lg1,sdr1-md1,rfdev2-1,rfdev2-2,srv1-lg1 }}} 1. Use the {{{paam28GHz-tutorial-cosmos.ndz}}} node image with Ubuntu 18.04, UHD 3.15, gnuradio 3.8, and a grc example used in this tutorial. Load {{{paam28GHz-tutorial-cosmos.ndz}}} on the server. {{{#!shell omf load -i paam28GHz-tutorial-cosmos.ndz -t srv1-lg1 }}} 1. Turn all the required resources on and check the status of all resources {{{#!shell omf tell -a on -t sdr1-in1,sdr1-in2,rfdev4-in1,rfdev4-in2,srv1-lg1 }}} {{{#!shell omf tell -a on -t sdr1-s1-lg1,sdr1-md1,rfdev2-1,rfdev2-2,srv1-lg1 }}} {{{#!shell omf stat -t all }}} 1. {{{ssh}}} to the server with option -Y for using GUI with gnuradio. {{{#!shell ssh -Y root@srv1-lg1, }}} === Experiment Execution === ==== Find and prepare USRPs ==== * Upon logging into the server, run eth_config.sh script. This sets up the 10G data interfaces DATA1, DATA2. After running the script, you should see that the data interfaces have the appropriate IP addresses assigned, as per the tables for [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb1#IPAddressAssignment SB1] and [https://wiki.cosmos-lab.org/wiki/Architecture/Domains/cosmos_sb2#IPAddressAssignment SB2]. * The IP addresses for SFP+0 (10G) on {{{sdr1-s1-lg1}}} and {{{sdr1-md1}}} were hard-coded to {{{10.117.2.1}}} and {{{10.117.3.1}}}, respectively. To access them from {{{srv1-lg1}}}, configure the network interface {{{eno1}}} as follows: {{{#!shell root@srv1-lg1:~# ifconfig eno1 10.117.1.1 netmask 255.255.0.0 mtu 9000 up root@srv1-lg1:~# sudo sysctl -w net.core.wmem_max=62500000 net.core.wmem_max = 62500000 root@srv1-lg1:~# sudo sysctl -w net.core.rmem_max=62500000 net.core.rmem_max = 62500000 root@srv1-lg1:~# ifconfig eno1: flags=4163 mtu 9000 inet 10.117.1.1 netmask 255.255.0.0 broadcast 10.117.255.255 inet6 fe80::1e34:daff:fe42:d4c prefixlen 64 scopeid 0x20 ether 1c:34:da:42:0d:4c txqueuelen 1000 (Ethernet) RX packets 15305870 bytes 21326136765 (21.3 GB) RX errors 0 dropped 46450 overruns 0 frame 0 TX packets 16288829 bytes 21257264625 (21.2 GB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 }}} * Run {{{und_find_devices}}} to make sure that both USRP N310s can be reached: {{{#!shell root@srv1-lg1:~# uhd_find_devices [INFO] [UHD] linux; GNU C++ version 7.5.0; Boost_106501; UHD_3.15.0.HEAD-0-gaea0e2de -------------------------------------------------- -- UHD Device 0 -------------------------------------------------- Device Address: serial: 315A35A addr: 10.117.2.1 claimed: True mgmt_addr: 10.116.2.1 mgmt_addr: 10.117.2.1 mgmt_addr: 10.118.2.1 product: n310 type: n3xx -------------------------------------------------- -- UHD Device 2 -------------------------------------------------- Device Address: serial: 3176DF7 addr: 10.117.3.1 claimed: True mgmt_addr: 10.116.3.1 mgmt_addr: 10.117.3.1 mgmt_addr: 10.118.3.1 product: n310 type: n3xx }}} ==== Configure IBM 28GHz PAAM ==== COSMOS uses a RESTful [https://wiki.cosmos-lab.org/wiki/Resources/Services/ArrayMgmt service for IBM PAAM management] ==== Run the experiment ==== * [Terminal 1] In the first terminal, start gnuradio companion and open the example experiment that established a single tone transmission: {{{#!shell root@srv1-lg1:~# gnuradio-companion example_paam_tone.grc }}} * [Terminal 1] In gnuradio-companion, start gnuradio-companion, configure the USRP sink (TX) with sdr1-s1-lg1 ("mgmt_addr=10.116.2.1,addr=10.117.2.1") and the USRP source (RX) with sdr1-md1 ("mgmt_addr=10.116.3.1,addr=10.117.3.1"). Set the carrier frequency to 3GHz (3e9) and the subdev to be "B:0" (RF2) on both TX and RX. In this example flowgraph, the sampling rate and the tone frequency are set to be 2.5MHz (2.5e6) and 1MHz (1e6), respectively. || [[Image(mmwavePaamBasicsFlowgraph.png, 600px)]] || * [Terminal 2] In the second terminal, change directory to paam_api/examples/ which contains the example API scripts. {{{#!shell root@srv1-lg1:~# cd paam_api/examples/ }}} * [Terminal 2] First, start PAAM #1 (rfdev2-1) in TX mode with H-polarization using 4 antenna elements on IC 1, and configure the TX beamforming direction to be in the broadside (0,0). Check the current consumption on 2v7_1 and make sure IC1 has been successfully initialized (e.g., 2v7_1 has a current consumption of 1.026A in the example below). {{{#!shell root@srv1-lg1:~/paam_api/examples# python3 setup_betaboard_v1.2.py -c ethernet -a rfdev2-1 --ic 1 -n 4 --txrx tx --pol h --dir 0 0 TRX mode selection: tx IC(s) used for experiment: [1] Number of Elements per IC: 4 Polarization: h Beam direction: (0, 0) IP address of TX: rfdev2-1 Opened port to FPGA Logged in to Petalinux Started command parser on Zynq sdpar_prog.py /version: Version is PAWR_v1.2.0 Using baseline FPGA control IP Reset the Phased Array Initialization of IC 0 was successful Initialization of IC 1 was successful Initialization of IC 2 was successful Initialization of IC 3 was successful elapsed time for init: 5.458436489105225 Time for PAWR Board utilities configuration: 0.07273483276367188 elapsed time for enable: 0.00990605354309082 elapsed time for steer beam: 0.0026137828826904297 PAAM ID: 0x 2 LO switch: PLL IF Switches TX_H: 0xF IF Switches TX_V: 0xF IF Switches RX_H: 0xF IF Switches RX_V: 0xF Index Name ADC Volt. Curr. 0 1v2 21 0.051 0.026 1 1v5 154 0.376 0.753 2 1v8 3 0.007 0.004 3 2v7_0 16 0.039 0.078 4 2v7_1 210 0.513 1.026 5 2v7_2 44 0.108 0.215 6 2v7_3 44 0.108 0.215 7 3v3_pll 367 0.897 0.448 8 5v_uzed 249 0.609 0.609 9 12v 124 0.303 1.010 10 0V 0 0.000 x 11 1V8 735 1.796 x Closed port to Zynq Good luck with the experiment!!! }}} * [Terminal 2] Similarly, start PAAM #2 (rfdev2-2) in RX mode with H-polarization using 4 antenna elements on IC 2, and configure the RX beamforming direction to be in the broadside (0,0). Check the current consumption on 2v7_2 and make sure IC2 has been successfully initialized (e.g., 2v7_2 has a current consumption of 0.821A in the example below). {{{#!shell root@srv1-lg1:~/paam_api/examples# python3 setup_betaboard_v1.2.py -c ethernet -a rfdev2-2 --ic 2 -n 4 --txrx rx --pol h --dir 0 0 TRX mode selection: rx IC(s) used for experiment: [2] Number of Elements per IC: 4 Polarization: h Beam direction: (0, 0) IP address of TX: rfdev2-2 Opened port to FPGA Logged in to Petalinux Started command parser on Zynq sdpar_prog.py /version: Version is PAWR_v1.2.0 Using baseline FPGA control IP Reset the Phased Array Initialization of IC 0 was successful Initialization of IC 1 was successful Initialization of IC 2 was successful Initialization of IC 3 was successful elapsed time for init: 5.453927516937256 Time for PAWR Board utilities configuration: 0.07044315338134766 elapsed time for enable: 0.008872270584106445 elapsed time for steer beam: 0.002682209014892578 PAAM ID: 0x 4 LO switch: PLL IF Switches TX_H: 0xF IF Switches TX_V: 0xF IF Switches RX_H: 0xF IF Switches RX_V: 0xF Index Name ADC Volt. Curr. 0 1v2 127 0.310 0.155 1 1v5 134 0.327 0.655 2 1v8 18 0.044 0.022 3 2v7_0 42 0.103 0.205 4 2v7_1 51 0.125 0.249 5 2v7_2 168 0.411 0.821 6 2v7_3 13 0.032 0.064 7 3v3_pll 34 0.083 0.042 8 5v_uzed 249 0.609 0.609 9 12v 129 0.315 1.051 10 0V 0 0.000 x 11 1V8 735 1.796 x Closed port to Zynq Good luck with the experiment!!! }}} ==== Observe the results ==== The left figure shows the frequency response of the received tone at 1MHz offset. Experiment can for example increase the link SNR by using more antenna elements (i.e., 8 for TX and 8 for RX) via the PAAM API, and the corresponding results are shown in the right figure. {{{#!shell root@srv1-lg1:~/paam_api/examples# python3 setup_betaboard_v1.2.py -c ethernet -a rfdev2-1 --ic 1 -n 8 --txrx tx --pol h --dir 0 0 root@srv1-lg1:~/paam_api/examples# python3 setup_betaboard_v1.2.py -c ethernet -a rfdev2-2 --ic 2 -n 8 --txrx rx --pol h --dir 0 0 }}} || [[Image(mmwavePaamBasicsFreq_n_4.png, 600px)]] || [[Image(mmwavePaamBasicsFreq_n_8.png, 600px)]] || ==== Finish the experiments ==== When the experiments are completed, make sure to turn off both PAAMs and see that the IC current consumption drops to the minimal values: {{{#!shell root@srv1-lg1:~/paam_api/examples# python3 reset_betaboard_v1.2.py -c ethernet -a rfdev2-1 ... Index Name ADC Volt. Curr. 0 1v2 30 0.073 0.037 1 1v5 45 0.110 0.220 2 1v8 1 0.002 0.001 3 2v7_0 38 0.093 0.186 4 2v7_1 45 0.110 0.220 5 2v7_2 47 0.115 0.230 6 2v7_3 37 0.090 0.181 7 3v3_pll 57 0.139 0.070 8 5v_uzed 264 0.645 0.645 9 12v 106 0.259 0.863 10 0V 0 0.000 x 11 1V8 735 1.796 x Closed port to Zynq PAAM board shutdown...!!! root@srv1-lg1:~/paam_api/examples# python3 reset_betaboard_v1.2.py -c ethernet -a rfdev2-2 ... Index Name ADC Volt. Curr. 0 1v2 109 0.266 0.133 1 1v5 140 0.342 0.684 2 1v8 23 0.056 0.028 3 2v7_0 9 0.022 0.044 4 2v7_1 11 0.027 0.054 5 2v7_2 51 0.125 0.249 6 2v7_3 10 0.024 0.049 7 3v3_pll 115 0.281 0.141 8 5v_uzed 223 0.545 0.545 9 12v 99 0.242 0.806 10 0V 0 0.000 x 11 1V8 735 1.796 x Closed port to Zynq PAAM board shutdown...!!! }}}