[[Include(WikiToC)]] == Measurement tools == This tutorial will demonstrate data collection and storage method using the OML interface with an SDR radio. === Description === This tutorial utilizes an existing application provided by the UHD driver. The application used is ''rx_multi_sample'' which is a c++ application that configures and reads samples from multiple the USRPs. A c++ based OML-wrapper has been added to this application to store FFTed blocks of samples to an OML database server. === Prerequisites === In order to access the test bed, create a reservation and have it approved by the reservation service. Access to the resources are granted after the reservation is confirmed. Please follow the process shown on [wiki:CosmosOverview/Workflow the COSMOS work flow page] to get started. === Set up === 1. Sign up for a [https://cosmos-lab.org/portal-2/ COSMOS account] 2. [UserGuide/CreateRes Create a resource reservation on sandbox 1] 3. [UserGuide/RemoteAccess/Console Login to your reserved domain.] 4. Load baseline-mobi.ndz on your resource. [UserGuide/OmfQuickStart - this is done via OMF commands.] * After loading the image, if utilizing a network based SDR radio (ie. ethernet with IP address) then follow the instructions [https://wiki.cosmos-lab.org/wiki/tutorials/n310_usage here to configure the network interface and detect the radio.] * If the SDR radio being used is PCIe based radio (ie. Krypton) then [https://wiki.cosmos-lab.org/wiki/tutorials/krypton_usage use the instructions here to load & start the PCIe driver.] === Description of ''rx_multi_receive'' application and how to use === Navigate to the ~/RX_MULTI_RECEIVE directory - this contains the files that build the ''rx_multi_receive'' application. This is a c++ application that can be run on the experiment server to collect a set of continuous samples. Use the --help option to display all the variables ''rx_multi_receive'' accepts. {{{#!shell-session root@node:~/RX_MULTI_RECEIVE# ./rx_multi_receive --help UHD RX Multi Receive Allowed options: --help help message --args arg single uhd device address args --secs arg (=1.5) number of seconds in the future to receive --nsamps arg (=10000) total number of samples to receive --freq arg (=900000000) RF center frequency in Hz for all channels --rate arg (=6250000) rate of incoming samples for all channels --gain arg (=0) gain for the RF chain for all channels --ant arg rx antenna selection for all channels --prefix arg enables file output with filename prefix --addr arg udp address: 10.10.0.10 --port arg (=1337) udp port: 1337 --sync arg (=now) synchronization method: now, pps, mimo --subdev arg subdev spec (homogeneous across motherboards) --dilv specify to disable inner-loop verbose --int-n tune USRP with integer-N tuning --channels arg (=0) which channel(s) to use (specify "0", "1", "0,1", etc) --oml-id arg OML sender ID - pass the hostname here --oml-domain arg (=omlcli) file name for OML database --oml-collect arg (=oml:3003) Storage options: network: local text file: }}} To use ''rx_multi_receive'' standalone on an experiment server we can using the following options {{{#!shell-session root@node:~/RX_MULTI_RECEIVE# ./rx_multi_receive --args="resource=rio0,type=x300" --nsamps 1024000 --freq 2440e6 --rate 100e6 --gain 15 --dilv }}} || --args="addr0=10.10.23.5" || specifies the SDR's IP address. || || --args="resource=rio0,type=x300" || if using Krypton radio. || || --freq 2440e6 || sets the center freq || || --rate 20e6 || sets sampling rate || || --gain 3 || sets receive path gain || || --nsamps 1024000 || number of samples to collect || Executing the application with only the above options will only configure the USRP's radio parameters and read in the specified number of samples. However the samples will be lost once the application exits. You should see output similar to the following. {{{#!shell-session root@node:~/RX_MULTI_RECEIVE# Creating the usrp device with: type=x300... [INFO] [UHD] linux; GNU C++ version 7.4.0; Boost_106501; UHD_3.14.0.HEAD-0-g6875d061 [INFO] [X300] X300 initialization sequence... [INFO] [X300] Connecting to niusrpriorpc at localhost:5444... [INFO] [X300] Using LVBITX bitfile /usr/local/share/uhd/images/usrp_x310_fpga_HG.lvbitx... [INFO] [X300] Radio 1x clock: 200 MHz [INFO] [GPS] Found an internal GPSDO: LC_XO, Firmware Rev 0.929a [INFO] [0/DmaFIFO_0] Initializing block control (NOC ID: 0xF1F0D00000000000) [INFO] [0/DmaFIFO_0] BIST passed (Throughput: 1296 MB/s) [INFO] [0/DmaFIFO_0] BIST passed (Throughput: 1302 MB/s) [INFO] [0/Radio_0] Initializing block control (NOC ID: 0x12AD100000000001) [INFO] [0/Radio_1] Initializing block control (NOC ID: 0x12AD100000000001) [INFO] [0/DDC_0] Initializing block control (NOC ID: 0xDDC0000000000000) [INFO] [0/DDC_1] Initializing block control (NOC ID: 0xDDC0000000000000) [INFO] [0/DUC_0] Initializing block control (NOC ID: 0xD0C0000000000000) [INFO] [0/DUC_1] Initializing block control (NOC ID: 0xD0C0000000000000) Using Device: Single USRP: Device: X-Series Device Mboard 0: X310 RX Channel: 0 RX DSP: 0 RX Dboard: A RX Subdev: UBX RX RX Channel: 1 RX DSP: 0 RX Dboard: B RX Subdev: UBX RX TX Channel: 0 TX DSP: 0 TX Dboard: A TX Subdev: UBX TX TX Channel: 1 TX DSP: 0 TX Dboard: B TX Subdev: UBX TX Number mboards = 1 Number of rx channels = 2 Setting RX Rate: 100.000000 Msps... Actual RX Rate: 100.000000 Msps... Setting RX Freq: 2440.000000 MHz... Actual RX Freq: 2440.000000 MHz... Setting RX Gain: 15.000000 dB... Actual RX Gain: 15.000000 dB... Setting device timestamp to 0... channel_nums.size() = 1 Begin streaming 1024000 samples, 1.500000 seconds in the future... samps_per_buff = 1024 num_buff_ptrs = 1000 mdbp_idx = 1000 Done! }}} To save the samples for post processing, specify the following OML arguments || --oml-id `hostname` || this identifies the source of the data being store || || --oml-domain rx_samples || file name for the OML database || || --oml-collect oml:3003 || network storage located on the console || The full command to execute would be {{{#!shell-session root@node:~/RX_MULTI_RECEIVE# ./rx_multi_receive --args="resource=rio0,type=x300" --nsamps 1024000 --freq 2440e6 --rate 20e6 --gain 25 --dilv --oml-id `hostname` --oml-collect "oml:3003" --oml-domain rx_samples }}} Rerun the application with these options and we'll see a few additional lines of output at the end indicating a connection to the OML database server. {{{#!shell-session Writing FFT blocks into rx_samples.sq3 @ oml:3003 May 20 23:12:24 INFO OML Client 2.11.1rc-dirty [OMSPv5] Copyright 2007-2014, NICTA INFO tcp:10.37.0.10:3003: Connected INFO tcp:10.37.0.10:3003: Waiting for buffered queue thread to drain... }}} === ''rx_multi_receive'' break down === The source file for this application is in ~/RX_MULTI_RECEIVE/main.cpp. Without going into the details off c++ and the entire contents of the source file, we'll make note of the following blocks of code: 1. A handle to the radio is constructed and is used thoughout the source to make reference to the radio. {{{#!c++ std::cout << std::endl; std::cout << boost::format("Creating the usrp device with: %s...") % args << std::endl; uhd::usrp::multi_usrp::sptr usrp = uhd::usrp::multi_usrp::make(args); }}} 2. Use the handle to configure the radio paramters passed in from the command line. {{{#!c++ //set the rx sample rate (sets across all channels) std::cout << boost::format("Setting RX Rate: %f Msps...") % (rate/1e6) << std::endl; usrp->set_rx_rate(rate); std::cout << boost::format("Actual RX Rate: %f Msps...") % (usrp->get_rx_rate()/1e6) << std::endl << std::endl; //set the rx center frequency std::cout << boost::format("Setting RX Freq: %f MHz...") % (freq/1e6) << std::endl; uhd::tune_request_t tune_request(freq); if(vm.count("int-n")) tune_request.args = uhd::device_addr_t("mode_n=integer"); for (unsigned int ch = 0; ch < num_rx_channels; ++ch) usrp->set_rx_freq(tune_request, ch); std::cout << boost::format("Actual RX Freq: %f MHz...") % (usrp->get_rx_freq()/1e6) << std::endl << std::endl; //set the rx rf gain std::cout << boost::format("Setting RX Gain: %f dB...") % gain << std::endl; for (unsigned int ch = 0; ch < num_rx_channels; ++ch) usrp->set_rx_gain(gain,ch); std::cout << boost::format("Actual RX Gain: %f dB...") % usrp->get_rx_gain() << std::endl << std::endl; }}} 3. A receive streamer is created and a loop is used to read out complex samples into a large buffer that is used during post processing samples. {{{#!c++ //setup streaming std::cout << std::endl; std::cout << boost::format( "Begin streaming %u samples, %f seconds in the future..." ) % total_num_samps % seconds_in_future << std::endl; uhd::stream_cmd_t stream_cmd(uhd::stream_cmd_t::STREAM_MODE_NUM_SAMPS_AND_DONE); stream_cmd.num_samps = total_num_samps; stream_cmd.stream_now = false; stream_cmd.time_spec = uhd::time_spec_t(seconds_in_future); rx_stream->issue_stream_cmd(stream_cmd); //tells all channels to stream while(num_acc_samps < total_num_samps) { //receive multi channel buffers size_t num_rx_samps = rx_stream->recv( MultiDeviceBufferPtrs.at(mdbp_idx++), samps_per_buff, md, timeout); //use a small timeout for subsequent packets timeout = 0.1; //handle the error code if (md.error_code == uhd::rx_metadata_t::ERROR_CODE_TIMEOUT) break; else if (md.error_code == uhd::rx_metadata_t::ERROR_CODE_OVERFLOW) continue; else if (md.error_code != uhd::rx_metadata_t::ERROR_CODE_NONE){ throw std::runtime_error(str(boost::format("Recv'd samples %i\nReceiver error %s") % num_acc_samps % md.strerror())); } if(verbose) std::cout << boost::format( "Received packet: %u samples, %u full secs, %f frac secs" ) % num_rx_samps % md.time_spec.get_full_secs() % md.time_spec.get_frac_secs() << std::endl; num_acc_samps += num_rx_samps; } // while() }}} === Instrumenting ''rx_multi_receive'' application with OML === 4. Once the read buffer is completely populated, create the fft object to convert the time samples into frequency bins. FFT is take for every 1024 time samples and saved into another buffer. Inside this block, we create an object that performs 1024pt FFTs. {{{#!c++ std::vector > time_buff( samps_per_buff ); std::vector > freq_buff( samps_per_buff ); std::vector signal_mag_instant( samps_per_buff ); fftwf_plan fft_p = fftwf_plan_dft_1d( samps_per_buff, (fftwf_complex*)&time_buff.front(), (fftwf_complex*)&freq_buff.front(), FFTW_FORWARD, FFTW_ESTIMATE); }}} We also construct an object for writing data to an OML server. Initialize the OML object with the OML options described above. {{{#!c++ // OML object for sending data CWriteOml oml; oml.init(oml_id, oml_domain, oml_collect ); }}} Create keyed-values that will be sent for storage. Here we specify a string label (for the key, ie. a variable) with an OML data type. After all the keyed-value pairs have been defined, we start a connection with the OML service. {{{#!c++ // Register measurement points oml.register_mp("mboard_id", OML_STRING_VALUE); oml.register_mp("mboard_serial", OML_STRING_VALUE); oml.register_mp("mboard_name", OML_STRING_VALUE); oml.register_mp("rx_id", OML_STRING_VALUE); oml.register_mp("rx_subdev_name", OML_STRING_VALUE); oml.register_mp("rx_subdev_spec", OML_STRING_VALUE); oml.register_mp("rx_ant", OML_STRING_VALUE); oml.register_mp("channel", OML_UINT32_VALUE); oml.register_mp("total_samps", OML_UINT32_VALUE); oml.register_mp("sample_size", OML_UINT32_VALUE); oml.register_mp("rx_freq", OML_DOUBLE_VALUE); oml.register_mp("rx_rate", OML_DOUBLE_VALUE); oml.register_mp("rx_gain", OML_DOUBLE_VALUE); oml.register_mp("SizeFFT", OML_UINT32_VALUE); oml.register_mp("Bins", OML_BLOB_VALUE); // Start recording session oml.start( ); }}} After the OML start has been issued we can set the keyed-value pairs with updated values from sensors. In this section the FFT is performed on chunks of 1024 complex samples and the magnitude is computed. These get stored in the OML as binary data (oml_blob). Once all the keyed-valued pairs are updated with set_key(), they are sented over to the OML server by issuing the insert command. This is done repeatedly until the entire buffer has been processed. {{{#!c++ oml.set_mp("rx_id", (void*)usrp_info["rx_id"].c_str() ); oml.set_mp("rx_subdev_name", (void*)usrp_info["rx_subdev_name"].c_str() ); oml.set_mp("rx_subdev_spec", (void*)usrp_info["rx_subdev_spec"].c_str() ); oml.set_mp("rx_ant", (void*)ant.c_str() ); oml.set_mp("channel", (void*)&ch ); oml.set_mp("total_samps", (void*)&total_num_samps ); ui32 = sizeof(MultiDeviceBuffer.at(ch).at(0)); oml.set_mp("sample_size", (void*)&ui32); d64 = (double)(usrp->get_rx_freq() / 1e6); oml.set_mp("rx_freq", (void*)&d64); d64 = (double)(usrp->get_rx_rate() / 1e6); oml.set_mp("rx_rate", (void*)&d64); d64 = (double)(usrp->get_rx_gain()); oml.set_mp("rx_gain", (void*)&d64); ui32 = samps_per_buff; oml.set_mp("SizeFFT", (void*)&ui32); unsigned int nfftBlocks = MultiDeviceBuffer.at(ch).size() / samps_per_buff; for (unsigned int j = 0; j < nfftBlocks; j++) { // copy samples into time_buff for FFT memcpy((void*)time_buff.data(), (void*)(MultiDeviceBuffer.at(ch).data() + j*samps_per_buff ), samps_per_buff * sizeof(std::complex) ); // Execute FFT. This performs fft on time_buff samples and store results in freq_buff. fftwf_execute(fft_p); // FWD FFT is NOT normalized so div by N. // compute magnitude for(unsigned int z = 0; z < signal_mag_instant.size(); z++) signal_mag_instant.at(z) = abs(freq_buff.at(z)); // save the magnitude values as an OML blob oml.set_mp_blob("Bins",(void*)&signal_mag_instant.front(), (unsigned int)signal_mag_instant.size() * sizeof(float) ); // send all OML variables to server for recording oml.insert(); } } // end for() }}} Once this block completes the connection to the OML server is taken down and no more data can be stored into the database. === View results from OML database === The database is store in /var/lib/oml2 of the console. Use the command line front-end tool (sqlite3) to query the database directly. For a detailed overview on sqlite3 CLI please refer ​http://www.sqlite.org/cli.html. An application has been made to parse this database. This application is very specific to this tutorial. If the format of the data base table changes, the application may no longer be able to parse the database file. Display all the tables in the database. {{{ console> ./db_parse --file /var/lib/oml2/rx_samples.sq3 --showtables name = _senders name = _experiment_metadata name = _mp__rx_samples Use following command to retrieve contents of table. ./db_parse --file /var/lib/oml2/rx_samples.sq3 --exec "SELECT * FROM " To retrieve contents of the blob: ./db_parse --file /var/lib/oml2/rx_samples.sq3 --blob2bin ch0_data.bin --ch 0 --table _mp__rx_samples --key Bins }}} We can recover the contents of the FFT data for each channel and save them into a binary file for viewing in octave. {{{#!shell-session console:~$ ./db_parse --file /var/lib/oml2/rx_samples.sq3 --blob2bin ch0_data.bin --ch 0 --table _mp__rx_samples --key Bins Executing query statement select LENGTH(Bins),Bins from _mp__rx_samples where channel=0 }}} In the console we should have a binary file by the name ch0_data.bin. This can be loaded and viewed in ocatve. {{{#!octave octave> mag0 = fReadBinaryFile('ch0_data.bin','single'); mag0 = reshape(mag0, 1024, length(mag0)/1024); image(mag0*1000) }}} || [[Image(mag0.png, width=500px)]] || === Instrumenting other applications === Steps to instrument other c++ applications: 1. Add ''CWriteOml.h'' & ''CWriteOml.cpp'' into your project. Include the header file in the relevant source file where OML functions would be called. > #include "CWriteOml.h" 2. Identify variables for sensor / measurement data to be recorded. These would be from your original source files. 3. Create a OML class object. > CWriteOml oml; 4. Call the member function ''init()'' to initialize with an ID, database file name and storage location. > CWriteOml::init(std::string oml_id, std::string oml_domain, std::string oml_collect); 5. Register the variables for sensor / measurement data (from above) with a variable name and corresponding type. This is a individual measurement point. > CWriteOml::register_mp(std::string str_variable, OmlValueT oml_value_type); 6. Once the collection of measurement points are registered, call ''start()'' to kick off the recording session. > CWriteOml::start() 7. Update measurement points with new value. > CWriteOml::set_mp(std::string key_str, void* val_ptr); > CWriteOml::set_mp_blob(std::string key_str, void* val_ptr, unsigned int omlblob_bytes); Use ''set_mp()'' for int32, int64, uint32, uint64, double, string.[[BR]] Use ''set_mp_blob()'' for binary data.[[BR]] These are the most common types being used. 8. When ready to record the collection of measurement points call ''insert()''. > CWriteOml::insert() 9. The application is done with the recording session, use ''stop()''. > CWriteOml::stop() 10. Compile and link the application with ''-loml2 -locomm''