Week 8: QT GUI

This week has been dedicated to the QT GUI blocks. More specifically, porting them to YAML and handling them in cpp_top_block.py. I have uploaded all the QT blocks in YAML format to my grc_yaml_blocks repository. There are some details missing though, because there is still some Cheetah-style code left to edit.

The QT blocks don’t play very well with the way I’ve designed the YAML files’ cpp_templates, because they have more than just the standard module::block::make(…); line in their make field. This is a bit unfortunate, as it messes with the #include statements as well, since they are generated by looking at the make field. Currently I am trying out moving all the other code into its own field called additional_make. This seems to fit the QT blocks so far, and I might go with this approach if it doesn’t introduce any new problems.

I have also been working on a few bug fixes for the Python generation code. Today’s blog post is of the shorter kind, but the second milestone is in a few days (Thursday), and I’ll write up a lengthier one, hopefully with a demonstration, for that in the weekend.

Advertisements

Week 7: Vacation

I spent the previous week participating in a military exercise, and there’s not much to report because of this. However, I did create a poster of my project for the 2017 GNU Radio Conference:

https://drive.google.com/open?id=0B9ht6cKX6zczUWQxSHhIV3ZBZ28

There are some parts that don’t work quite yet or are bugged, and these are mostly pertaining to parameters, build options and the ‘Execute’ button. Hierarchical blocks don’t work yet either, but this is true for the upstream python3 branch as well.

The next step for me now is working towards the second milestone, which I have, together with my mentors, decided to move from September 17 to September 21. This milestone includes the ability to create QT applications written in C++ from GRC.

Week 6: Milestone #1 Demo

This week’s blog post comes a few days earlier than usual. This is because I have a military exercise starting tomorrow, which means I won’t get any work done until next weekend. However, my SOCIS project has reached its first milestone, and I will dedicate this post to a demonstration of the project’s current state. This demonstration will be a bit more detailed than the one from week 3, so everyone can try out my changes. I will show you a simple flow graph with two signal sources, an add block, a throttle block and a file sink.

First of all, you’ll have to clone my repository with the following commands:

git clone https://github.com/hauk142/gnuradio.git

cd gnuradio

Second, switch to my python3_demo branch and checkout VOLK:

git checkout python3_demo

git submodule init

git submodule update

Next, you are ready to build GNU Radio. For the purpose of this demonstration, the CMake command below disables some components. Feel free to enable them, but it’ll cost you some time when building.

mkdir build

cd build

cmake .. -DENABLE_GR_COMEDI=OFF -DENABLE_DOXYGEN=OFF -DENABLE_GR_DTV=OFF -DENABLE_GR_DIGITAL=OFF -DENABLE_GR_FEC=OFF -DENABLE_GR_UHD=OFF -DENABLE_GR_TRELLIS=OFF

make

After make finishes, you are ready to install and run GRC:

sudo make install

export PYTHONPATH=/usr/local/lib/python3.6/dist-packages/

export LD_LIBRARY_PATH=/usr/local/lib/

gnuradio-companion

The two ‘export’ commands will have to be run every time you run GRC in a new terminal. You can put them in a shell script for easy access, but remember that you need to use source for it to work.

Now GRC should be open and you can continue by opening a file called fsink.grc. You can find it in the demonstration directory from the cloned GitHub repository.

flow3.png

 

Your screen should look something like the image above. This flow graph is almost ready to be executed, all you have to do is double-click the file sink and choose in which file you want your signals to end up. Next, you can press the “play” button at the top, and dismiss the warning. GRC will freeze for a few seconds (will be fixed), and a terminal will appear:

run.png

 

Congratulations, you’ve just generated, built and executed your first program with GRC’s brand new C++ output functionality!

It might seem like nothing’s happening, but actually, GNU Radio is filling your selected file with the output from the signal sources. You can press Enter to quit and the file can now be inspected however you’d like. You might notice an error upon pressing Enter. This is an issue that will be fixed once I figure out the cause. Please note that I haven’t added support for GUI flow graphs yet, so you won’t be able to create a flow graph with a file source and a QT plot to view the contents of the file. Also, I have only added a few YAML blocks in this branch, but you can find more in my grc_yaml_blocks repo.

I modified the display_qt example from the gr-qtgui module, opened the file, and got this plot:

plot.png

 

I have included this program in the plot directory in my python3_demo branch, and you can run it like this:

cd plot

mkdir build

cd build

cmake ..

make

export PYTHONPATH=/usr/local/lib/python3.6/dist-packages/

export LD_LIBRARY_PATH=/usr/local/lib/

./display_qt ‘/home/hkon/signals’

Where signals is the file you generated with GRC.

So that’s it! I hope, if you tried to follow my steps, that you managed to get through the demonstration without running into an error you couldn’t resolve (and if you did, please don’t hesitate to ask) 🙂 The next step for my SOCIS project is adding support for GUI flow graphs.

 

Week 5: Parameters and Variables

This week’s focus has been the transpiling functionality for parameters and variables. This is done using python’s re module and simple regex matching. A more sophisticated approach would be to make use of the ast module, but I have decided not to include this as part of my SOCIS project. The reason for this is that the university semester has begun (the coding period kick-off was shifted due to delays) and I want to keep my work load manageable. I’m keen to keep working on this after SOCIS though!

I have also created a GitHub repository for YAML files for blocks with C++ support. So far, I have uploaded most of the gr-analog blocks, all of the ZeroMQ blocks and a couple of gr-blocks blocks. The addition of C++ support brings some changes to the blocks, which I will show you below:

keke
(larger image here)

The presence of the ‘has_python, has_cpp’ flags enables GRC to check if all the blocks in the flowgraph support the current output language. If not, GRC will refuse to generate code. This is implemented in my python3_dev branch.

Also, the cpp_templates field includes information essential to the generation of C++ code. includes contains the required include statements and callbacks is exactly the same as its Python counterpart. make contains the make statement for the block. You can find it in the top_block’s constructor. The link field simply tells CMakeLists.txt which libraries to link against. Finally, translations fixes the problem of enums or bools in a block’s parameters. Very few blocks use enums, but there are some that use bools, and ‘True’ is written with a lowercase ‘t’ in C++. The parts that need editing are translated when the user generates the C++ code, after the block templates have been rendered.

Week 4: CppTopBlockGenerator

This week’s focus has been the CppTopBlockGenerator, which is a subclass of the standard TopBlockGenerator and can be found in grc/core/generator/cpp_top_block.py. I have also been working on the C++ hier block generation, which still needs some work. The next step is working out a way to take care of parameters and variables. My current plan is that simple expressions will be transpiled, along with some popular objects such as numpy-arrays and similar.

I have also been spending some time trying out different ways to decide which GNU Radio modules to build the C++ files against when generating CMakeLists.txt. This obviously needs some attention since different blocks rely on different modules.

One possibility is to keep a kind of cache in GRC over the installed modules. You can’t create flowgraphs with blocks from modules that you don’t have installed, so this could have been a viable option, but the problem with this approach is that CMakeLists.txt may have to be edited to remove any modules that aren’t required, especially if that CMakeLists.txt is going to be used with a different installation of GNU Radio. 

Another solution would be to “scan” the flowgraph for the modules required to build it. For example, I can parse the #include statements from the blocks’ YAML files to decide which modules are required. This may or may not take some more work, but it generally sounds like a more elegant approach that will give a smoother and more streamlined user approach. 

Week 3: CMakeLists.txt Template and Demo

This week’s focus has been the CMakeLists.txt template. However, the most notable change has been my progress with the CppTopBlockGenerator class. At the moment, GRC is able to produce the planned C++ files, but there are still some details and quirks remaining. I’ll give a small demonstration to show the current state of the project:

As previously mentioned, my work is mainly done on the python3_dev branch, and to compile it and try it out for yourself you will have to apply the small fixes from my Week 1 post. Please note, however, that the fix pertaining to uses_var_id() has already been applied in that branch. When you launch GRC, you will notice a new addition to the options block:
DeepinScreenshot_gnuradio-companion_20170813163422

This option governs the output language, and will decide whether your flowgraph produces a .py file or a .cpp file, an .hpp file and CMakeLists.txt. The option is manipulated with a basic drop-down menu:

DeepinScreenshot_gnuradio-companion_20170813164018.png
You can now just click “Generate” as you would normally, and GRC will generate the three aforementioned files. Please remember to set an ID in the options block, otherwise you  will get an error. To demonstrate, I created a flowgraph with a signal source connected through a throttle block to a null sink. GRC yielded the following files (Pastebin link):

top_block.cpp
top_block.hpp
CMakeLists.txt

As you can see, there are still some pieces missing and parts that are Python in there, but it’s something. I’m glad I can finally see some actual results from my work 🙂 The next steps are adding support for hier blocks and further refinement of the CppTopBlockGenerator class, before I take on parameter and variable handling.

Now, if you try to do exactly what I did in this brief walkthrough, you won’t be able to generate the same files. This is because I have added C++ components to the blocks’ YAML files, which aren’t tracked in Git at the moment. The block tree will be converted to YAML eventually, but is still using XML at the moment. You can find the files I have edited here:

analog_sig_source_x.block.yml
blocks_throttle.block.yml
blocks_null_sink.block.yml
options.block.yml
stream.domain.yml

Just replace the autoconverted ones in ~/.cache/grc_gnuradio/ with these, and you should be all set to generate the flowgraph I demonstrated above.

Week 2: Source File Template

This main focus for this week has been the source file template. The templates are coming along pretty well, and most of the work remaining to get C++ output from non-GUI flowgraphs is related to coordinating the Generator classes with the Mako templates. Actually, I did manage to trick GRC to generate a C++ header file earlier today, but it was pretty messy and not quite ready for this blog yet 🙂 Other than that, I have been working on resolving some miscellaneous quirks within GRC.

 

GNU Radio’s master branch currently uses XML for describing the blocks, and Cheetah as the template engine. These are in the process of being phased out however, with Mako replacing Cheetah and YAML replacing the XML files.

This is the YAML file for the Throttle block:
DeepinScreenshot_gedit_20170806230521
I have omitted the top three lines, describing the id, label and flags. The remainder of the file lists the data required to use the block in your program. The parameters key contains the options for the block. You can find these by double-clicking the block in GRC:

DeepinScreenshot_gnuradio-companion_20170807000232

You can find each option in the YAML file, there’s one for each ““. You can also see the label (the text you see in the GUI), the type, default value and the conditions for hiding when you are viewing your entire flowgraph.

The next two keys are the inputs and outputs, also visible from within GRC:

DeepinScreenshot_gnuradio-companion_20170806235406

The third, templates, incorporates the code-related part of the block and does not have anything to do with the GRC GUI. This part defines the code that, together with the other blocks and the Mako templates, make up your generated code. More specifically, the required libraries, the call to the block’s constructor and the callbacks.

Week 1: Header File Template

This week’s theme has been the header file template. The contents of the header file are mainly the #include statements, the top block (or hier block) class declaration, and declarations of other things such as blocks and variables. This does not amount to much code in comparison to the source file, which will include everything else such as constructors, definitions, main() and so on. However, it’s a great starting point. This week’s work hasn’t exactly yielded much of an immediately visible profit, but this is due to the fact that all the blocks still are Python only and the generation code is more or less untouched at this point. The C++ output will come 🙂 There’s also a small practical change, from now on I will be pushing daily commits to a python3_dev branch and keeping the larger changes on the python3 branch.

 

The python3 branch from the main repository does not run without error right now, but I’ve gathered the steps necessary to get it running:

First, you’ll have to edit grc/scripts/gnuradio-companion at line 84:

def check_python_version():
         if sys.verion_info.major > 2:
                 die(RuntimeError(‘No python3 support yet.’), ‘Stay tuned…’)

The program will die here if you’re running it with Python 3, of course, so you’ll want to fix this.

Second, you’ll have to take a look at grc/core/generator/top_block.py at line 206 and 212:

def uses_var_id():
(…)
callbacks[var_id] = [callback for callback in callbacks_all if uses_var_id()]

This creates an error because callback is used within the uses_var_id() method. I suspect this is a Python 3 migration problem. This error will hinder GRC from generating any flow graph files. The fix is easy though:

def uses_var_id(callback):
(…)
callbacks[var_id] = [callback for callback in callbacks_all if uses_var_id(callback)]

So, with those fixes in place, you should be able to run GRC and generate the top block Python file.

EDIT: In fact, there are a couple more steps you’ll need to take to get it running:

To avoid getting the “domain not registered” errors, you’ll have to copy the YAML files stream.domain.yml and message.domain.yml from grc/blocks/ to your block cache. This is probably at ~/.cache/grc_gnuradio/

The next step is adding “generator” at the end of grc/CMakeLists.txt at line 152, like this:

DIRECTORY core gui generator

That’s it, you’re all set!

Week 0: Introduction

I am Håkon Vågsether, second year student in Cybernetics and Robotics at the Norwegian University of Science and Technology in Trondheim, Norway. I am participating in ESA’s Summer of Code in Space (SOCIS) program this summer, contributing to the GNU Radio project. Due to some administrative delays, the SOCIS schedule has been shifted this year, but we’re finally getting started.

I will be working on implementing support for C++ output for the GNU Radio Companion (GRC), mentored by Sebastian Koslowski and Derek Kozel. My work will be on the python3 branch from the main repository. The project will last for 10 weeks, and each week will have a specific theme, with tasks specified in my project proposal. This week has been spent doing initial administrative work such as revising the timeline and setting up this blog.

The proposal can be found here.

Learn more about GNU Radio.

gnuradio_logo

Image source: http://www.gnuradio.org