PARP Research Group University of Murcia, Spain


User interface

Creating powerful graphical user interfaces in QVision applications is quite simple. The framework offers a versatile default input GUI which gives the user complete control on every desired parameter, as well as on the execution flow of every worker. Besides, it also offers a complete set of classes (contained in the Graphical User Interface group) to create graphical widgets easily integrable with any previously created processing block structure, composed by several workers and other kind of input/output blocks.

Input widgets

These widgets do literally discover every dynamic property contained in each worker object (thanks to a self introspection technique), and conveniently offer the adequate widget (slider, button, checkbox, edit widget, or a combination of them) for the user to inspect and modify their corresponding values at execution time (as well as through the command line, during the initialization of the application). QVision has predefined widgets for dynamic properties of common basic types, such as boolean, integer, double, character string, etc.

The developer of a worker class, then, just has to take care of declaring the parameters of the algorithms he wants to be modificable by the user at execution time as dynamic properties contained in the worker object, of the input type (declaring them using the InputFlag flag). The introspection mechanism takes care of all the rest. Take, for example, the constructor of the following Canny edge detector worker:

[...]

class QVCannyEdgeDetector: public QVWorker
        {
        public:
                QVCannyEdgeDetector(QString name): QVWorker(name)
                        {
                        addProperty<double>("cannyHigh", inputFlag, 150, "High threshold for Canny operator", 50, 1000);
                        addProperty<double>("cannyLow", inputFlag, 50, "Low threshold for Canny operator", 10, 500);
                        [...]
                        }

                void iterate()
                        {
                        [...]
                        }
        }

Observe that two double input properties are declared. Let us concentrate on the first one. It needs a name ("cannyHigh"), the aforementioned inputFlag flag (which declares it as an input parameter for the worker), a default value (150), a short description, and an interval for valid values ([50,1000]). With just this property declaration in the constructor of the worker, the default GUI generated by QVision will generate an adequate input widget that will allow the user to dynamically change the property value during execution. There are also default input widgets for int, bool and Qstring types. See, for example, the following snapshot, corresponding to a QVision application with a Canny worker, which contains several input properties of bool, int, and double types:

qvisioninterface_input_widgets.png

Once the input properties of each worker have been conveniently declared in the corresponding constructors, input widgets will be created automatically by simple declaration of a QVDefaultGUI instance in the main function. We describe this class in the next section.

The default graphical user interface

Class QVDefaultGUI is used in QVision applications to create a default GUI that offers the user the possibility to control the execution of the different workers registered in the application, the flow of the input video camera objects, and modify the input parameters defined as mentioned before (and which are not linked to the output of other workers, in whose case they obviously should not be controlled directly by the user).

A single instance object of the QVDefaultGUI class should be created right after the QVApplication object in the main function, and before the call to exec():

int main(int argc, char *argv[])
        {
        QVApplication app(argc, argv,
                "Example program for QVision library. Obtains several features from input video frames."
                );
        
        QVDefaultGUI interface;

        [...]

        return app.exec();
        }

The following screenshot shows the main QVDefaultGUI for a QVision example application:

qvisioninterface_default_gui.png

There are three main widget areas in the QVDefaultGUI window:

A. Menu area

Contains the Quit, Window and Help options. The first one, naturally, quits the application. The second is a typical window menu that allows the user to show/hide all the windows of the application. Finally, the last one shows a descriptive help of the application, as defined by the programmer when declaring the main QVApplication object and the help text corresponding to the properties of the involved workers. For example, here is the help window which is shown for an example application which contains, among some others, a Canny Operator Worker:

qvisioninterface_help.png

And here is some of the code responsible for the information shown:

[...]

class QVCannyEdgeDetector: public QVWorker
        {
        QVCannyEdgeDetector(QString name): QVWorker(name)
                {
                addProperty<double>("cannyHigh", inputFlag, 150, "High threshold for Canny operator", 50, 1000);
                addProperty<double>("cannyLow", inputFlag, 50, "Low threshold for Canny operator", 10, 500);
                addProperty<bool>("applyIPE", inputFlag, FALSE, "If we want to apply the IPE algorithm");
                addProperty<double>("paramIPE", inputFlag, 5.0, "IPE parameter (max. allowed distance to line)", 1.0, 25.0);
                addProperty<bool>("intersectLines", inputFlag, TRUE, "If we want IPE to postprocess polyline (intersecting lines)");
                addProperty<int>("minLengthContour", inputFlag, 25, "Minimal length of a contour to be considered", 1, 150);
                addProperty<int>("showNothingCannyImage", inputFlag, 0, "If we want nothing|Canny|original image to be shown",0,2);
                addProperty<bool>("showContours", inputFlag, TRUE, "If we want contours to be shown");
        
                addProperty< QVImage<uChar,1> >("Output image", outputFlag);
                addProperty< QVImage<uChar,3> >("Input image", inputFlag|outputFlag);
                addProperty< QList<QVPolyline> >("Output contours", outputFlag);
                }

        [...]

        }

int main(int argc, char *argv[])
        {
        QVApplication app(argc, argv,
                "Example program for QVision library. Obtains several features from input video frames."
                );
        [...]

        QVCannyEdgeDetector cannyWorker("Canny Operator Worker");

        [...]

        QVMPlayerCamera camera("Video");

        [...]

        }

Several important facts should be noted here:

  • The main help of the application is defined when declaring the QVApplication object in the main() function.
  • Only input properties of the worker are documented, and thus appear in the help window.
  • There are several input properties that are present in every worker, because they are inherited from the base QVWorker class (for example, the first three properties which appear in the help for the Canny Operator Worker).
  • The QVMPlayerCamera object (which is the input block which provides a continuous image source for other workers) has also its associated help, with convenient input properties (number of rows, columns, and so on). In fact, it is internally just a special kind of worker, and as such, it is treated here just the same way.

B. Workers area

This area contains a tabbed widget for every worker in our application. By selecting the adequate tab, we can access to both input and output parameters of the selected worker. Each of these widgets is connected to a corresponding property of the worker (only if they are not linked to other workers, of course). The user can thus modify the values of every unlinked input parameter at will in execution time. There is also a control area, which allows us to pause, resume, and stop the execution for the worker, as well as accessing to a time statistics widget.

Here is a list with the detailed description of the buttons and their functionality:

Pause button
qvisioninterface_pause.png
This button suspends the processing of the worker. Every worker synchronized with it will be stopped as well, until the resume button is pushed.

Resume button
qvisioninterface_play.png
If the worker is paused, pushing this button resumes its execution.

Step button
qvisioninterface_step.png
This button can be pushed when a worker is paused to make it iterate just once again (i.e., it executes a complete iteration and pauses again).

Stop button
qvisioninterface_stop.png
This button finishes the processing of the worker. Its properties will be automatically unlinked and frozen, so any worker connected to them will keep on reading the same (frozen) values from then on.

CPU statistics button
qvisioninterface_cpustats.png
This button opens a detailed cpu usage plot for the worker. It shows a window with the cpu time statistical plot of the time flags defined in the iterate() function of the worker. You can see an example of this plot in the following figure:

qvisioninterface_cpustat_canny.png

Function QVWorker::timeFlag() can be used in the body of the QVWorker::iterate() function to configure a time flag in the execution of a worker.

Note:
CPU stat plot depends on real execution time between time flags, so when two or more workers compete for one CPU, times can differ a lot with respect to executions in environments in which each worker runs on its own CPU.

C. Cameras area

Analogous to the workers area, this is another tabbed widget with a tab for each camera declared in the QVision application (in fact, internally they are just special kind of workers). Here we can pause, resume, and stop the video input flow, as well as reopening the source (video file, camera, or whatever) changing the desired parameters (size, deinterlacing or whatever).

Here is a description of the main camera control buttons and their functionality:

Pause button
qvisioninterface_pause.png
This button stops the camera from "publishing" new frames from the video source (if the camera is on real-time mode it will keep reading frames, but won't send them to the workers connected to it; for video files and non-realtime mode, the video will be effectively paused; in any case, connected workers will keep on receiving the same paused frame).

Resume button
qvisioninterface_play.png
This button resumes grabbing.

Step button
qvisioninterface_step.png
When the camera is paused, this button makes it read the next frame in the video input, keeping the camera paused afterwards.

Stop button
qvisioninterface_stop.png
This button stops the camera from reading frames from the input source. The camera must be reopened to work again.

Here is an screenshot of the camera widget interface:

qvisioninterface_camera.png

Observe that you can easily change the input video source at execution time, by reopening it with the new desired URL, size and features (deinterlacing and loop mode). The widget also provides some pieces of information on the video source, such as the current position, video size and FPS. Finally, and of course just for video files, the user can also use the position slider to directly move the video source to a desired time position.

The image canvas widget

Class QVImageCanvas can be used to create a very flexible and useful widget to show QVImage objects. This widget is in fact a property container, and as such it will read the image to be displayed from an output dynamic property contained in another property container (usually a QVWorker object).

It is used as follows. First, the programmer must create the QVImageCanvas object in the main() function of a program; then, a link to a QVImage type property from a property holder must be created. For example:

[...]

CannyWorker cannyWorker("Canny");

[...]

QVImageCanvas imageCanvas;
cannyWorker.linkProperty(imageCanvas,"Canny image");

[...]

When the application executes, it will automatically create a window like the following:

qvimagecanvas_penguin.png

You can see it has a zoom number indicator (z = 1), horizontal and vertical rules (in pixels), and some buttons. These can be used to control zooming and moving around a zoomed area of the image. A detailed explanation of each one follows:

Zoom in button
qvimagecanvas_zoomin.png
This button zooms the image in. Canvas window doesn't change size, but the area of the image displayed becomes smaller by a factor of 4 (width and height gets divided by 2). The label of the left bottom corner displays the text
z=<zoom>
where <zoom> is a number indicating the zoom factor that divides width and height in the actual zoom level.

Zoom out button
qvimagecanvas_zoomout.png
This button divides zoom factor by 2, if it is equal or greater than 2. If the canvas window is bigger than the image at the final zoom, it is adjusted to the size of the latter.

Zoom restore button
qvimagecanvas_zoomrestore.png
This button sets zoom factor by 1, and resizes canvas window to its original size.

Select zoom region button
qvimagecanvas_zoomregion.png
This button lets you select a rectangle in the image to zoom, adjusting zoom factor and canvas window size to it.

Select polyline button
qvimagecanvas_polyline.png
This button lets you select points in the image. The created points list can then be accessed to in the program by reading the "poly select" imageCanvas output property. You can expand this button by pushing it for a few seconds, and then you will be able to select a polyline representation, a point list representation, or generate a circular polyline, by clicking and dragging.

Select ROI button
qvimagecanvas_roi.png
This button lets you select a rectangle in the image. That rectangle can be again accessed to using the "rect select" imageCanvas output property.

Drag mode
qvimagecanvas_drag.png
This button activates drag mode. With it you can move around the image displayed in the canvas window, if the zoom factor forces the canvas window to show only a sub-region of the image, by holding click and dragging on the shown image.

In the following figure you can see a zoomed canvas window showing a sub-region of the original image:

qvimagecanvas_penguinpixels.png

Another interesting feature of this widget is that at a zoom factor equal or larger than 32, the canvas renders the gray-scale pixel value over every pixel if the image is gray-scale, or the three different values for each of the RGB channels over every pixel, if the image is RGB, as shown below:

qvimagecanvas_penguinpixelnumbers.png

The canvas also knows how to show numeric float values, when showing images of type QVImage<sFloat,1> or QVImage<sFloat,3>.

Numeric plot widget

Class QVNumericPlot shows a widget which will read a set of integer or double values and plot their time evolution during the corresponding worker execution.

The abscissa axis will display the number of iterations (or the number of elapsed seconds, depending on the boolean time parameter of the constructor QVNumericPlot::QVNumericPlot ). The ordinate axis will scale to the maximum value of the integer or double values read from the worker (or workers) in the time interval displayed. One QVNumericPlot widget can read integer or double values from several workers, but obviously, for this to work correctly, either a) the workers must be synchronized, or b) the QVNumericPlot should display the elapsed time in the abscissa axis (not the number of iterations).

To show the usage of the class, first we need a worker that produces some integer or double output values which we are interested in monitoring. The following code shows an example:

class MyWorker: public QVWorker
        {
        public:
                MyWorker(QString name): QVWorker(name)
                        {
                        addProperty<int>("val_int", outputFlag);
                        addProperty<double>("val_double", outputFlag);
                        [...]
                        }

                void iterate()
                        {
                        int     val_int;
                        double  val_double;
                        [...]
                        // Calculate values for 'val_int' and 'val_double' variables.
                        [...]
                        setPropertyValue<int>("val_int", val_int);
                        setPropertyValue<double>("val_double", val_double);
                        [...]
                        }
        }

And then, in the main function, we can create a QVNumericPlot object and link it with those properties, like this:

#include <QVNumericPlot>

void main()
        {
        [...]
        MyWorker myWorker("worker");
        QVNumericPlot numericPlot("Values int y double",false);
        myWorker.linkProperty("val_int",numericPlot);
        myWorker.linkProperty("val_double",numericPlot);
        [...]
        }

The result will be an application showing the following window widget:

qvision_numericplot_window.png

If the numeric plot would have been declared with true as last parameter (instead of false), then the abscissas axis would show time in seconds (instead of number of iterations), like this:

qvision_numericplot_window_true.png

Histogram plot widget

Class QVHistogramPlot shows a widget which will read a QList<double> from a worker's output property, and will continuously plot its evolution in time during worker execution.

The abscissas axis will correspond to the QList size, while the ordinate axis will scale to the maximum value of the double values in the list read from the worker.

To show the usage of the class, we need again a worker which produces the QList<double> of values to be plotted. The following code shows an example:

class MyWorker: public QVWorker
        {
        public:
                MyWorker(QString name): QVWorker(name)
                        {
                        addProperty<QList<double> >("histogram", outputFlag);
                        [...]
                        }

                void iterate()
                        {
                        QList<double> histogram;
                        [...]
                        // Calculate values for the 'histogram' variable.
                        [...]
                        setPropertyValue<QList<double> >("histogram", histogram);
                        [...]
                        }
        }

And then, in the main function, we can create a QVHistogramPlot and link it with that property, like this:

#include <QVHistogramPlot>

void main()
        {
        [...]
        MyWorker myWorker("worker");
        QVHistogramPlot histPlot("Histogram", false, 1);
        myWorker.linkProperty(histPlot,"histogram");
        [...]
        }

The result will be an application showing the following window widget:

qvision_histogramplot_window.png

The second and third parameters in the constructor of the QVListPlot correspond to time (true) or iteration (false) based operation, and the refreshing interval (in hundreths of seconds, in the first case, or in absolute number of iterations, in the second case).

List plot widget

Class QVListPlot is another widget to show QList<double> properties from workers, just like QVHistogramPlot. The only difference is in the way of plotting the lists of values (colored lines, instead of bars), and, for this same reason, the possibility of plotting several QList<double> properties simultaneously.

Again, the abscissas axis correspond to the maximum size of the input QList's, while the ordinate axis will scale to the maximum value of the double values in them.

This widget is used just as before, but now several lists can be linked simultaneously:

#include <QVListPlot>

void main()
        {
        [...]
        MyWorker myWorker("worker");
        QVListPlot listPlot("Lists plot", false, 1);
        myWorker.linkProperty(listPlot,"list1");
        myWorker.linkProperty(listPlot,"list2");

        [...]
        }

Here is a snapshot of the corresponding output widget:

qvision_listplot_window.png

Again, the second and third parameters in the constructor of the QVListPlot control the time or iteration based operation, and the refreshing interval.

CPU performance plot widget

The developer of a worker class can divide the processing of each call to the QVWorker::iterate() function in a sequence of different stages whose computing times he wants to measure. The programmer must simply use calls to the QVWorker::timeFlag() function to mark the desired computing stages. For example, the following code of the iterate method of a worker

MyWorker::iterate()
        {
        [...]
        timeFlag("Read parameters");
        [...]
        timeFlag("Call to getComponentTree for low areas");
        [...]
        timeFlag("Prune low areas from image");
        [...some other processing stages and corresponding timeFlag's...]
        }

will stablish some performance "breakpoints" in the function. The QVWorker::timeFlag() function registrates the time elapsed between each two of those breakpoints, and stores some time statistics in the worker, which can be later displayed by simply pressing the CPU statistics button in the desired worker tab of the default GUI window. Of course, computational load of these timeFlags is extremely low, and they can be used ubiquitously without almost affecting global performance.

Here is a screenshot for the above coding example:

componenttree_cpustat.png

For advanced users, which could not be interested in using the default GUI, the class QVCPUPlot can still be used to display the CPU usage statistics of a worker. For example, the following main function

void main()
        {
        [...]
        MyWorker myWorker("name");
        [...]
        QVCPUPlot cpuPlot("CPU Plot", true, 10);
        cpuPlot.linkProperty(myWorker); 
        }

will create in execution time the following window, displaying time statistics for the different time segments specified with the QVWorker::timeFlag() method, just as before.

3D OpenGL Widgets

Todo:
Comment 3D openGL widgets, with some example...

Command line parameters in QVision applications

Dynamic properties of certain types, contained in workers or camera objects in a QVision application, are automatically detected by the QVApplication object before the call to the QVApplication::exec() method, again thanks to their self introspection capabilities. The QVApplication object also parses the input console command line used to launch the application. By doing both things, this object allows the user to stablish initial values for those input dynamic properties through parameters in the command line. These command line assignable dynamic properties line must be input properties of type integer (int), double (double), boolean (bool), or character strings (QString).

Using the --help command line parameter, every QVision application displays an usage reference text (equivalent to that shown in the online help of the application, as mentioned at the beginning of this chapter), including a description of the command line assignable dynamic properties detected in the application, including their valid value range, their default value, and a short description for them. The properties are displayed grouped with other properties contained in the same worker object.

For example, a theoretical QVision application named qvapplication could be executed with the following command line:

 ./qvapplication --help 

Which will make it display the following message:

Usage: ./qvapplication [OPTIONS]
QVision application which... (some sort description here, as given to the QVApplication constructor by the programmer).

Input parameters for Video:
  --Rows=[int] (def. 0)                                                      Rows to open the camera.
  --Cols=[int] (def. 0)                                                   Columns to open the camera.
  --RealTime=[true,false](def. false)               If the camera should be opened in real time mode.
  --Deinterlaced=[true,false](def. false)        If the camera should be opened in deinterlaced mode.
  --NoLoop=[true,false](def. false)                   If the camera should be opened in no loop mode.
  --URL=[text] (def. '')                                                     URL of the video source.

Input parameters for Canny Operator Worker:
  --max worker iterations=[int] (def. -1)  Stablishes maximal number of iterations to execute worker.
  --stats enabled=[true,false] (def. true)      Stablishes if the worker's cpu stats will be enabled.
  --cannyHigh=[50...1000] (def. 150)                               High threshold for Canny operator.
  --cannyLow=[10...500] (def. 50)                                   Low threshold for Canny operator.
  [...]

[...Some other input parameters corresponding to other workers...]

We can see, for example, that the worker Canny Operator Worker includes an input property named cannyHigh, of double type. This property has a valid range of values between 50 and 1000, and its default value is 150. Along with a short text definition for the property (High threshold for Canny operator</i>), all these characteristics are specified in the call to the method QVPropertyContainer::addProperty which adds the property in the constructor of the worker object:

class QVCannyEdgeDetector: public QVWorker
        {
        public:
                QVCannyEdgeDetector(QString name): QVWorker(name)
                        {
                        addProperty<double>("cannyHigh", inputFlag, 150, "High threshold for Canny operator", 50, 1000);
                        [...]
                        }
        [...]
        }

The following command line starts the application, specifying some initial values for the property URL in the camera object (with a given video source URL, see section Video source identifier URL formats for details), and the property cannyHigh in the Canny Operator Worker object:

 ./qvapplication --URL=http://perception.inf.um.es/videos/misc/penguin.dv --cannyHigh=300 

The command line parameters specifying values for the worker properties must be separated by spaces in the command line. They must start with a double dash, followed by the name of the property (which must itself be quoted if it contains spaces; for example, --"some property"=1000). Next must follow an = (equal sign), and the value we want to initially store in the property.

Optionally, the name of the worker can be specified in the parameter, between the double dash and the property name, to resolve name conflicts, when two workers have a property referenced with the same name. For example, the previous command would be equivalent to the following:

./qvapplication --Video:URL=http://perception.inf.um.es/public_data/videos/misc/penguin.dv --"Canny Operator Worker":"cannyHigh"=100 

In case of unresolved name conflict, every property with the given name of every worker in the application will be initialized with the given value.




QVision framework. PARP research group, copyright 2007, 2008.