Performance monitoring

See the resources Bela is using in real time

When a process is running, you may want to monitor the CPU usage, in order to know how much CPU power is left on the board for you to use.

Another important metric for Xenomai programs is that time-critical tasks should always run in primary (Xenomai) mode and never in secondary (kernel) mode or switch back and forth. A mode switch occurs every time the task switches from primary to secondary mode or viceversa.

When using the IDE, the cumulative CPU usage of the Xenomai tasks is displayed on the main toolbar, provided you ticked the appropriate box in the settings. The “IDE” percentage is an estimate of the CPU used by the IDE and its sub-processes (e.g.: the compiler or the scope). As this value is an estimate, you will occasionally see it go above 100%.

Every time a mode switch is detected in the bela-audio thread, this is logged to the IDE window.

You can monitor CPU usage and mode switches in more detail either activating the “Xenomai stats” option in the IDE or from the command line.

Monitoring CPU and mode switches from the IDE carries quite some CPU overhead.

The file /proc/xenomai/sched/stat contains the current CPU usage for all the Xenomai tasks (on Bela images < v0.3, the file is /proc/xenomai/stat). To monitor it, once you have sshed into the board, you can run:

$ watch -n 0.5 cat /proc/xenomai/sched/stat

Here you can see the CPU used by every Xenomai thread when running in primary (Xenomai) mode. The CPU usage of tasks running in secondary (kernel) mode, or switching back an forth between the two will be displayed, as a cumulative value, through the usual top command:

$ top -p `pgrep projectName`

Mode switches are listed in the MSW column and this value should NEVER increase for the bela-audio thread (it usually starts at 2). Non real-time critic tasks that continuously switch back and forth between primary and secondary mode, for which MSW increments rapidly and steadily, are less evil, but should still be avoided whenever possible.

More info on mode switches, how to detect them and how to avoid them can be found at the URLs below:

  • https://xenomai.org/2014/08/porting-a-linux-application-to-xenomai-dual-kernel/#Chasing_the_unwanted_mode_switches
  • https://xenomai.org/2014/06/finding-spurious-relaxes/

We are using Xenomai 2.6.3 and the option CONFIG_XENO_OPT_DEBUG_SYNCH_RELAX is enabled in our kernel) .