Developing microblx blocks

Overview

Generally, building a block entails the following:

1. declaring configuration: what is the static configuration of a block

2. declaring ports: what is the input/output of a block

3. declaring types: which data types are communicated / used as configuration

4. declaring block meta-data: providing further information about a block

5. declaring and implementing hook functions: how is the block initialized, started, run, stopped and cleaned up?

   1. reading configuration values: retrieving and using configuration from inside the block

   2. reading and writing data from resp. to ports

6. declaring the block: how to put everything together

7. registration of blocks and types in module functions: make block prototypes and types known to the system

The following describes these steps in detail and is based on the (heavily) documented random number generator block (std_blocks/random/).

Note

Instead of manually implementing the above, a tool ubx-genblock is available which can generate blocks including interfaces from a simple description. See Generating blocks with ubx_genblock.

Declaring configuration

Note

Since microblx v0.9, static block definitions must use the “proto” types ubx_proto_config_t, ubx_proto_port_t and ubx_proto_block_t to define prototype blocks. At runtime (i.e. in hooks etc) the non-_proto_ versions are used as before

Configuration is described with a { 0 } terminated array of ubx_proto_config_t types:

ubx_proto_config_t rnd_config[] = {
    { .name="min_max_config", .type_name = "struct random_config" },
    { 0 },
};

The above defines a single configuration called min_max_config of the type struct random_config.

Note

custom types like struct random_config must be registered with the system. (see section Declaring types.) Primitives (int, float, uint32_t, …) are available from the stdtypes module.

To reduce boilerplate validation code in blocks, min and max attributes can be used to define the expected array length of configuration values. For example:

ubx_proto_config_t rnd_config[] = {
    { .name="min_max_config", .type_name = "struct random_config", .min=1, .max=1 },
    { 0 },
};

These specifiers require that this block must be configured with exactly one struct random_config value. Checking will take place before the transition to inactive (i.e. before init).

In fewer cases, configuration takes place in state inactive and must be checked before the transition to active. That can be achieved by defining the config attribute CONFIG_ATTR_CHECKLATE.

Legal values of min and max are summarized below:

min	max	result
0	0	no checking (disabled)
0	1	optional config
1	1	mandatory config
0	CONFIG_LEN_MAX	zero to many
0	undefined	zero to many
N	M	must be between N and M

Declaring ports

Like configurations, ports are described with a { 0 } terminated array of ubx_proto_port_t types:

ubx_proto_port_t rnd_ports[] = {
    { .name="seed", .in_type_name="unsigned int" },
    { .name="rnd", .out_type_name="unsigned int" },
    { 0 },
};

Depending on whether an in_type_name, an out_type_name or both are defined, the port will be an in-, out- or a bidirectional port.

Declaring block meta-data

char rnd_meta[] =
    "{ doc='A random number generator function block',"
    "  realtime=true,"
    "}";

Additional meta-data can be defined as shown above. The following keys are commonly used so far:

• doc: short descriptive documentation of the block

• realtime: is the block real-time safe, i.e. there are no memory allocation / deallocation and other non deterministic function calls in the step function.

Declaring/implementing block hook functions

The following block operations can be implemented to realize the blocks behavior. All are optional.

int rnd_init(ubx_block_t *b);
int rnd_start(ubx_block_t *b);
void rnd_stop(ubx_block_t *b);
void rnd_cleanup(ubx_block_t *b);
void rnd_step(ubx_block_t *b);

These functions will be called according to the microblx block life-cycle finite state machine:

Block lifecycle FSM

They are typically used for the following:

• init: initialize the block, allocate memory, drivers: check if the device exists. Return zero if OK, non-zero otherwise.

• start: become operational, open/enable device, carry out last checks. Cache pointers to ports, apply configurations.

• step: read from ports, compute, write to ports

• stop: stop/close device. stop is often not used.

• cleanup: free all memory, release all resources.

Storing block local state

As multiple instances of a block may exists, NO global variables may be used to store the state of a block. Instead, the ubx_block_t defines a void* private_data pointer which can be used to store local information. Allocate this in the init hook:

b->private_data = calloc(1, sizeof(struct random_info))

if (b->private_data == NULL) {
        ubx_err(b, "Failed to alloc random_info");
        goto out_err;
}

Retrieve and use it in the other hooks:

struct block_info *inf;

inf = (struct random_info*) b->private_data;

Reading configuration values

Configurations can be accessed in a type safe manner using the cfg_getptr_<TYPE> family of functions, which are available for all basic types. For example, the following snippet retrieves a scalar int config and uses a default 47 if unconfigured:

long len;
const int *cfgval;
int value;

if ((len = cfg_getptr_int(b, "myconfig", &cfgval)) < 0)
    goto out_err;

value = (len > 0) ? *cfgval : 47;

Defining type safe configuration accessors for custom types can be achieved using the macros described in section Declaring type safe accessors.

The following example from the random (std_blocks/ubx/random.c) block shows how this is done for struct min_max_config:

def_cfg_getptr_fun(cfg_getptr_random_config, struct random_config)

int rnd_start(ubx_block_t *b)
{
     long len;
     const struct random_config* rndconf;

     /*...*/

     /* get and store min_max_config */
     len = cfg_getptr_random_config(b, "min_max_config", &rndconf);

     if (len < 0) {
             ubx_err(b, "failed to retrieve min_max_config");
             return -1;
     } else if (len == 0) {
             /* set a default */
             inf->min = 0;
             inf->max = INT_MAX;
     } else {
             inf->min = rndconf->min;
             inf->max = rndconf->max;
     }
}

Like the first example, the generated accessor cfg_getptr_random_config returns <0 on error, 0 if unconfigured, or the array length (>0) if configured. If >0, rndconf points to the actual configuration data.

Copy configs or use pointer directly?

In the above example, the configuration values are copied to the internal info struct. This is done to be able to assign defaults should no configuration have been given by the user. If this is not required (e.g. for mandatory configurations), it is perfectly OK to use the pointers retrieved via cfg_getptr… functions directly. The following table summarizes the permitted changes in each block state:

block state	allowed config changes
preinit	resizing and changing values
inactive	changing values
active	no changes allowed

Due to possible resizing in preinit, config ptr and length should be re-retrieved in init.

When to read configuration: init vs start?

It depends: if needed for initalization (e.g. a char array describing which device file to open), then read in init. If it’s not needed in init (e.g. like the random min-max values in the random block example), then read it in start.

This choice affects reconfiguration: in the first case the block has to be reconfigured by a stop, cleanup, init, start sequence, while in the latter case only a stop, start sequence is necessary.

Reading from and writing to ports

Writing to ports can be done using the write_<TYPE> or write_<TYPE>_array functions. For example:

/* writing to a port */
unsigned int val = 1;
write_uint(my_outport, &val);

/* reading from a port */
long len;
int val;

len = read_int(my_inport, &val);

if (len < 0)
       ubx_err(b, "port read failed");
       return -1;
else if (len == 0) {
       /* no data on port */
       return 0;
} else {
       ubx_info(b, "new data: %i", val);
}

...

For more see std_blocks/ramp/ramp.c.

Type safe read/write functions are defined for all basic types and available via the <ubx.h> header. Defining similar functions for custom types can be done using the macros described in Declaring type safe accessors.

Declaring the block

The block aggregates all of the previous declarations into a single data-structure that can then be registered in a microblx module:

ubx_proto_block_t random_comp = {
    .name = "myblocks/random",
    .type = BLOCK_TYPE_COMPUTATION,
    .meta_data = rnd_meta,
    .configs = rnd_config,
    .ports = rnd_ports,

    .init = rnd_init,
    .start = rnd_start,
    .step = rnd_step,
    .cleanup = rnd_cleanup,
};

Declaring types

All types used for configurations or ports must be declared and registered. This is necessary because microblx needs to know the size of the transported data. Moreover, it enables type reflection which is used by logging or the webinterface.

In the random block example, we used a struct random_config, that is defined in types/random_config.h:

struct random_config {
    int min;
    int max;
};

It can be declared as follows:

#include “types/random_config.h”
#include “types/random_config.h.hexarr”
ubx_type_t random_config_type = def_struct_type(struct random_config, &random_config_h);

This fills in a ubx_type_t data structure called random_config_type, which stores information on types. Using this type declaration the struct random_config can then be registered with a node (see “Block and type registration” below).

Supported type system

Microblx registers types as named structs (TYPE_CLASS_STRUCT). The header file associated with each struct is passed verbatim to LuaJIT’s ffi.cdef at runtime. This imposes two rules on type headers:

1. No #include directives — LuaJIT’s ffi.cdef does not run the C preprocessor. Use only C types that the LuaJIT FFI already knows (all standard integer types such as int, unsigned long, float, double, etc. are built-in; int32_t, uint8_t and the rest of the <stdint.h> family are also recognised natively).

2. Keep one registered struct per header file. Auxiliary declarations (enums, helper unions) used only by that struct may appear in the same file. Do not define two independently registered structs in the same header.

The following constructs are all supported.

Plain struct (the common case):

/* types/my_point.h */
struct my_point { double x; double y; double z; };

Named enum used as a struct field:

The enum declaration must appear in the same header as the struct:

/* types/test_with_enum.h */
enum test_color { RED=0, GREEN=1, BLUE=2 };

struct test_with_enum {
    enum test_color col;
    int val;
};

Only one ubx_type_t is registered — for the struct:

#include “types/test_with_enum.h”
#include “types/test_with_enum.h.hexarr”
ubx_type_t t = def_struct_type(struct test_with_enum, &test_with_enum_h);

Enum fields are represented as numbers in Lua. When setting a config value from Lua or a USC model, symbolic string values are accepted and converted automatically by the LuaJIT FFI:

-- numeric and symbolic forms are equivalent:
{ name=”blk”, config = { color = 2 } }
{ name=”blk”, config = { color = “BLUE” } }

Named union used as a struct field:

/* types/test_with_union.h */
union test_variant { int i; float f; };

struct test_with_union {
    union test_variant v;
    unsigned char tag;
};

cdata.tolua converts the union to a Lua table containing all members. A cdata.struct2tab hook (see cdata.struct2tab hooks) can be registered to expose only the active member:

cdata.struct2tab[“union test_variant”] = function(cd)
    return tonumber(cd.i)   -- expose only integer member
end

Anonymous union inside a struct:

Anonymous union fields are promoted to the struct’s namespace by the LuaJIT FFI. cdata.tolua promotes them to the top-level Lua table as well:

/* types/test_with_anon_union.h */
struct test_with_anon_union {
    union {
        int   i;
        float f;
    };
    unsigned char selector;
};

Setting this type in a USC configuration works the same way:

{ name=”blk”, config = { value = { i=42, selector=0 } } }

Anonymous enum field inside a struct:

An anonymous enum used as a field type introduces its constants into the surrounding scope. The field is accessed and set like any other integer:

/* types/test_with_anon_enum.h */
struct test_with_anon_enum {
    enum { KIND_INT=0, KIND_FLOAT=1 } kind;
    int value;
};

As with named enums, symbolic strings are accepted in USC configs:

{ name=”blk”, config = { value = { kind=”KIND_FLOAT”, value=3.14 } } }

cdata.struct2tab hooks

For any named struct or union, a custom Lua→table converter can be installed in the cdata.struct2tab table. The key must include the C type keyword (”struct foo” or ”union bar”):

local cdata = require “cdata”

-- override default struct conversion
cdata.struct2tab[“struct test_with_enum”] = function(cd)
    local names = { [0]=”RED”, [1]=”GREEN”, [2]=”BLUE” }
    return { col = names[tonumber(cd.col)], val = tonumber(cd.val) }
end

-- select only the active union member
cdata.struct2tab[“union test_variant”] = function(cd)
    return tonumber(cd.i)
end

Anonymous structs and unions cannot be hooked because they have no name; register a hook on the containing struct instead.

Declaring type safe accessors

The following macros are available to define type safe accessors for accessing configuration and reading/writing from ports:

def_type_accessors(SUFFIX, TYPENAME)

/* will define the following functions */
long read_SUFFIX(const ubx_port_t* p, TYPENAME* val);
int write_SUFFIX(const ubx_port_t *p, const TYPENAME *val);
long read_SUFFIX_array(const ubx_port_t* p, TYPENAME* val, const int len);
int write_SUFFIX_array(const ubx_port_t* p, const TYPENAME* val, const int len);
long cfg_getptr_SUFFIX(const ubx_block_t *b, const char *cfg_name, const TYPENAME **valptr);

Using these is strongly recommended for most blocks.

Variants:

• def_port_accessors(SUFFIX, TYPENAME) will define the port but not the config accessors.

• def_cfg_getptr_fun(FUNCNAME, TYPENAME) will only define the config accessor

• def_port_writers(FUNCNAME, TYPENAME) and def_port_readers(FUNCNAME, TYPENAME) will only define the port write or read accessors respectively.

What is this .hexarr file

The file types/random_config.h.hexarr contains the contents of the file types/random_config.h converted to an array const char random_config_h [] using the tool tools/ubx-tocarr. This char array is stored in the ubx_type_t private_data field (the third argument to the def_struct_type macro). At runtime, this type model is loaded into the luajit ffi, thereby enabling type reflection features such as logging or changing configuration values via the webinterface. The conversion from .h to .hexarray is done via a simple Makefile rule.

This feature is very useful but optional. If no type reflection is needed, don’t include the .hexarr file and pass NULL as a third argument to def_struct_type.

Block and type registration

So far we have declared blocks and types. To make them known to the system, these need to be registered when the respective module is loaded in a microblx node. This is done in the module init function, which is called when a module is loaded:

1: static int rnd_module_init(ubx_node_t* ni)
2: {
3:        ubx_type_register(nd, &random_config_type);
4:        return ubx_block_register(nd, &random_comp);
5: }
6: UBX_MODULE_INIT(rnd_module_init)

Line 3 and 4 register the type and block respectively. Line 6 tells microblx that rnd_module_init is the module’s init function.

Likewise, the module’s cleanup function should deregister all types and blocks registered in init:

static void rnd_module_cleanup(ubx_node_t *nd)
{
    ubx_type_unregister(nd, "struct random_config");
    ubx_block_unregister(nd, "ubx/random");
}
UBX_MODULE_CLEANUP(rnd_module_cleanup)

Real-time logging

Microblx provides logging infrastructure with loglevels similar to the Linux Kernel. Loglevel can be set on the (global) node level (e.g. by passing it -loglevel N to ubx-launch or be overridden on a per block basis. To do the latter, a block must define and configure a loglevel config of type int. If it is left unconfigured, again the node loglevel will be used.

The following loglevels are supported:

• UBX_LOGLEVEL_EMERG (0) (system unusable)

• UBX_LOGLEVEL_ALERT (1) (immediate action required)

• UBX_LOGLEVEL_CRIT (2) (critical)

• UBX_LOGLEVEL_ERROR (3) (error)

• UBX_LOGLEVEL_WARN (4) (warning conditions)

• UBX_LOGLEVEL_NOTICE (5) (normal but significant)

• UBX_LOGLEVEL_INFO (6) (info message)

• UBX_LOGLEVEL_DEBUG (7) (debug messages)

The following macros are available for logging from within blocks:

ubx_emerg(b, fmt, ...)
ubx_alert(b, fmt, ...)
ubx_crit(b, fmt, ...)
ubx_err(b, fmt, ...)
ubx_warn(b, fmt, ...)
ubx_notice(b, fmt, ...)
ubx_info(b, fmt, ...)
ubx_debug(b, fmt, ...)

Note that ubx_debug will only be logged if UBX_DEBUG is defined in the respective block and otherwise compiled out without any overhead.

To view the log messages, you need to run the ubx-log tool in a separate window.

Important: The maximum total log message length is 127 characters (UBX_LOG_MSG_MAXLEN), so keep messages short or increase the limit for your build.

Note that the old (non-rt) macros ERR, ERR2, MSG and DBG are deprecated and shall not be used anymore.

Outside of the block context, (e.g. in module_init or module_cleanup, you can log with the lowlevel function

ubx_log(int level, ubx_node_t *nd, const char* src, const char* fmt, ...)

/* for example */
ubx_log(UBX_LOGLEVEL_ERROR, ni, __FUNCTION__, "error %u", x);

The ubx core uses the same logger mechanism, but uses the log_info resp. logf_info variants. See libubx/ubx.c for examples.

SPDX License Identifiers

Microblx uses a macro to define module licenses in a form that is both machine readable and available at runtime:

UBX_MODULE_LICENSE_SPDX(MPL-2.0)

To dual-license a block, write:

UBX_MODULE_LICENSE_SPDX(MPL-2.0 BSD-3-Clause)

It is strongly recommended to use this macro. The list of licenses can be found on http://spdx.org/licenses

Generating blocks with ubx_genblock

The ubx-genblock tool generates a microblx block including a Makefile. After this, only the hook functions need to be implemented in the .c file:

Example: generate stubs for a myblock block (see examples/block_model_example.lua for the block generator model).

$ ubx-genblock -d myblock -c /usr/local/share/ubx/examples/blockmodels/block_model_example.lua
    generating myblock/bootstrap
    generating myblock/configure.ac
    generating myblock/Makefile.am
    generating myblock/myblock.h
    generating myblock/myblock.c
    generating myblock/myblock.usc
    generating myblock/types/vector.h
    generating myblock/types/robot_data.h

Run ubx-genblock -h for full options.

The following files are generated:

• bootstrap autoconf bootstrap script

• configure.ac autoconf input file

• Makefile.am automake input file

• myblock.h block interface and module registration code (don’t edit)

• myblock.c module body (edit and implement functions)

• myblock.usc simple microblx system composition file, see below (can be extended)

• types/vector.h sample type (edit and fill in struct body)

• robot_data.h sample type (edit and fill in struct body)

If the command is run again, only the .c file will NOT be regenerated. This can be overridden using the -force option.

Compile the block

$ cd myblock/
$ ./bootstrap
$ ./configure
$ make
$ make install

Launch block using ubx-launch

$ ubx-launch -webgraph -c myblock.usc

Run ubx-launch -h for full options. Browse to http://localhost:8888 to see the running block graph.

Block Interface Guidelines

• use long (signed) for ubx type related lengths and sizes. This is sufficently large and errors can be returned as negative values (example: cfg_getptr_uint32).

• (i)blocks that allow configuring type and length of data to be handled should use the canonical config names type_name and data_len.