Triceps Multithreading Concepts

The multithreading support has solidified to the point where I can start documenting it. The full description will have to wait until I finalize the Perl API but the concepts are pretty much settled.

The idea of the multithreading support in Triceps is to make writing the multithreaded model easier. To make writing the good code easy and writing the bad code hard. But of course you don't have to use it, you can always make your own if you wish (just as you could before now).

Without the further ado, the diagram of a multithreaded Triceps application:

Fig. 1. Triceps application.

The Triceps application is embodied in the class App. It's possible to have multiple Apps in one program.

Each thread has multiple parts to it. First, of course, there is the OS-level (or, technically, library-level, or Perl-level) thread where the code executes. And then there is a class that represents this thread and its place in the App. To reduce the naming conflict, this class is creatively named Triead (pronounced still "thread"). In the discussion I use the word "thread" for both concepts, the OS-level thread and the Triead, and it's usually clear from the context which one I mean. But sometimes it's particularly important to make the distinction, and then I name one or the other explicitly.

The class Triead itself is largely opaque, allowing only a few methods for introspection. But there is a control interface to it, called TrieadOwner. The Triead is visible from the outside, the TrieadOwner object is visible only in the OS thread that owns the Triead. The TrieadOwner manages the thread state and acts as the intermediary in the thread's communications with the App.

The data is passed between the threads through the Nexuses. A Nexus is unidirectional, with data going only one way, however it may have multiple writers and multiple readers. All the readers see the exact same data, with rowops going in the exact same order (well, there will be other policies in the future as well, but for now there is only one policy).

A Nexus passes through the data for multiple labels, very much like an FnReturn does (and indeed there is a special connection between them). A Nexus also allows to export the row types and table types from one thread to another.

A Nexus gets connected to the Trieads to though the Facets. A Facet is a connection point between the Nexus and the Triead. Each Facet is for either reading or writing. And there may be only one Facet between a given Nexus and a given Triead, you can't make multiple connections between them. As a consequence, a thread can't both write and read to the same Nexus, it can do only one thing. This might actually be an overly restrictive limitation and might change in the future but that's how things work now.

Each Nexus also has a direction: either direct ("downwards") or reverse ("upwards").  And yes, the reverse Nexuses allow to build the models with loops. However the loops consisting of only the direct Nexuses are not allowed, nor of only reverse Nexuses. They would mess up the flow control. The proper loops must contain a mix of direct and reverse Nexuses.

The direct Nexuses have a limited queue size and stop the writers when the queue fills up, until the data gets consumed, thus providing the flow control. The reverse Nexuses have an unlimited queue size, which allows to avoid the circular deadlocks.

Normally an App is built once and keeps running in this configuration until it stops. But there is a strong need to have the threads dynamically added and deleted too. For example, if the App running as a server and clients connect to it, each client needs to have its thread(s) added on connection and then deleted when the client disconnects. This is handled through the concept of fragments. There is no Fragment class but when you create a Triead, you can specify a fragment name for it. Then it becomes possible to shut down and dispose the threads in a fragment after the fragment's work is done.

initialization templates

Some of the helper initialization templates have been already shown in the examples, and here finally is the systematic description.

Trices has the common approach of the building of the complex objects by the chained mehod calls (shown on a fictional class Object):

Autoref<Object> o = Object::make()->addOption1(arg)->addOption2(arg);

Here Object::make() is a convenience wrapper for "new Object", because new has an inconvenient priority. The new or make() returns a pointer to the newly constructed object, and then each method in the chain returns the same pointer (this, from its standpoint) to facilitate the chaining.

While the chain executes, the pointer stays a simple pointer, not a reference. So the methods in the chain can't throw any exceptions, or the memory will leak. Instead they collect the error messages in an Errors object that has to be checked afterwards, like:

if (o->getErrors()->hasError()) ...

Note that the object gets a reference to it created first, so that on an error it would be properly destroyed.

The convenience template checkOrThrow() allows to do the check along with the chain, and if an error is found, convert it to an exception:

Autoref<Object> o = checkOrThrow(Object::make()->addOption1(arg)->addOption2(arg));

It does all the right things with the references.

Some objects  need to be initialized after all the options have been set, since it's much easier to check things once and get the interaction of the options right rather than check on every option. And since the initialization might create the references to the object, to get it right, it has to be done after the "main" reference is created.

The template initialize() takes care of it:

Autoref<Object> o = initialize(Object::make()->addOption1(arg)->addOption2(arg));

For some objects the initialization can't fail (nor any other errors can be created by the options). For the others, the error needs to be checked afterwards. The template initializeOrThrow() takes care of both the initialization, the check, and exception throwing on errors:

Autoref<Object> o = initializeOrThrow(Object::make()->addOption1(arg)->addOption2(arg));

That's basically it. All these templates are defined in common/Initialize.h.

perl wrapping for the C++ objects

There is one more item in the C++ part of Triceps that I haven't touched upon yet. It's not a part of the C++ API as such but the connection between the C++ and Perl APIs. You need to bother about it if you want to write more of the components in C++ and export them into Perl.

When exporting the C++ (or any compiled language) API into Perl (or into any scripting language) there are two things to consider:

1. The script must never crash the interpterer. The interpreted program might die but the interpreter itself must never crash. If you've ever deal with wksh, you know how horrible is the debugging of such crashes.

2. Perl has its memory management by reference counting, which needs to be married with the memory management at the C++ level.

The solution to the second problem is fairly straightforward: have an intermediate wrapper structure. Perl has its reference counting for the pointer to this structure.  When you construct a Perl object, return the pointer to a newly allocated instance of this structure. When the Perl reference count gows down to zero, if calls the method DESTROY for this object, and then you destroy this structure.

And inside this structure will be the C++ reference to the actual object. When the wrapper structure gets created, it gets a new reference and when the wrapper structure gets destroyed, it releases this reference.

Here is a small example of how the RowType object gets created and destroyed:


WrapRowType *
Triceps::RowType::new(...)
    CODE:
        RowType::FieldVec fld;

        .....

        Onceref<RowType> rt = new CompactRowType(fld);
        Erref err = rt->getErrors();
        if (!err.isNull() && !err->isEmpty()) {
            setErrMsg("Triceps::RowType::new: " + err->print());
            XSRETURN_UNDEF;
        }

        RETVAL = new WrapRowType(rt);
    OUTPUT:
        RETVAL

void
DESTROY(WrapRowType *self)
    CODE:
        delete self;

Now to the first problem. How does Perl know to call a particular XS method for a object? From the package this object is blessed to. If the package happens to be an XS package, the XS method will be called. However it's entirely possible to re-bless the object to a completely different package. If the user does this, a completely wrong method may be called and will crash when it tries to reach a wrong object at the pointer.


Things get even worse for the other arguments of the methods. For example, the argument "other" here:

int
equals(WrapRowType *self, WrapRowType *other)

Well, Perl lets you provide a code snippet that would check that the object is as expected in the typemap file. But how would that snipped know? Going just by the blessed package is unreliable.


Triceps solves this problem by placing an 8-byte magic code at the front of every wrap object. Each class has its own magic value for this field. 8 bytes allows to have great many unique codes, and is quick to check because it's just one CPU word.


This magic code is defined as:



struct WrapMagic {
    char v_[8]; // 8 bytes to make a single 64-bit comparison
    bool operator!=(const WrapMagic &wm) const
    {
        return (*(int64_t *)v_) != (*(int64_t *)wm.v_);
    }
};


Then the wrapper contains the fields:

    WrapMagic magic_;
    Autoref<Class> ref_; // referenced value

    static WrapMagic classMagic_;

The check is done with:

    // returns true if the magic value is bad
    bool badMagic() const
    {
        return magic_ != magic;
    }

And the typemap entry is:

O_WRAP_OBJECT
    if( sv_isobject($arg) && (SvTYPE(SvRV($arg)) == SVt_PVMG) ) {
        $var = ($type)SvIV((SV*)SvRV( $arg ));
        if ($var == 0 || $var->badMagic()) {
            setErrMsg( \"${Package}::$func_name(): $var has an incorrect magic for $ntype\" );
            XSRETURN_UNDEF;
        }
    } else{
        setErrMsg( \"${Package}::$func_name(): $var is not a blessed SV reference to $ntype\" );
        XSRETURN_UNDEF;
    }

It checks that this is an object, of an XS package, and then that the pointer to the C/C++ object is not NULL, and that the value at this pointer starts with the right magic.

setErrMsg() is the Triceps finction for setting its error code, and will soon be replaced with a Perl error indication.

The repeatable build of the wrapper object is handled in Triceps with a template that you don't need to use directly and a macro that you do use. For example,  the definition for the RowType wrap in wrap/Wrap.h is:

DEFINE_WRAP(RowType);

The second component, the magic definition goes into Wrap.cpp:

WrapMagic magicWrapRowType = { "RowType" };

Just make sure that the string contains no more than 7 characters.

Some objects in Triceps are reached through the special references that know both the object and its type (such as rows and row handles). For them there is a separate macro:

DEFINE_WRAP2(const RowType, Rowref, Row);
DEFINE_WRAP2(Table, Rhref, RowHandle);

The arguments are the type class, the reference class and finally the object class itself.

And there is one more twist: sometimes the object are self-contained but when you use them, you must use them only with a correct parent object. Right now there is only one such class: the Tray must be used with its correct Unit, and the Perl code checks it. In this case the wrapper has the reference to both Tray and the Unit, and is defined as:

DEFINE_WRAP_IDENT(Unit, Tray);

a better unit tracer

I've done a few small improvements to the Unit::Tracer in the C++ code.

The first one, I've moved the message buffer support from Unit::StringTracer into the base class Unit::Tracer. The buffer is used in pretty much any tracer, so it looks like a good idea. And if some subclass doesn't want to use it, it doesn't have to.

So, now in Unit::Tracer is a protected element for the use of subclasses:

Erref buffer_;

And a couple of methods for manipulating this buffer:

virtual Erref getBuffer();
virtual void clearBuffer();

They are virtual to let the subclasses do whatever they please, but the default implementation is exactly as it was in the StringTracer: return the buffer_ element, and put a new clean Errors object into the buffer_ reference.

The next improvement allows to add a custom row printer. Now not only the Perl code but also C++ code can print the rows in tracing (though you have to provide that C++ code that prints the interesting portion of the row or the whole row). This separation of the general trace logic and of the row printing will eventually make it to Perl too , but not yet.

There are two ways to do it. First, there is a virtual method

virtual void printRow(string &res, const RowType *rt, const Row *row);

You can re-define it in your subclass and do the printing. The job of this method is to append the information from the row row of the type rt to the result string res. Append, not replace.

The second way is by providing a pointer to a simple C-style function of the type:

typedef void RowPrinter(string &res, const RowType *rt, const Row *row);

The arguments are exactly the same as for the method. This pointer can be given to the Tracer constructor:

Tracer(RowPrinter *rp = NULL);
StringTracer(bool verbose = false, RowPrinter *rp = NULL);
StringNameTracer(bool verbose = false, RowPrinter *rp = NULL);

The default implementation of the method printRow() simply calls the function at this pointer if it's not NULL. So if you redefine this method in your subclass, the row printer function pointer will stop working.

synchronization with events

The events feel like an easier form of synchronization to use than dealing directly with the condition variables, and for the simple cases they are. However once you try to do something real-sized, it always turns out that you want to do something more while holding that mutex, not just set a flag and sleep (or the other way around, not just wake up) as the internals of an event do. And no, holding a separate mutex and waiting for an event with it doesn't work, because the separate mutex doesn't get unlocked when the thread goes to sleep.

And so far in Triceps always, always, every single time an event had become some handcrafted logic around a mutex and condition variable. There is one event in it now but it's about to be split up and converted.

Perhaps a good solution to make the events more general and more usable would be to split the mutex from the rest of the event logic, make it like the mutex-condvar pair that is sometimes called a monitor (or in the Ptwrap library term, a pmcond). Or, to turn the same idea sideways, make public not only the combined API but also the constituent parts, the mutex and the logic that is protected behind the mutex.

It has also turned out that for the auto-reset events the ability to read the event state without resetting it comes quite handy.

more printf for the errors

I've been using strprintf() repeatedly for the error messages and exceptions, and I've come up with a better way for it.

First, Ive added a var-args version of strprintf(), in common/Strprintf.h:

string vstrprintf(const char *fmt, va_list ap);

You can use it to create strings from other functions taking the printf-like arguments.

Next go the extensions to the Errors class. The consistent theme there is "check if the Errors reference (Erref) is NULL, if it is, allocate a new Errors, and then add a formatted error message to it". So I've added the new methods not to Errors but to Erref. They check if the Erref object is NULL, allocate a new Errors object into it if needed, and then format the arguments. The simplest one is:

void f(const char *fmt, ...);

It adds a simple formatted message, always marked as an error. You use it like this:

Erref e; // initially NULL
...
e.f("a message with integer %d", n);

The message may be multi-line, it will be split appropriately, like in Errors::appendMultiline().

The next one is even more smart:

bool fAppend(Autoref<Errors> clde, const char *fmt, ...);

It first checks that the child errors object is not NULL and contains an error, and if it does then it does through the dance of allocating a new Errors object if needed, appends the formatted message, and the child errors. The message goes before the child errors, unlike the method signature. So you can use it blindly like this to do the right thing:

Autoref<Errors> checkSubObject(int idx);
...
for (int i = 0; i < sz; i++)
  e.fAppend(checkSubObject(i),  "error in the sub-object %d:", i);

Same as before, you can use the multi-line error messages.

Next goes the same thing for Exception:

 static Exception f(const char *fmt, ...);
 static Exception fTrace(const char *fmt, ...);

these are the static factory methods that create an Exception object with the message, and either without and with the stack trace. They are used like

throw Exception::f("a message with integer %d", n);

And the similar methods for construction with the nested errors:

static Exception f(Onceref<Errors> err, const char *fmt, ...);
static Exception fTrace(Onceref<Errors> err, const char *fmt, ...);

Unlike the Erref method, these work unconditionally (since their result is normally used in throw, and it's too late to do anything by that time), so you better make sure in advance that there is a child error. A typical usage would be like this:

try {
  ...
} catch (Exception e) {
  throw Exception(e.getErrors(), "error at stage %d:",  n);
}

Again, in the resulting exception the message goes before the nested errors.