Notes on types

Built-in types of data (i.e. of Datum objects) are kept in datum.py.

A type specifies:

• its name
• whether it is an image subtype (it's tricky to write a method for this, since ImgType is defined after Type)
• whether it is an "internal type" used in the expression evaluator and not for connections (e.g. IdentType, FuncType and NoneType)
• optional serialisation and deserialisation methods taking and returning Datum, which convert to and from JSON-serialisable values - i.e. primitive types, tuples, lists and dicts; no objects.

The Datum class has a type list, which contains singletons of all the type objects. To register a type:

• create a type object
• append to the Datum types list
• if required, register a connector brush with connbrushes.register.
• Deal with binary and unary operators in expr expressions if required (see below).

An example

This code is taken from the example1.py plugin file. This (in part) declares a function which takes two numbers, and produces an object containing the dividend and remainder of those numbers. These are held in a custom object:

# Our data will be an object of class TestObject.

class TestObject:
    def __init__(self, div, rem):
        self.div = div
        self.rem = rem

    def __str__(self):
        return f"({self.div}, {self.rem})"

Now we need to provide the type singleton:

# now the type singleton, which controls how Datum objects which hold TestObjects
# serialise and deserialise themselves (turn themselves into JSON-serialisable
# data and back).

# I'm naming the class with an underscore - the type object will be without this.
class _TestObjectType(Type):
    def __init__(self):
        # just call the superconstructor telling it the name of the type
        # and in this case, that the stringification (the result of __str__ on the
        # value) is short enough to fit into an expr node's graph box.
        super().__init__('testtuple', outputStringShort=True)

    # now we have to write code which converts Datums of this type into
    # stuff which can be converted to JSON and back again. Converting
    # into JSON-serialisable is termed "serialisation" and reconstructing
    # the original Datum object and all its data is "deserialisation".

    def serialise(self, d):
        # how to serialise data of this type: serialise() methods must return
        # a tuple of typename and contents.
        # The contents must be JSON-serialisable, and must contain both the
        # data to be saved and the serialised source information.

        # First convert TestObject to something we can serialise
        serialisedObject = d.val.div, d.val.rem
        # and create the serialised datum of the name and contents
        return self.name, (serialisedObject, d.getSources().serialise())

    def deserialise(self, d, document):
        # given a serialised tuple generated by serialise(), produce a Datum
        # of this type.
        serialisedObject, serialisedSources = d        # first generate the contents
        # deserialise the serialised sources data
        sources = SourceSet.deserialise(serialisedSources, document)
        # then pass to the datum constructor along with the type singleton.
        return Datum(self, serialisedObject, sources) 

We register the type singleton, but keep a reference to the object so we can use it in our own code when we create Datum objects. We also provide a connector brush, so that connections of this type are rendered differently in the graph:

# create the singleton and register it, but keep hold of the variable so
# we can use it to create new Datum objects.
TestObjectType = _TestObjectType()
Datum.registerType(TestObjectType)

# add a brush for the connections in the graph
pcot.connbrushes.register(TestObjectType, QColor("darkMagenta"))

See example1.py for how this new type is used.

Operators

Previously operators were entirely hardwired in utils.ops. This stopped us creating new types. We could define something like binop(self,other) in the type classes, but this wouldn't allow us to add new types as the RHS for operations which have built-in types as the LHS.

The Simplest Thing That Can Possibly Work is a dictionary of operation functions keyed (in the case of binops) by a tuple of types.

So this is how operators work now, relying on two registration processes.

The first is the registration of the operator lexeme (e.g. "*" or "+") and precedence, and an associated function to call. This happens as part of Parser. The function calls a binop() function in the ops module, passing in the operator ID.

The second is the registration of operator ID (e.g. Operator.ADD) and types in the ops module, with an associated function to call. This is often a wrapper function around a lambda: the wrapper knows to unpack (say) image and number data, and the lambda says they should be processed with addition.

Adding a new type with operator semantics

• Create a subclass of datum.Type
• add serialisation methods if required
• call Datum.registerType() with the type
• If required, add a new connector brush with connbrushes.register()
• To use the type, use the Type object with the Datum constructor and Datum.get() method.

Adding operator semantics

• call ops.registerBinop and ops.registerUnop to register functions to perform the required operations. The function should take Datum objects and return a Datum.