co3.co3 module¶

CO3 is an abstract base class for scaffolding object hierarchies and managing operations with associated database schemas. It facilitates something like a “lightweight ORM” for classes/tables/states with fixed transformations of interest. The canonical use case is managing hierarchical document relations, format conversions, and syntactical components.

Generic collation syntax:

class Type(CO3):

    @collate
    def group(self, key):
        # disambiguate key
        ...

    @collate('key', groups=['group1', 'group2'])
    def key(self):
        # key-specific logic
        ...

On multi-key attachment

One possible quirk of the current collation registry scheme is the rather black and white nature of key attachment. You either specify a single key, possibly to several groups, or allow any key via passthrough under an implicit group. There’s no explicit “multi-key” pattern to make use of here, be it through “restricted passthrough” (method still parameterized by the key, but only allows keys from a provided list) or just simple duplicated attachment. To demonstrate via the above example:

class Type(CO3):

    @collate(['key1', 'key2'], groups=['group1', 'group2'])
    def keys(self, key):
        # accept key as arg, but can only be 'key1' or 'key2'
        ...

This could be integrated straightforwardly in the existing registration handler, but for the time being, it muddies the waters too much for the convenience it provides. For starters, this isn’t an all too common pattern, and you possibly open up a slippery slope of the allowed key spec for a given method (wildcards/regex patterns? combinatorial O(nm) key (n) group (m) pairs to register?). It can also be handled in a few very simple ways if needed, either via full passthrough with an internal check:

@collate(groups=['group1', 'group2'])
def keys(self, key):
    if key not in ['key1', 'key2']:
        return None

    ...

or with a central handler and separate collation points (at least when the key list is small):

def _handle_supported_keys(self, key):
    # expects only supported keys, e.g., 'key1' and 'key2'
    ...

@collate('key1')
def key1(self):
    self._handle_supported_keys('key1')

@collate('key2')
def key2(self):
    self._handle_supported_keys('key2')

The former scales better and allows general key rejection patterns if needed, while the latter integrates a bit better with the formal collation process, e.g., will throw ValueErrors based on key mismatches automatically.

class co3.co3.CO3[source]¶

Bases: object

Base class supporting the central “COllate, COllect, COmpose” paradigm.

Collate: organize and transform conversion outputs, possibly across class components
Collect: gather core attributes, conversion data, and subcomponents for DB insertion
Compose: construct object-associated DB table references through the class hierarchy

on action groups

Group keys are simply named collections to make it easy for storage components to be attached to action subsets. They do _not_ augment the action registration namespace, meaning the action key should still be unique; the group key is purely auxiliary.

Action methods can also be attached to several groups, in case there is overlapping utility within or across schemas or storage media. In this case, it becomes particularly critical to ensure registered collate methods really are just “gathering results” from possibly heavy-duty operations, rather than performing them when called, so as to reduce wasted computation.

New: collation caching

To help facilitate the common pattern of storing collation results, a collate_cache parameter has been added to store key-group indexed collation results. (Note: now requires explicit superclass instantiation.)

__init__()[source]¶

property attributes¶: Method to define how a subtype’s inserts should be handled under collect for canonical attributes, i.e., inserts to the type’s table.

collate(key, group=None, args=None, kwargs=None)[source]¶: Note

This method is sensitive to group specification. By default, the provided key will be checked against the default None group, even if that key is only attached to non-default groups. Collation actions are unique on key-group pairs, so more specificity is generally required to correctly execute desired actions (otherwise, rely more heavily on the default group).

collation_attributes(key, group)[source]¶

Return “connective” collation component data, possibly dependent on instance-specific attributes and the action arguments. This is typically the auxiliary structure that may be needed to attach to responses from registered collate calls to complete inserts.

Note: this method is primarily used by Mapper.collect(), and is called just prior to collector send-off for collation inserts and injected alongside collation data. Common structure in collation components can make this function easy to define, independent of action group for instance.

property components¶: Method to define how a subtype’s inserts should be handled under collect for constituent components that need handling.

group_registry = {}¶

key_registry = {}¶

class co3.co3.FormatRegistryMeta(name, bases, attrs)[source]¶

Bases: type

Metaclass handling collation registry at the class level.

co3.co3.collate(key, groups=None)[source]¶

Collation decorator for CO3 subtype action registry.

Dynamic decorator; can be used as collate without any arguments, or with all. In the former case, key will be a function, so we check for this.

Usage

Collation registration is the process of exposing various actions for use in hierarchical collection (see Mapper.collect). Collation keys are unique identifiers of a particular action that emits data. Keys can belong to an arbitrary number of groups, which serve as semantically meaningful collections of similar actions. Group assignment also determines the associated collation component to be used as a storage target; the results of actions $K_G$ belonging to group $G$ will all be stored in the attached $G$-component. Specification of key-group relations can be done in a few ways:

Explicit key-group specification: a specific key and associated groups can be provided as arguments to the decorator:

@collate('key', groups=['group1', 'group2'])
def _key(self):
    # key-specific logic
    ...

The registry dictionaries will then have the following items:

key_registry = {
    ...,
    'key': (_key, ['group1', 'group2']),
    ...
}
group_registry = {
    ...,
    'group1': [..., 'key', ...],
    'group2': [..., 'key', ...],
    ...
}

If groups is left unspecified, the key will be attached to the default None group.

Implicit key-group association: in some cases, you may want to support an entire “action class,” and associate any operations under the class to the same storage component. Here we still use the notion of connecting groups to components, but allow the key to be dynamically specified and passed through to the collation method:
```
@collate
def group(self, key):
    # disambiguate key
    ...
```
and in the registries:
```
key_registry = {
    ...,
    None: {..., 'group': group, ...},
    ...
}
group_registry = {
    ...,
    'group': [..., None, ...],
    ...
}
```
A few important notes:
- Implicit key-group specifications attach the group to a single method, whereas in the explicit case, groups can be affiliated with many keys. When explicitly provided, only those exact key values are supported. But in the implicit case, any key is allowed; the group still remains a proxy for the entire action class, but without needing to map from specifically stored key values. That is, the utility of the group remains consistent across implicit and explicit cases, but stores the associations differently.
- The None key, rather than point to a (<method>, <group-list>) tuple, instead points to a dictionary of group-method pairs. When attempting execute a key under a particular group, the group registry indicates whether the key is explicitly supported. If None is present for the group, then key_registry[None][<group-name>] can be used to recover the method implicitly affiliated with the key (along with any other key under the group).
- When any method has been implicitly registered, any key (even when attempting to specify an explicit key) will match that group. This can effectively mean keys are not unique when an implicit group has been registered. There is a protection in place here, however; in methods like CO3.collate and Mapper.collect, an implicit group must be directly named in order for a given key to be considered. That is, when attempting collation outside specific group context, provided keys will only be considered against explicitly registered keys.