co3.mapper module

Used to house useful objects for storage schemas (e.g., SQLAlchemy table definitions). Provides a general interface for mapping from CO3 class names to storage structures for auto-collection and composition.

Example:

mapper = Mapper[sa.Table]()

mapper.attach(
    Type,
    attr_comp=TypeTable,
    coll_comp=CollateTable,
    coll_groups={
        'name': NameConversions
    }
)

Development log

  • Overruled design decision: Mappers were previously designed to map from a specific CO3 hierarchy to a specific Schema. The intention was to allow only related types to be attached to a single schema, at least under a particular Mapper. The type restriction has since been removed, however, as it isn’t particularly well-founded. During collect(), a particular instance collects data from both its attributes and its collation actions. It then repeats the same upward for parent types (part of the same type hierarchy), and down to components (often not part of the same type hierarchy). As such, to fully collect from a type, the Mapper needs to leave registration open to various types, not just those part of the same hierarchy.

class co3.mapper.ComposableMapper(schema, attr_compose_map=None, coll_compose_map=None)[source]

Bases: Mapper, Generic

class design

Heavily debating between multiple possible design approaches here. The main purpose of this subtype is make clear the need for additional compositional mapping details, namely functions that can produce pairwise join conditions for both the attribute tree (vertical traversal) and the collation components (horizontal traversal). Here’s a few remarks:

  • I want the necessary maps to provided/stored outside of compose calls to reduce overhead for downstream callers. It’s awkward to have think about the exact attr-to-attr associations each time you want a type’s associated composition, especially when they don’t change under the same Mapper (i.e., if you have the Mapper reference, the compositional associations should be implicit).

  • The barebones spec here appears to be two pairwise “composer” maps: one for attribute comps, and one for collation comps. For now I think this makes sense as additional init params, but there may later be reason to wrap this up a bit more.

  • Considering the full deprecation for the Composer type, or whether this could be the place where it serves some purpose. Aesthetically, there’s symmetry with the collect and Collector method-type pairing, but that isn’t a good enough reason to justify a separate type here. The difference is that Collector instances actually store type references, whereas the considered Composer type would effectively just be a convenient collection of utility functions. Still possibly useful, but not as clearly justifiable.

  • If a separate Composer type were to be justified here, it would serve as a “reusable connective tissue” for possibly many Mappers with the same kinds of edge-wise relationships. Can think of it like this:

    • Schemas collect up “nodes” (Components). These are explicit storage structures in a DB, and can include some explicit attribute connections (foreign keys), although those are connections made on the DB side.

    • Mappers provide an exoskeleton for a Schema’s nodes. It structures Components into attributes and collation types, and additionally ties them to external CO3 types. The handy analogy here has been that attribute comps connect vertically (in a tree like fashion; point up for parents and down for children), and collation comps point horizontally (or perhaps more aptly, outward; at each node in the attribute tree, you have a “circle” of collation comps that can point to it, and are not involved as formal tree nodes. Can maybe think of these like “ornaments” or bulbs or orbitals).

    • While the Mappers may provide the “bones,” there’s no way to communicate across them. While I might know that one attribute is the “parent” of another, I don’t know why that relationship is there. A Composer, or the composer details to be provided to this class, serve as the “nerves” to be paired with the bone, actually establishing a line of communication. More specifically, the nerves here are attribute-based mappings between pairs of Components, i.e., (generalized) join conditions.

  • Note that, by the above logic, we should then want/need a type to manage the functions provided to attach_many. These functions help automatically characterize the shape of the type skeleton in the same way the proposed Composer wrapper would. In fact, the barebones presentation here is really just the same two function signatures as are expected by that method. The above analogy simply made me ask why the “bones” wouldn’t be reusable if the “nerves” were going to be. So we should perhaps coordinate a decision on this front; if one goes, the other must as well. This may also help me keep it simpler for the time being.

  • One other aspect of a dedicated Composer type (and by the above point, a hypothetical type to aid in attach_many specification) could have some sort of “auto” feature about it. With a clear enough “discovery system,” we could encourage certain kinds of Schemas and components are named and structured. Such an auto-composer could “scan” all components in a provided Schema and attempt to find common attributes across tables that are unlinked (i.e., the reused column names implicit across types in the attribute hierarchy; e.g., File.name -> Note.name), as well as explicit connections which may suggest collation attachment (e.g., note_conversions.name --FK-> Note.name). This, of course, could always be overridden with manual specification, but being aware of some automatic discovery structures could help constrain schema definitions to be more in-line with the CO3 operational model. That all being said, this is a large amount of complexity and should likely be avoided until necessary.

Instance variables

  • type_compose_cache: index for previously computed compositions. This index

    is reset if either attach or attach_many is called to allow possible new type propagation.

__init__(schema, attr_compose_map=None, coll_compose_map=None)[source]
Parameters:

schema (Schema[TypeVar(C, bound= ComposableComponent)]) – Schema object holding the set of components eligible as attachment targets for registered CO3 types

attach(*args, **kwargs)[source]
Parameters:

  • type_ref – CO3 subtype to map to provided storage components

  • attr_comp – storage component for provided type’s canonical attributes

  • coll_comp – storage component for provided type’s default/unnamed collation targets

  • coll_groups – storage components for named collation groups; dict mapping group names to components

attach_many(*args, **kwargs)[source]

Auto-register a set of types to the Mapper’s attached Schema. Associations are made from types to both attribute and collation component names, through attr_name_map and coll_name_map, respectively. Collation targets are inferred through the registered groups in each type.

Parameters:

  • type_ref – reference to CO3 type

  • attr_name_map – function mapping from types/classes to attribute component names in the attached Mapper Schema

  • coll_name_map – function mapping from types/classes & action groups to collation component names in the attached Mapper Schema. None is passed as the action group to retrieve the default collection target.

compose(co3_ref, groups=None, compose_args=None, compose_kwargs=None)[source]

Compose tables up the type hierarchy, and across through action groups to collation components.

Note

Comparing to ORM, this method would likely also still be needed, since it may not be explicitly clear how some JOINs should be handled up the inheritance chain (for components / sa.Relationships, it’s a little easier).

Parameters:

obj – either a CO3 instance or a type reference

class co3.mapper.Mapper(schema)[source]

Bases: Generic

Mapper base class for housing schema components and managing relationships between CO3 types and storage components (of type C).

Mappers are responsible for two primary tasks:

  1. Attaching CO3 types to database Components from within a single schema

  2. Facilitating collection of Component-related insertion data from instances of attached CO3 types

Additionally, the Mapper manages its own Collector and Composer instances. The Collector receives the inserts from .collect() calls, and will subsequently be “dropped off” at an appropriate Database’s Manager to actually perform the requested inserts (hence why we tie Mappers to Schemas one-to-one).

Dev note

The Composer needs reconsideration, or at least its positioning directly in this class. It may be more appropriate to have at the Schema level, or even just dissolved altogether if arbitrary named Components can be attached to schemas.

  • Consider pushing this into a Mapper factory; on init, could check if provided Schema wraps up composable Components or not

__init__(schema)[source]
Parameters:

schema (Schema[TypeVar(C, bound= Component)]) – Schema object holding the set of components eligible as attachment targets for registered CO3 types

attach(type_ref, attr_comp, coll_comp=None, coll_groups=None, strict=True)[source]
Parameters:

  • type_ref (type[CO3]) – CO3 subtype to map to provided storage components

  • attr_comp (Union[str, TypeVar(C, bound= Component)]) – storage component for provided type’s canonical attributes

  • coll_comp (Union[str, TypeVar(C, bound= Component), None]) – storage component for provided type’s default/unnamed collation targets

  • coll_groups (dict[str | None, Union[str, TypeVar(C, bound= Component)]] | None) – storage components for named collation groups; dict mapping group names to components

Return type:

None

attach_many(type_list, attr_name_map, coll_name_map=None, strict=False)[source]

Auto-register a set of types to the Mapper’s attached Schema. Associations are made from types to both attribute and collation component names, through attr_name_map and coll_name_map, respectively. Collation targets are inferred through the registered groups in each type.

Parameters:

  • type_ref – reference to CO3 type

  • attr_name_map (Callable[[type[CO3]], Union[str, TypeVar(C, bound= Component)]]) – function mapping from types/classes to attribute component names in the attached Mapper Schema

  • coll_name_map (Optional[Callable[[type[CO3], str], Union[str, TypeVar(C, bound= Component)]]]) – function mapping from types/classes & action groups to collation component names in the attached Mapper Schema. None is passed as the action group to retrieve the default collection target.

Return type:

None

collect(obj, keys=None, groups=None)[source]

Stages inserts up the inheritance chain, and down through components.

Note

Even with ORM, a method like this would be needed to trace up parent tables and how inserts should be handled for inheritance. ORM would make component inserts a little easier perhaps, since they can be attached as attributes to constructed row objects and a sa.Relationship will handle the rest. Granted, we don’t do a whole lot more here: we just call collect over those components, adding them to the collector session all the same.

Parameters:

  • obj (CO3) – CO3 instance to collect from

  • keys (list[str]) – keys for actions to collect from

  • group – group contexts for the keys to collect from. If None, explicit group contexts registered for the keys will be inferred (but implicit groups will not be detected).

Return type:

list

Returns: collector receipts for staged inserts

get_attr_comp(co3_ref)[source]
Return type:

Optional[TypeVar(C, bound= Component)]

get_coll_comp(co3_ref, group=str | None)[source]
Return type:

Optional[TypeVar(C, bound= Component)]