All Documentation

Data Structures

Defines the Container base class.

class python.gink.impl.container.Container(database: Database, muid: Muid)

Abstract base class for mutable data types (directories, sequences, etc).

clear(bundler: Bundler | None = None, comment: str | None = None) → Muid

Removes all entries from this container, returning the muid of the clearance.

Note that this will also remove entries that aren’t visible because they’ve been hidden until some future time with something like .remove(…, dest=10.0).

classmethod create(bundler: Bundler | None = None, database: Database | None = None): Creates an instance of the given class, adding immediately if no bundler is provided.

dump(*, as_of: ~datetime.datetime | ~datetime.timedelta | ~datetime.date | int | float | str | None = None, file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>): Dumps the contents of this container to file (default stdout).

abstract dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: return the contents of this container as a string

classmethod get_behavior(): Gets the behavior tag/enum for the particular class.

get_describing(as_of: datetime | timedelta | date | int | float | str | None = None) → Iterable[Container]: Returns the properties and groups associated with this thing.

classmethod get_global_instance(database: Database | None = None)

Gets a proxy to the “magic” global instance of the given class.

For each container type there’s a pre-existing global instance with address Muid(timestamp=-1, medallion=-1, offset=<behavior>). This container type can be written to by any instance, and may be used to coordinate between database instances or just for testing/demo purposes.

classmethod get_medallion_instance(*, medallion=0, database: Database | None = None)

Gets a proxy to the magic personal instance for this container type.

For each combination of medallion and container type, there’s implicitly a pre-existing instance: Muid(timestamp=-1, medallion=<medallion>, offset=<behavior>). This instance should only be written to by the owner of the medallion, and may be used to store local configuration state (though note this info is still visible to other instances). Additionally, ownership metadata such as the username, hostname, and process id will be written to the personal directory for each chain at chain start. This info may then be used for “blame” i.e. to track who made what changes when.

Resets either a specific key or the whole container to a particular past timestamp.

(The optional key argument only makes sense when the container is a directory).

Note that this actually creates new entries to literally “re”-set items. So it’ll still be possible to look at before the reset time and see history. Also, that means that unseen changes made before the reset but not received by this node won’t be overwritten. If you want to ensure that all entries written before the current time are removed (even if they’re not seen yet), then use a clear operation (possibly followed by a reset to get old values).

This function returns the bundler (either passed-in or created on the fly).

set_name(name: str, *, bundler=None, comment=None) → Muid

Sets the name of the container, overwriting any previous name for this container.

Giving multiple things the same name is not recommended.

abstract size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

class python.gink.impl.box.Box(muid: Muid | None = None, database: Database | None = None, arche: bool = False, bundler: Bundler | None = None, contents=None, comment: str | None = None)

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: Dumps the contents of this box to a string.

get(default=None, *, as_of: datetime | timedelta | date | int | float | str | None = None): gets the value in the box, optionally as_of a time

set(value, *, bundler=None, comment=None)

Sets a value in the box, returns the muid address of the entry.

If bundler is specified, then simply adds an entry to that bundler. If no bundler is specified, then creates one just for this entry, sets it’s comment to the comment arg (if set) then adds it to the database.

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

contains the Directory class definition

class python.gink.impl.directory.Directory(*ordered, arche: bool | None = None, bundler: Bundler | None = None, contents=None, muid: Muid | None = None, database=None, comment: str | None = None)

the Gink mutable mapping object

blame(key: str | int | bytes | None = None, as_of: datetime | timedelta | date | int | float | str | None = None) → Dict[str | int | bytes, Attribution]: returns a dictionary mapping keys to who’s responsible for each change

delete(key, *, bundler=None, comment=None)

Removes a value from the mapping, returning the muid address of the change.

If bundler is specified, then simply adds an entry to that bundler. If no bundler is specified, then creates one just for this entry, sets it’s comment to the comment arg (if set) then adds it to the database.

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: Dumps the contents of this directory to a string.

get(key, default=None, *, as_of: datetime | timedelta | date | int | float | str | None = None): gets the value associate with a key, default if missing, optionally as_of a time

has(key: str | int | bytes, *, as_of=None): returns true if the given key exists in the mapping, optionally at specific time

items(*, as_of=None): returns an iterable of key,value pairs, as of the effective time (or now)

keys(*, as_of=None): returns an iterable of all the keys in this directory

log(key: str | int | bytes) → Iterable[Attribution]: Get the history of modifications for a particular key.

pop(key, default=None, *, bundler=None, comment=None)

If key exists in the mapping, returns the corresponding value and removes it.

Otherwise returns default. In the case that the key is found and removed, then the change is added to the bundler (or comitted immedately with comment if no bundler is specified.)

popitem(*, bundler=None, comment=None)

Remove and return a (key, value) tuple, or raises KeyError if empty.

Order is determined by implementation of the store.

set(key: str | int, value, *, bundler=None, comment=None) → Muid

Sets a value in the mapping, returns the muid address of the entry.

If bundler is specified, then simply adds an entry to that bundler. If no bundler is specified, then creates one just for this entry, sets it’s comment to the comment arg (if set) then adds it to the database.

setdefault(key, default=None, *, bundler=None, respect_deletion=False)

Insert key with a value of default if key is not in the directory.

Return the value for key if key is in the directory, else default.

If respect_deletion is set to something truthy then it won’t make any changes if the most recent entry in the directory for the key is a delete entry. In this case it will return whatever has been passed into respect_deletion.

show_blame(as_of: ~datetime.datetime | ~datetime.timedelta | ~datetime.date | int | float | str | None = None, file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>): dumps the blame map to <file> in a human-readable format

show_log(key: str | int | bytes, file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, limit=10): writes the history of modifications to <file> in a human-readable format

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

update(from_what, *, bundler=None, comment=None)

Performs a shallow copy of key/value pairs from the argument.

When from_what hasattr “keys”, then will try: for k in E: D[k] = E[k] otherwise will try: for k, v in E: D[k] = v

values(*, as_of=None): returns a list of values in the directory as of the given time

Contains the Vertex, Verb, and Edge classes (all needed for graph database functionality).

class python.gink.impl.graph.Verb(*, arche=False, muid: Muid | None = None, database: Database | None = None, contents: Iterable[Edge] | None = None)

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: Dump all the edges for this verb.

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

class python.gink.impl.graph.Vertex(*, arche: bool = False, muid: Muid | None = None, bundler: Bundler | None = None, database: Database | None = None)

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: return the contents of this container as a string

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

Contains the key set class definition

class python.gink.impl.key_set.KeySet(arche: bool | None = None, bundler: Bundler | None = None, contents=None, muid: Muid | None = None, database=None, comment: str | None = None)

add(key: str | int | bytes, *, bundler: Bundler | None = None, comment: str | None = None): Adds a specified value to the key set

contains(key: str | int | bytes, as_of: datetime | timedelta | date | int | float | str | None = None): Returns a boolean stating whether the specified key is in the key set

difference(s: Container[str | int | bytes], *, as_of: datetime | timedelta | date | int | float | str | None = None) → Iterable[str | int | bytes]: Returns an iterable of keys in the key set that are not in the specified sets/lists/tuples

difference_update(s: Iterable[str | int | bytes], bundler: Bundler | None = None, comment: str | None = None): Updates the key set, removing elements found in the specified iterables.

discard(key: str | int | bytes, bundler: Bundler | None = None, comment: str | None = None): Deletes a specified entry from the key set

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: return the contents of this container as a string

intersection(s: Iterable[str | int | bytes], *, as_of: datetime | timedelta | date | int | float | str | None = None) → Iterable[str | int | bytes]: Returns an iterable with elements common to the key set and the specified iterables

intersection_update(s: Iterable[str | int | bytes], bundler: Bundler | None = None, comment: str | None = None): Updates the key set, keeping only elements found in the key set and the specified iterables.

isdisjoint(s: Iterable[str | int | bytes], *, as_of: datetime | timedelta | date | int | float | str | None = None) → bool: Returns a boolean stating whether the key set contents overlap with the specified set/list/tuple Sets are disjoint if and only if their intersection is an empty set.

issubset(superset: Container[str | int | bytes], *, as_of: datetime | timedelta | date | int | float | str | None = None) → bool: Returns a Boolean stating whether the key set is a subset of the specified set/list/tuple

issuperset(subset: Iterable[str | int | bytes], *, as_of: datetime | timedelta | date | int | float | str | None = None) → bool: Returns a Boolean stating whether the key set contains the specified set or list of keys

items(*, as_of: datetime | timedelta | date | int | float | str | None = None): returns an iterable of all items in the key set

pop(key: str | int | bytes, bundler: Bundler | None = None, comment: str | None = None)

If key exists in the key set, returns it and removes it.

Otherwise returns default. In the case that the key is found and removed, then the change is added to the bundler (or committed immedately with comment if no bundler is specified.)

remove(key: str | int | bytes, bundler: Bundler | None = None, comment: str | None = None): Deletes a specified entry from the key set, but returns KeyError if not found

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

symmetric_difference(s: Iterable[str | int | bytes], *, as_of: datetime | timedelta | date | int | float | str | None = None) → Set[str | int | bytes]: Returns a new set with elements in either the key set or the specified iterable, but not both.

symmetric_difference_update(s: Iterable[str | int | bytes], bundler: Bundler | None = None, comment: str | None = None): Updates the key set, keeping only elements found in either the key set or the specified set, not both.

union(s: Iterable[str | int | bytes], *, as_of: datetime | timedelta | date | int | float | str | None = None) → Set[str | int | bytes]: Returns a new set with elements from both the key set and the specified set

update(keys: Iterable[str | int | bytes], bundler: Bundler | None = None, comment: str | None = None): Adds multiple specified values to the key set

Contains the pair map class definition

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: return the contents of this container as a string

has(key: Tuple[Vertex, Vertex] | Tuple[Muid, Muid], *, as_of=None): returns true if the given key exists in the mapping, optionally at specific time

items(*, as_of=None): returns an iterable of key,value pairs, as of the effective time (or now)

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

Contains the pair set class definition

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: return the contents of this container as a string

exclude(pair: Tuple[Vertex, Vertex] | Tuple[Muid, Muid], *, bundler: Bundler | None = None, comment: str | None = None): Excludes a pair of Vertexs from the pair set

get_pairs(*, as_of: datetime | timedelta | date | int | float | str | None = None) → Set[Tuple[Muid, Muid]]: Returns a set of muid pairs in the pair set at a given time

include(pair: Tuple[Vertex, Vertex] | Tuple[Muid, Muid], *, bundler: Bundler | None = None, comment: str | None = None): Includes a pair of Vertexs in the pair set

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

Contains the Property Container class.

delete(describing: Container | Edge, *, bundler=None, comment=None) → Muid: Removes the value (if any) of this property on object pointed to by describing.

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: Dumps the contents of this property to a string.

get(describing: Container | Edge, default: str | int | float | datetime | bytes | bool | list | tuple | dict | None | Container = None, *, as_of: datetime | timedelta | date | int | float | str | None = None) → str | int | float | datetime | bytes | bool | list | tuple | dict | None | Container: Gets the value of the property on the object it’s describing, optionally in the past.

Sets the value of the property on the particular object addressed by describing.

Overwrites the value of this property on this object if previously set. Returns the muid of the new entry.

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

update(from_what, *, bundler=None, comment=None)

Performs a shallow copy of key/value pairs from the argument.

When from_what hasattr “keys”, then will try: for k in E: D[k] = E[k] otherwise will try: for k, v in E: D[k] = v

Contains the Group Container class.

class python.gink.impl.group.Group(*, contents: Set[Muid | Container] | None = None, muid: Muid | None = None, database=None)

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: Dumps the contents of this group to a string.

get_members(*, as_of: datetime | timedelta | date | int | float | str | None = None) → Set[Container]: Returns pairs of (muid, contents) for the sequence at the given time.

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: returns the number of elements contained

Append obect to the end of the queue.

If expiry is set, the added entry will be removed at the specified time.

Returns the ((position-ts, entry-muid), value) at the specified index.

Index my be negative, in which case starts looking at the end. Raises IndexError if not present.

dumps(as_of: datetime | timedelta | date | int | float | str | None = None) → str: return the contents of this container as a string

Adds all of the items in iterable at once to this sequence.

expiries, if present, may be either a single expiry to be applied to all new entries, or a iterable of expiries of the same length as the data

Since all items will be appended to the sequence in the same transaction, they will all have the same timestamp, and so it won’t be possible to move anything between them.

returns the bundler (either passed or created on the fly)

Return the first index of the value at the given time (or now).

Raises a ValueError if the value isn’t present and raise_if_missing is True, otherwise just returns None.

Inserts thing before index.

The resulting entry expires at expiry time if specified, which must be in the future.

If no bundler is passed, applies the changes immediately, with comment. Otherwise just appends the necessary changes to the passed bundler.

returns the muid of the entry

items(*, as_of: datetime | timedelta | date | int | float | str | None = None) → Iterable: Returns pairs of (muid, contents) for the sequence at the given time.

(Re)move and return an item at index (default last).

If nothing exists at the specified index will raise an IndexError. If bundler is specified, simply adds the change to that, otherwise applies it. If comment is specified and no bundler then will make change with that comment.

If dest is specified, it may be a time to hid the entry until, or a time in the past to reposition the entry to (the list is ordered by timestamps).

Remove first occurance of value.

Raises ValueError if the value is not present.

size(*, as_of: datetime | timedelta | date | int | float | str | None = None) → int: Tells the size at the specified as_of time.

Removes or moves an entry by muid.

muid: what to move bundler: what to add this change to comment: make an immediate change with this comment dest: new location in the list or time in the future; integer values are interpreted to be before the given index if positive or after the given index if negative

returns: the muid of the change

Stores

Contains AbstractStore class.

class python.gink.impl.abstract_store.AbstractStore

abstract base class for the gink data store

Stores both the bundles received and the contents of those bundles unpacked so that you can examine entries, container definitions, etc.

Warning! Since data stores are viewed as part of the internal implementation, this interface may change at any time without warning on a minor version change.

abstract apply_bundle(bundle: BundleWrapper | bytes, callback: Callable[[BundleWrapper], None] | None = None, claim_chain: bool = False) → bool

Tries to add data from a particular bundle to this store.

Returns true if the data is actually added, false if data already exists, and will throw an exception in the case of an invalid extension.

close(): Safely releases resources.

abstract find_chain(medallion: int, timestamp: int) → Chain: Find the chain with the matching medallion and the first chain start prior to the timestamp argument passed in.

get_bundle_infos(limit_to: Mapping[Chain, int | float] | None = None) → List[BundleInfo]: Gets a list of bundle infos; mostly for testing.

abstract get_by_describing(desc: Muid, as_of: int = -1) → Iterable[FoundEntry]: Returns all the containers (properties) that describe desc.

abstract get_by_name(name, as_of: int = -1) → Iterable[FoundContainer]: Returns info about all things with the given name.

abstract get_chain_tracker(limit_to: Mapping[Chain, int | float] | None = None) → ChainTracker: Returns a tracker showing what this store has at the time this function is called.

abstract get_comment(*, medallion: int, timestamp: int) → str | None: Gets the comment associated with a particular bundle, if stored.

abstract get_container(container: Muid) → Container | None: Gets the container definition associated with a particular address.

abstract get_edge_entries(*, as_of: int, verb: Muid | None = None, source: Muid | None = None, target: Muid | None = None) → Iterable[FoundEntry]: Returns all the edge entries with specified verb and/or subject and/or object.

abstract get_entry(muid: Muid) → Entry | None: Return the entry builder for a entry if it’s visible in the store.

abstract get_entry_by_key(container: Muid, key: str | int | bytes | Muid | None | Tuple[Muid, Muid], as_of: int) → FoundEntry | None: Gets the most recent entry for a given key at as_of

abstract get_identity(chain: Chain, lock: Lock | None = None, /) → str: The comment on the first bundle of each chain identifies who or what is responsible for it.

abstract get_keyed_entries(container: Muid, behavior: int, as_of: int) → Iterable[FoundEntry]: Gets all active entries for a given container as of the given time.

abstract get_last(chain: Chain) → BundleInfo: Returns metadata for the last bundle in a specified chain.

get_one(cls, index: int = -1): gets one instance of the given class

abstract get_ordered_entries(container: Muid, as_of: int, limit: int | None = None, offset: int = 0, desc: bool = False) → Iterable[PositionedEntry]: Get data for Sequence and Registry data types.

abstract get_positioned_entry(entry: Muid, as_of: int = -1) → PositionedEntry | None: Returns the position and contents of an entry, if available, at as_of time.

abstract get_reset_changes(to_time: int, container: Muid | None, user_key: str | int | bytes | None, recursive=False) → Iterable[Change]

Generates reset entries that will change things back to how they were at given time.

If muid isn’t specified, generates reset entries for all keyed objects in store. If muid is specified, generates reset entries for that object, for the specified key if present, otherwise for all keys.

If needed_only is set to True (the default), then no entry will be generated for a particular muid,key pair if the current value matches the value at the specified time. If needed_only is set to False, generates new entries for everything.

If recursive is set to true, then will go and recursively update all entries in child objects that were referenced at to_time.

abstract get_some(cls, last_index: int | None = None) → Iterable

Gets several indexes of the given class.

Starts counting from the end if last_index is negative.

cls may be one of: BundleBuilder, EntryBuilder, MovementBuilder, or one of the key classes: BundleInfo, EntryStorageKey, MovementKey

Used by the database class to show a log of entries.

abstract list_containers() → Iterable[Tuple[Muid, Container]]

Gets the address and definition of each regular container.

Does not include the instance/medallion containers or the global containers.

maybe_reuse_chain(identity: str, callback: Callable[[BundleWrapper], None] | None = None) → BundleInfo | None: Tries to find a chain for reuse. The callback is used for refresh.

refresh(callback: Callable[[BundleWrapper], None] | None = None) → int

Checks the source file for bundles that haven’t come from this process and calls the callback.

Intended to allow the process to send bundles to peers and otherwise get the model in line with the file.

Returns the number of transactions processed.

Contains the LmdbStore class.

class python.gink.impl.lmdb_store.LmdbStore(file_path: str | bytes | Path | None = None, reset=False, retain_bundles=True, retain_entries=True, map_size: int = 1073741824)

apply_bundle(bundle: BundleWrapper | bytes, callback: Callable[[BundleWrapper], None] | None = None, claim_chain: bool = False) → bool

Tries to add data from a particular bundle to this store.

Returns true if the data is actually added, false if data already exists, and will throw an exception in the case of an invalid extension.

close(): Safely releases resources.

find_chain(medallion: int, timestamp: int) → Chain: Find the chain with the matching medallion and the first chain start prior to the timestamp argument passed in.

get_bundles(callback: Callable[[BundleWrapper], None], *, limit_to: Mapping[Chain, int | float] | None = None, **_)

Calls callback for all bunles stored, limited to those designed by the limit_to (if present).

Calls the callback with each bundle currently in the store.

Calls in order received by this store, which may not correspond to the bundle creation times. But we still expect dependency order to be respected, that is if B1 references objects from B0, then B0 should come before B1.

This is done callback style because we don’t want to leave dangling transactions in the store, which could easily happen if we offered up an iterator interface instead.

If the limit_to[chain] is x, then return all entries in that chain with timestamp <= x.

The peer_has data can be used to optimize what the store is sending to only what the peer needs, but it can be ignored, and it’s up to the callback to drop unneeded bundles.

get_by_describing(desc: Muid, as_of: int = -1) → Iterable[FoundEntry]: Returns all the containers (properties) that describe desc.

get_by_name(name, as_of: int = -1) → Iterable[FoundContainer]: Returns info about all things with the given name.

get_chain_tracker(limit_to: Mapping[Chain, int | float] | None = None) → ChainTracker: Returns a tracker showing what this store has at the time this function is called.

get_comment(*, medallion: int, timestamp: int) → str | None: Gets the comment associated with a particular bundle, if stored.

get_container(container: Muid) → Container | None: Gets the container definition associated with a particular address.

get_edge_entries(*, as_of: int, verb: Muid | None = None, source: Muid | None = None, target: Muid | None = None) → Iterable[FoundEntry]: Returns all the edge entries with specified verb and/or subject and/or object.

get_entry(muid: Muid) → Entry | None: Return the entry builder for a entry if it’s visible in the store.

Gets a single entry (or none if nothing in the database matches).

When “key” is None, assumes that the container is a box and returns the most recent entry for “container” written before the as_of time.

When “key” is a UserKey (i.e. str or int) then assumes that “container” is a directory, so grabs the latest value written for that key by the given time.

When “key” is a Muid, assumes that “container” is a queue and that the “key” is the muid for the desired entry.

get_identity(chain: Chain, trxn: Transaction | None = None, /) → str: The comment on the first bundle of each chain identifies who or what is responsible for it.

get_keyed_entries(container: Muid, behavior: int, as_of: int) → Iterable[FoundEntry]: gets all the active entries in a direcotry as of a particular time

get_last(chain: Chain) → BundleInfo: Returns metadata for the last bundle in a specified chain.

get_ordered_entries(container: Muid, as_of: int, limit: int | None = None, offset: int = 0, desc: bool = False) → Iterable[PositionedEntry]: Get data for Sequence and Registry data types.

get_positioned_entry(entry: Muid, as_of: int = -1) → PositionedEntry | None: Returns the position and contents of an entry, if available, at as_of time.

get_reset_changes(to_time: int, container: Muid | None, user_key: str | int | bytes | None, recursive=True) → Iterable[Change]

Generates reset entries that will change things back to how they were at given time.

If muid isn’t specified, generates reset entries for all keyed objects in store. If muid is specified, generates reset entries for that object, for the specified key if present, otherwise for all keys.

If needed_only is set to True (the default), then no entry will be generated for a particular muid,key pair if the current value matches the value at the specified time. If needed_only is set to False, generates new entries for everything.

If recursive is set to true, then will go and recursively update all entries in child objects that were referenced at to_time.

get_some(cls, last_index: int | None = None): gets several instance of the given class

list_containers() → Iterable[Tuple[Muid, Container]]

Gets the address and definition of each regular container.

Does not include the instance/medallion containers or the global containers.

refresh(callback: Callable[[BundleWrapper], None] | None = None) → int

Checks the source file for bundles that haven’t come from this process and calls the callback.

Intended to allow the process to send bundles to peers and otherwise get the model in line with the file.

Returns the number of transactions processed.

Contains the MemoryStore class, and implementation of the AbstractStore interface.

class python.gink.impl.memory_store.MemoryStore

Stores the data for a Gink database in memory.

(Primarily for use in testing and to be used as a base clase for log-backed store.)

apply_bundle(bundle: BundleWrapper | bytes, callback: Callable[[BundleWrapper], None] | None = None, claim_chain: bool = False) → bool

Tries to add data from a particular bundle to this store.

Returns true if the data is actually added, false if data already exists, and will throw an exception in the case of an invalid extension.

find_chain(medallion: int, timestamp: int) → Chain: Find the chain with the matching medallion and the first chain start prior to the timestamp argument passed in.

get_bundles(callback: Callable[[BundleWrapper], None], *, limit_to: Mapping[Chain, int | float] | None = None, **_)

Calls callback for all bunles stored, limited to those designed by the limit_to (if present).

Calls the callback with each bundle currently in the store.

Calls in order received by this store, which may not correspond to the bundle creation times. But we still expect dependency order to be respected, that is if B1 references objects from B0, then B0 should come before B1.

This is done callback style because we don’t want to leave dangling transactions in the store, which could easily happen if we offered up an iterator interface instead.

If the limit_to[chain] is x, then return all entries in that chain with timestamp <= x.

The peer_has data can be used to optimize what the store is sending to only what the peer needs, but it can be ignored, and it’s up to the callback to drop unneeded bundles.

get_by_describing(desc: Muid, as_of: int = -1): Returns all the containers (properties) that describe desc.

get_by_name(name, as_of: int = -1): Returns info about all things with the given name.

get_chain_tracker(limit_to: Mapping[Chain, int | float] | None = None) → ChainTracker: Returns a tracker showing what this store has at the time this function is called.

get_comment(*, medallion: int, timestamp: int) → str | None: Gets the comment associated with a particular bundle, if stored.

get_container(container: Muid) → Container | None: Gets the container definition associated with a particular address.

get_edge_entries(*, as_of: int, verb: Muid | None = None, source: Muid | None = None, target: Muid | None = None) → Iterable[FoundEntry]: Returns all the edge entries with specified verb and/or subject and/or object.

get_entry(muid: Muid) → Entry | None: Return the entry builder for a entry if it’s visible in the store.

get_entry_by_key(container: Muid, key: str | int | bytes | Muid | None | Tuple[Muid, Muid], as_of: int) → FoundEntry | None: Gets the most recent entry for a given key at as_of

get_identity(chain: Chain, lock: bool | None = None, /) → str: The comment on the first bundle of each chain identifies who or what is responsible for it.

get_keyed_entries(container: Muid, behavior: int, as_of: int) → Iterable[FoundEntry]: Gets all active entries for a given container as of the given time.

get_last(chain: Chain) → BundleInfo: Returns metadata for the last bundle in a specified chain.

get_ordered_entries(container: Muid, as_of: int, limit: int | None = None, offset: int = 0, desc: bool = False) → Iterable[PositionedEntry]: Get data for Sequence and Registry data types.

get_positioned_entry(entry: Muid, as_of: int = -1) → PositionedEntry | None: Returns the position and contents of an entry, if available, at as_of time.

get_reset_changes(to_time: int, container: Muid | None, user_key: str | int | bytes | None, recursive=False) → Iterable[Change]

Generates reset entries that will change things back to how they were at given time.

If muid isn’t specified, generates reset entries for all keyed objects in store. If muid is specified, generates reset entries for that object, for the specified key if present, otherwise for all keys.

If needed_only is set to True (the default), then no entry will be generated for a particular muid,key pair if the current value matches the value at the specified time. If needed_only is set to False, generates new entries for everything.

If recursive is set to true, then will go and recursively update all entries in child objects that were referenced at to_time.

get_some(cls, last_index: int | None = None)

Gets several indexes of the given class.

Starts counting from the end if last_index is negative.

cls may be one of: BundleBuilder, EntryBuilder, MovementBuilder, or one of the key classes: BundleInfo, EntryStorageKey, MovementKey

Used by the database class to show a log of entries.

list_containers() → Iterable[Tuple[Muid, Container]]

Gets the address and definition of each regular container.

Does not include the instance/medallion containers or the global containers.

implementation of the LogBackedStore class

class python.gink.impl.log_backed_store.LogBackedStore(filepath: Path | str, *, exclusive=False, reset=False)

A Store backed by a simple append-only file.

apply_bundle(bundle: BundleWrapper | bytes, callback: Callable[[BundleWrapper], None] | None = None, claim_chain: bool = False) → bool

Tries to add data from a particular bundle to this store.

Returns true if the data is actually added, false if data already exists, and will throw an exception in the case of an invalid extension.

close(): Closes the underlying file.

Connections

Contains the Peer class that manages a connection to another gink instance.

class python.gink.impl.connection.Connection(host: str | None = None, port: int | None = None, socket: socket | None = None)

Manages a connection to another gink database.

Eventually there will be two subclasses: one to manage websocket connections, and another subclass to manage raw socket connections.

abstract close(reason=None): End the connection and release resources.

fileno(): Return the file descriptor of the underlying socket.

is_closed() → bool: a way to check if the connection is still active

abstract receive() → Iterable[SyncMessage]: receive a (possibly empty) series of encoded SyncMessages from a peer.

abstract send(sync_message: SyncMessage): Send an encoded SyncMessage to a peer.

contains the Listener class that listens on a port for incomming connections

class python.gink.impl.listener.Listener(connection_class=<class 'python.gink.impl.websocket_connection.WebsocketConnection'>, ip_addr: str = '', port: int = 8080)

Listens on a port for incoming connections.

accept(greeting: SyncMessage) → Connection: Method to call when the underlying socket is “ready”.

fileno(): Gives the file descriptor for use in socket.select and similar.

Contains the WsPeer class to manage a connection to a websocket (gink) peer.

Manages the connection to one peer via a websocket.

Set is_client to indicate that the provided socket is a client connection. If there’s no socket provided then one will be established, and is_client is implied.

close(reason=None): End the connection and release resources.

receive() → Iterable[SyncMessage]: receive a (possibly empty) series of encoded SyncMessages from a peer.

send(sync_message: SyncMessage) → int: Send an encoded SyncMessage to a peer.

Other Modules

Contains the Attribution class.

class python.gink.impl.attribution.Attribution(timestamp: int, medallion: int, identity: str, abstract: str | None = None): An object that encapsulates everything about who’s responsible for a bundle.

Contains the BundleInfo class.

class python.gink.impl.bundle_info.BundleInfo(*, builder: Header | None = None, encoded: bytes = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', **kwargs)

Metadata about a particular change set relevant for syncing.

as_acknowledgement() → SyncMessage: convert to an ack message that can be sent to a peer

static from_ack(sync_message: SyncMessage): reverse of as_ack

static from_bytes(data: bytes) → BundleInfo: the opposite of __bytes__

get_chain() → Chain: Gets a Chain tuple saying which chain this change set came from.

Defines the ChainTracker class.

class python.gink.impl.chain_tracker.ChainTracker(sync_message: SyncMessage | None = None)

Keep track of what data a particular instance has.

has(what: Muid | BundleInfo) → bool: Reports if the instance tracked by this object has the given data.

mark_as_having(bundle_info: BundleInfo): Indicates has everything along the chain in bundle_info up to its timestamp.

to_greeting_message() → SyncMessage: Constructs a SyncMessage containing a Greeting with the tracked data. The entries will be sorted in [medallion, chain_start] order.

Utility functions for encoding and decoding values, keys, and other binary data.

Generally the things in this file are intended to help in the implementation, and not be visible to users of the Gink library. They are NOT considered part of the public API and can change at any time without a corresponding increase in the major revision number.

class python.gink.impl.coding.LocationKey(entry_muid: Muid, placement: Muid)

Key used in the locations table to track the current location of entries.

entry_muid: Muid: Alias for field number 0

static from_bytes(data: bytes): inverse of __bytes__

placement: Muid: Alias for field number 1

just a class to serialize / deserialize keys used to store entries

container: Muid: Alias for field number 0

expiry: int | None: Alias for field number 3

static from_builder(builder: Entry, new_info: BundleInfo, offset: int): Create an EntryStorageKey from an Entry itself, plus address information.

static from_bytes(data: bytes, using: int | bytes | Entry = 4): creates an entry key from its binary format, using either the entry(bytes) or behavior

get_placed_time() → int

Gets the time that a specific entry key was inserted into the database.

This is a little weird because queue entries are sorted by time by default, but that time can overridden by explicity encoding a position. But even though we know now where we want the entry to be, we still need to know when it was placed there in order to let users ask what the order previously was.

get_queue_position() → int: Pulls out the effective timestamp (ordering position) from the middle_key.

middle: str | int | bytes | QueueMiddleKey | Muid | None | Tuple[Muid, Muid]: Alias for field number 1

placer: Muid: Alias for field number 2

replace_time(timestamp: int): create a entry key that can be used for seeking before the given time

class python.gink.impl.coding.PlacementBuilderPair(placement: Placement, builder: EntryBuilder)

Parsed entry data.

builder: Entry: Alias for field number 1

placement: Placement: Alias for field number 0

class python.gink.impl.coding.QueueMiddleKey(effective_time: MuTimestamp)

Used to order non-keyed entries by timestamp and modification change.

effective_time: int: Alias for field number 0

static from_bytes(data: bytes): inverse of __bytes__

class python.gink.impl.coding.RemovalKey(container: Muid, removing: Muid, movement: Muid)

Key used in the removals table to track soft-deletes of entries.

container: Muid: Alias for field number 0

static from_bytes(data: bytes): inverse of __bytes__

movement: Muid: Alias for field number 2

removing: Muid: Alias for field number 1

creates an entry that will delete the given key from the container

I’m allowing a null key in the argument then barfing if it’s null inside in part because it results in an easier to use API.

Determines what a container “contains” in a given entry.

The full entry storage pair is required because if it points to something that pointer might be relative to the entry address.

python.gink.impl.coding.decode_key(from_what: Entry | Key | bytes) → str | int | bytes | None: extracts the key from a proto entry

python.gink.impl.coding.decode_muts(data: bytes, _q_struct=<_struct.Struct object>) → int | None

unpacks 8 bytes of data into a MuTimestamp by assuming big-endian encoding

Treats 0 as “None” and -1 as “integer infinity” (i.e. highest unsigned 64 bit number)

python.gink.impl.coding.decode_value(value_builder: Value) → str | int | float | datetime | bytes | bool | list | tuple | dict | None: decodes a protobuf value into a python value.

python.gink.impl.coding.encode_key(key: str | int | bytes, builder: Key | None = None) → Key: Encodes a valid key (int or str or bytes) into a protobuf Value.

python.gink.impl.coding.encode_muts(number: int | float | None, _q_struct=<_struct.Struct object>) → bytes: packs a microsecond timestamp into a big-endian integer, with None=>0 and Inf=>-1

python.gink.impl.coding.encode_value(value: str | int | float | datetime | bytes | bool | list | tuple | dict | None, value_builder: Value | None = None) → Value: encodes a python value (number, string, etc.) into a protobuf builder

python.gink.impl.coding.entries_equiv(pair1: PlacementBuilderPair, pair2: PlacementBuilderPair) → bool

Checks the contained value/pointee/whatever to see if the entries are equiv.

Used to see if the effective value in a container is the same even if it has a new entry.

python.gink.impl.coding.normalize_entry_builder(entry_builder: Entry, entry_muid: Muid): Make all relative muid references absolute muid refereces within an entry.

python.gink.impl.coding.serialize(thing) → bytes: Converts a protobuf builder or a timestamp into binary data.

python.gink.impl.coding.wrap_change(builder: Entry) → Change: A simple utility function to create a change and then copy the provided entry into it.

contains the Database class

class python.gink.impl.database.Database(store: AbstractStore | str | None = None, identity='docs@build-24422000-project-1027556-gink', web_server=None, web_server_addr: tuple = ('localhost', 8081))

A class that mediates user interaction with a datastore and peers.

bundle(bundler: Bundler) → BundleInfo: seals bundler and adds the resulting bundle to the local store

connect_to(target: str): initiate a connection to another gink instance

dump(*, include_global_containers=True, as_of: ~datetime.datetime | ~datetime.timedelta | ~datetime.date | int | float | str | None = None, file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>): writes the contents of the database to file

get_attribution(timestamp: int, medallion: int, *_) → Attribution

Takes a timestamp and medallion and figures out who/what to blame the changes on.

After the timestamp and medallion it will ignore other ordered arguments, so that it can be used via get_attribution(*muid).

get_by_name(name: str, as_of: datetime | timedelta | date | int | float | str | None = None) → List: Returns all containers of the given type with the given name.

get_chain() → Chain | None: gets the chain this database is appending to (or None if it hasn’t started writing yet)

get_container(muid: Muid, *, container_builder: Container | None = None, behavior: int | None = None): Gets a pre-existing container associated with a particular muid

get_store() → AbstractStore: returns the store managed by this database

log(limit: int | None = -10, *, include_starts=False) → Iterable[Attribution]: Gets a list of attributions representing all bundles stored by the db.

Resets the database to a specific point in time.

Note that it literally just “re”-sets everything in one big bundle to the values that existed at that time, so you can always go and look at the state of the database beforehand.

translates an abstract time into a real timestamp

date and datetime behave as you might expect (turned into unix time)

integers and floats that look like timestamps or microsecond timestamps are treated as such.

small integers are treated as “right before the <index> bundle”

run(until: datetime | timedelta | date | int | float | str | None = None, console: SelectableConsole | None = None, connect_to: list[str] | None = None): Waits for activity on ports then exchanges data with peers. Optionally connects to other database instances.

show_log(limit: int | None = -10, *, include_starts=False, file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>): Just prints the log to stdout in a human-readable format.

start_listening(ip_addr='', port: str | int = '8080')

Listen for incoming connections on the given port.

Note that you’ll still need to call “run” to actually accept those connections.

class python.gink.impl.deferred.Deferred(offset: int, bundler)

Version of a muid that references a bundle.

We need a custom subclass here because we want to return something that can be used as a muid, but we don’t have the timestamp and medallion set until the bundle has been sealed. This class allows us to return an address object that will give “None” when asked for timestamp/medallion before the bundle has been sealed, and the appropriate values after sealing.

defines a dummy class

class python.gink.impl.dummy.Dummy: a placeholder class

A bunch of helper functions for interacting with lmdb databases.

python.gink.impl.lmdb_utilities.count_items(cursor) → int

Counts the number of items in the table that the cursor is associated with.

Primarily intended to be a debugging utility.

python.gink.impl.lmdb_utilities.to_last_with_prefix(cursor, prefix, suffix=None, boundary=None) → bytes | None

Positions cursor on the last entry with prefix, optionally before boundary or prefix+suffix

Returns the key under the cursor when something is found, None otherwise. If no match is found, the new position of the cursor is undefined.

contains the Muid class (basically a way to represent global addresses)

class python.gink.impl.muid.Muid(timestamp: MuTimestamp, medallion: Medallion, offset: int)

Defines a global address of an object in the Gink system.

classmethod create(context, builder: ~proto.muid_pb2.Muid | ~python.gink.impl.dummy.Dummy = <python.gink.impl.dummy.Dummy object>, offset: int | None = None)

Creates a muid.

The context argument should either be a BundleInfo

static from_bytes(data: bytes): does the inverse of bytes(muid)

static from_str(hexed: str): the inverse of str(muid)

medallion: int: Alias for field number 1

offset: int: Alias for field number 2

put_into(builder: Muid): Puts the data from this muid into the builder.

timestamp: int: Alias for field number 0

a couple of NamedTuple definitions, primarily for internal usage

class python.gink.impl.tuples.Chain(medallion: Medallion, chain_start: MuTimestamp)

Pair of numbers to identify a blockchain in gink.

chain_start: int: Alias for field number 1

medallion: int: Alias for field number 0

class python.gink.impl.tuples.FoundContainer(address, builder)

address: Muid: Alias for field number 0

builder: Container: Alias for field number 1

class python.gink.impl.tuples.FoundEntry(address: Muid, builder: EntryBuilder)

Entry information returned by the store for keyed containers.

The address is necessary because some muids in the proto may be relative.

address: Muid: Alias for field number 0

builder: Entry: Alias for field number 1

class python.gink.impl.tuples.PositionedEntry(position: MuTimestamp, positioner: Muid, entry_muid: Muid, builder: EntryBuilder)

represents data in a positioned data structure like Sequence

builder: Entry: Alias for field number 3

entry_muid: Muid: Alias for field number 2

position: int: Alias for field number 0

positioner: Muid: Alias for field number 1

class python.gink.impl.tuples.SequenceKey(position: MuTimestamp, entry_muid: Muid)

How data is ordered in a Sequence

entry_muid: Muid: Alias for field number 1

position: int: Alias for field number 0

Various types classes for use throughout the codebase.

class python.gink.impl.typedefs.Deletion: Used internally to indicate that a key/value assocation has been removed.

class python.gink.impl.typedefs.Inclusion: Used to indicate adding something to a set or group.