squirrel.cache

class ContentCache

Bases: object

Simple memory cache for file contents.

Squirrel manages data in small entities: nuts. Only the meta-data for each nut is stored in the database, content data has to be read from file. This cache helps to speed up data access for typical seismological access patterns.

Content data for stations, channels and instrument responses is small in size but slow to parse so it makes sense to cache these indefinitely once read. Also, it is usually inefficient to read a single station from a station file, so it is better to cache the contents of the complete file even if only one station is requested (it is likely that other stations from that file will be used anyway).

Content data for waveforms is large in size and we usually want to free the memory allocated for them after processing. Typical processing schemes require batches of waveforms to be available together (e.g. cross-correlations between pairs of stations) and there may be overlap between successive batches (e.g. sliding window processing schemes).

This cache implementation uses named accessors and batch window counting for flexible content caching. Loaded contents are held in memory as long as an accessor is holding a reference to it. For each accessor a batch counter is maintained, which starts at 0 and is incremented using calls to advance_accessor(). Content accesses are tracked with calls to get(), which sets a “last access” attribute on the cached item to the current value of the batch counter (each accessor has its own last access attribute on the items it uses). References to items which have not been accessed during the latest batch by the accessor in question are released during advance_accessor(). put() inserts new items into the cache. has() checks if there already is content cached for a given item. To remove all references held by a given accessor, clear_accessor() can be called.

Example usage

For meta-data content to be cached indefinitely, no calls to advance_accessor() or clear_accessor() should be made. For waveform content one would call advance_accessor() after each move of a sliding window or clear_accessor() after each processed event. For a process requiring data from two independent positions of extraction, e.g. for cross-correlations between all possible pairs of a set of events, two separate accessor names could be used.

put(nut)

Insert a new/updated item into cache.

Parameters:nut (Nut) – Content item with attached data object.

get(nut, accessor='default')

Get a content item and track its access.

Parameters:

• nut (Nut) – Content item.
• accessor (str) – Name of accessing consumer. Giving a new name initializes a new accessor.

Returns:

Content data object

has(nut)

Check if item’s content is currently in cache.

Parameters:nut (Nut) – Content item. Returns:bool

advance_accessor(accessor='default')

Increment batch counter of an accessor.

Parameters:accessor (str) – Name of accessing consumer. Giving a new name initializes a new accessor.

clear_accessor(accessor='default')

Clear all references held by an accessor.

Parameters:accessor (str) – Name of accessing consumer.

clear()

Empty the cache.