Command reference

squirrel scan

Scan and index files and directories.

squirrel scan

usage: squirrel scan [--help] [--loglevel LEVEL] [--progress DEST]
                     [--add PATH [PATH ...]] [--include REGEX]
                     [--exclude REGEX] [--optimistic] [--format FORMAT]
                     [--add-only KINDS] [--persistent NAME] [--update]
                     [--dataset FILE]

Read and cache meta-data of all files in formats understood by Squirrel under selected paths. Subdirectories are recursively traversed and file formats are auto-detected unless a specific format is forced with the --format option. Modification times of files already known to Squirrel are checked by default and re-indexed as needed. To speed up scanning, these checks can be disabled with the --optimistic option. With this option, only new files are indexed during scanning and modifications are handled “last minute” (i.e. just before the actual data (e.g. waveform samples) are requested by the application).

Usually, the contents of files given to Squirrel are made available within the application through a runtime selection which is discarded again when the application quits. Getting the cached meta-data into the runtime selection can be a bottleneck for application startup with large datasets. To speed up startup of Squirrel-based applications, persistent selections created with the --persistent option can be used.

After scanning, information about the current data selection is printed.

General options:

--help, -h

Show this help message and exit.

--loglevel LEVEL

Set logger level. Choices: critical, error, warning, info, debug. Default: info.

--progress DEST

Set how progress status is reported. Choices: terminal, log, off. Default: terminal.

Data collection options:

--add PATH [PATH ...], -a PATH [PATH ...]

Add files and directories with waveforms, metadata and events. Content is indexed and added to the temporary (default) or persistent (see --persistent) data selection.

--include REGEX

Only include files whose paths match the regular expression REGEX. Examples: --include='\.MSEED$' would only match files ending with .MSEED. --include='\.BH[EN]\.' would match paths containing .BHE. or .BHN.. --include='/2011/' would match paths with a subdirectory 2011 in their path hierarchy.

--exclude REGEX

Only include files whose paths do not match the regular expression REGEX. Examples: --exclude='/\.DS_Store/' would exclude anything inside any .DS_Store subdirectory.

--optimistic, -o

Disable checking file modification times for faster startup.

--format FORMAT, -f FORMAT

Assume input files are of given FORMAT. Choices: datacube, mseed, pyrocko_events, pyrocko_stations, sac, stationxml, tdms_idas, virtual, yaml. Default: detect.

--add-only KINDS

Restrict meta-data scanning to given content kinds. KINDS is a comma-separated list of content kinds. Choices: waveform, station, channel, response, event. By default, all content kinds are indexed.

--persistent NAME, -p NAME

Create/use persistent selection with given NAME. Persistent selections can be used to speed up startup of Squirrel-based applications.

--update, -u

Allow adding paths and datasets to existing persistent selection.

--dataset FILE, -d FILE

Add files, directories and remote sources from dataset description file. This option can be repeated to add multiple datasets. Run squirrel template to obtain examples of dataset description files.