util

Utility functions for Pyrocko.

exception URLErrorSSL(urlerror)

urlopen_ssl_check(*args, **kwargs)

progressbar_module()

setup_logging(programname='pyrocko', levelname='warning')

Initialize logging.

Parameters:

• programname – program name to be written in log
• levelname – string indicating the logging level (‘debug’, ‘info’, ‘warning’, ‘error’, ‘critical’)

This function is called at startup by most pyrocko programs to set up a consistent logging format. This is simply a shortcut to a call to logging.basicConfig().

data_file(fn)

exception DownloadError

exception PathExists

download_file(url, fpath, username=None, password=None, status_callback=None, **kwargs)

download_dir(url, fpath, username=None, password=None, status_callback=None, **kwargs)

class Stopwatch

Simple stopwatch to measure elapsed wall clock time.

Usage:

s = Stopwatch()
time.sleep(1)
print s()
time.sleep(1)
print s()

wrap(text, line_length=80)

Paragraph and list-aware wrapping of text.

class BetterHelpFormatter(*args, **kwargs)

format_option(option)

From IndentedHelpFormatter but using a different wrap method.

format_description(description)

progressbar(label, maxval)

progress_beg(label)

Notify user that an operation has started.

Parameters:label – name of the operation

To be used in conjuction with progress_end().

progress_end(label='')

Notify user that an operation has ended.

Parameters:label – name of the operation

To be used in conjuction with progress_beg().

exception ArangeError

arange2(start, stop, step, dtype=<class 'float'>, epsilon=1e-06, error='raise')

Return evenly spaced numbers over a specified interval.

Like numpy.arange() but returning floating point numbers by default and with defined behaviour when stepsize is inconsistent with interval bounds. It is considered inconsistent if the difference between the closest multiple of step and stop is larger than epsilon * step. Inconsistencies are handled according to the error parameter. If it is set to 'raise' an exception of type ArangeError is raised. If it is set to 'round', 'floor', or 'ceil', stop is silently changed to the closest, the next smaller, or next larger multiple of step, respectively.

polylinefit(x, y, n_or_xnodes)

Fit piece-wise linear function to data.

Parameters:

• x,y – arrays with coordinates of data
• n_or_xnodes – int, number of segments or x coordinates of polyline

Returns:

(xnodes, ynodes, rms_error) arrays with coordinates of polyline, root-mean-square error

plf_integrate_piecewise(x_edges, x, y)

Calculate definite integral of piece-wise linear function on intervals.

Use trapezoidal rule to calculate definite integral of a piece-wise linear function for a series of consecutive intervals. x_edges and x must be sorted.

Parameters:

• x_edges – array with edges of the intervals
• x,y – arrays with coordinates of piece-wise linear function’s control points

class GlobalVars

reuse_store = {}

decitab_nmax = 0

decitab = {}

decimate_fir_coeffs = {}

decimate_iir_coeffs = {}

re_frac = None

decimate_coeffs(q, n=None, ftype='iir')

decimate(x, q, n=None, ftype='iir', zi=None, ioff=0)

Downsample the signal x by an integer factor q, using an order n filter

By default, an order 8 Chebyshev type I filter is used or a 30 point FIR filter with hamming window if ftype is ‘fir’.

Parameters:

• x – the signal to be downsampled (1D NumPy array)
• q – the downsampling factor
• n – order of the filter (1 less than the length of the filter for a ‘fir’ filter)
• ftype – type of the filter; can be ‘iir’ or ‘fir’

Returns:

the downsampled signal (1D NumPy array)

exception UnavailableDecimation

Exception raised by decitab() for unavailable decimation factors.

gcd(a, b, epsilon=1e-07)

Greatest common divisor.

lcm(a, b)

Least common multiple.

mk_decitab(nmax=100)

Make table with decimation sequences.

Decimation from one sampling rate to a lower one is achieved by a successive application of decimation() with small integer downsampling factors (because using large downampling factors can make the decimation unstable or slow). This function sets up a table with downsample sequences for factors up to nmax.

zfmt(n)

get_working_system_time_range()

Check time range supported by the systems’s time conversion functions.

Returns system time stamps of start of year of first/last fully supported year span. If this is before 1900 or after 2100, return first/last century which is fully supported.

Returns:(tmin, tmax, year_min, year_max)

is_working_time(t)

julian_day_of_year(timestamp)

Get the day number after the 1st of January of year in timestamp.

Returns:day number as int

day_start(timestamp)

Get beginning of day for any point in time.

Parameters:timestamp – time instant as system timestamp (in seconds) Returns:instant of day start as system timestamp

month_start(timestamp)

Get beginning of month for any point in time.

Parameters:timestamp – time instant as system timestamp (in seconds) Returns:instant of month start as system timestamp

year_start(timestamp)

Get beginning of year for any point in time.

Parameters:timestamp – time instant as system timestamp (in seconds) Returns:instant of year start as system timestamp

iter_days(tmin, tmax)

Yields begin and end of days until given time span is covered.

Parameters:tmin,tmax – input time span Yields:tuples with (begin, end) of days as system timestamps

iter_months(tmin, tmax)

Yields begin and end of months until given time span is covered.

Parameters:tmin,tmax – input time span Yields:tuples with (begin, end) of months as system timestamps

iter_years(tmin, tmax)

Yields begin and end of years until given time span is covered.

Parameters:tmin,tmax – input time span Yields:tuples with (begin, end) of years as system timestamps

decitab(n)

Get integer decimation sequence for given downampling factor.

Parameters:n – target decimation factor Returns:tuple with downsampling sequence

ctimegm(s, format='%Y-%m-%d %H:%M:%S')

Convert string representing UTC time to system time.

Parameters:

• s – string to be interpreted
• format – format string passed to strptime()

Returns:

system time stamp

Interpretes string with format '%Y-%m-%d %H:%M:%S', using strptime.

Note

This function is to be replaced by str_to_time().

gmctime(t, format='%Y-%m-%d %H:%M:%S')

Get string representation from system time, UTC.

Produces string with format '%Y-%m-%d %H:%M:%S', using strftime.

Note

This function is to be repaced by time_to_str().

gmctime_v(t, format='%a, %d %b %Y %H:%M:%S')

Get string representation from system time, UTC. Same as gmctime() but with a more verbose default format.

Note

This function is to be replaced by time_to_str().

gmctime_fn(t, format='%Y-%m-%d_%H-%M-%S')

Get string representation from system time, UTC. Same as gmctime() but with a default usable in filenames.

Note

This function is to be replaced by time_to_str().

exception TimeStrError

exception FractionalSecondsMissing

Exception raised by str_to_time() when the given string lacks fractional seconds.

exception FractionalSecondsWrongNumberOfDigits

Exception raised by str_to_time() when the given string has an incorrect number of digits in the fractional seconds part.

str_to_time(s, format='%Y-%m-%d %H:%M:%S.OPTFRAC')

Convert string representing UTC time to floating point system time.

Parameters:

• s – string representing UTC time
• format – time string format

Returns:

system time stamp as floating point value

Uses the semantics of time.strptime() but allows for fractional seconds. If the format ends with '.FRAC', anything after a dot is interpreted as fractional seconds. If the format ends with '.OPTFRAC', the fractional part, including the dot is made optional. The latter has the consequence, that the time strings and the format may not contain any other dots. If the format ends with '.xFRAC' where x is 1, 2, or 3, it is ensured, that exactly that number of digits are present in the fractional seconds.

stt(s, format='%Y-%m-%d %H:%M:%S.OPTFRAC')

Convert string representing UTC time to floating point system time.

Parameters:

• s – string representing UTC time
• format – time string format

Returns:

system time stamp as floating point value

Uses the semantics of time.strptime() but allows for fractional seconds. If the format ends with '.FRAC', anything after a dot is interpreted as fractional seconds. If the format ends with '.OPTFRAC', the fractional part, including the dot is made optional. The latter has the consequence, that the time strings and the format may not contain any other dots. If the format ends with '.xFRAC' where x is 1, 2, or 3, it is ensured, that exactly that number of digits are present in the fractional seconds.

time_to_str(t, format='%Y-%m-%d %H:%M:%S.3FRAC')

Get string representation for floating point system time.

Parameters:

• t – floating point system time
• format – time string format

Returns:

string representing UTC time

Uses the semantics of time.strftime() but additionally allows for fractional seconds. If format contains '.xFRAC', where x is a digit between 1 and 9, this is replaced with the fractional part of t with x digits precision.

tts(t, format='%Y-%m-%d %H:%M:%S.3FRAC')

Get string representation for floating point system time.

Parameters:

• t – floating point system time
• format – time string format

Returns:

string representing UTC time

Uses the semantics of time.strftime() but additionally allows for fractional seconds. If format contains '.xFRAC', where x is a digit between 1 and 9, this is replaced with the fractional part of t with x digits precision.

mystrftime(fmt=None, tt=None, milliseconds=0)

gmtime_x(timestamp)

plural_s(n)

ensuredirs(dst)

Create all intermediate path components for a target path.

Parameters:dst – target path

The leaf part of the target path is not created (use ensuredir() if a the target path is a directory to be created).

ensuredir(dst)

Create directory and all intermediate path components to it as needed.

Parameters:dst – directory name

Nothing is done if the given target already exists.

reuse(x)

Get unique instance of an object.

Parameters:x – hashable object Returns:reference to x or an equivalent object

Cache object x in a global dict for reuse, or if x already is in that dict, return a reference to it.

deuse(x)

class Anon(**dict)

Dict-to-object utility.

Any given arguments are stored as attributes.

Example:

a = Anon(x=1, y=2)
print a.x, a.y

select_files(paths, selector=None, regex=None, show_progress=True)

Recursively select files.

Parameters:

• paths – entry path names
• selector – callback for conditional inclusion
• regex – pattern for conditional inclusion
• show_progress – if True, indicate start and stop of processing

Returns:

list of path names

Recursively finds all files under given entry points paths. If parameter regex is a regular expression, only files with matching path names are included. If additionally parameter selector is given a callback function, only files for which the callback returns True are included. The callback should take a single argument. The callback is called with a single argument, an object, having as attributes, any named groups given in regex.

Examples

To find all files ending in '.mseed' or '.msd':

select_files(paths,
    regex=r'\.(mseed|msd)$')

To find all files ending with '$Year.$DayOfYear', having set 2009 for the year:

select_files(paths,
    regex=r'(?P<year>\d\d\d\d)\.(?P<doy>\d\d\d)$',
    selector=(lambda x: int(x.year) == 2009))

base36encode(number, alphabet='0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ')

Convert positive integer to a base36 string.

base36decode(number)

Decode base36 endcoded positive integer.

exception UnpackError

Exception raised when unpack_fixed() encounters an error.

unpack_fixed(format, line, *callargs)

Unpack fixed format string, as produced by many fortran codes.

Parameters:

• format – format specification
• line – string to be processed
• callargs – callbacks for callback fields in the format

The format is described by a string of comma-separated fields. Each field is defined by a character for the field type followed by the field width. A questionmark may be appended to the field description to allow the argument to be optional (The data string is then allowed to be filled with blanks and None is returned in this case).

The following field types are available:

Type	Description
A	string (full field width is extracted)
a	string (whitespace at the beginning and the end is removed)
i	integer value
f	floating point value
@	special type, a callback must be given for the conversion
x	special field type to skip parts of the string

match_nslc(patterns, nslc)

Match network-station-location-channel code against pattern or list of patterns.

Parameters:

• patterns – pattern or list of patterns
• nslc – tuple with (network, station, location, channel) as strings

Returns:

True if the pattern matches or if any of the given patterns match; or False.

The patterns may contain shell-style wildcards: *, ?, [seq], [!seq].

Example:

match_nslc('*.HAM3.*.BH?', ('GR', 'HAM3', '', 'BHZ'))   # -> True

match_nslcs(patterns, nslcs)

Get network-station-location-channel codes that match given pattern or any of several given patterns.

Parameters:

• patterns – pattern or list of patterns
• nslcs – list of (network, station, location, channel) tuples

See also match_nslc()

exception SoleError

Exception raised by objects of type Sole, when an concurrent instance is running.

class Sole(pid_path)

Use POSIX advisory file locking to ensure that only a single instance of a program is running.

Parameters:pid_path – path to lockfile to be used

Usage:

from pyrocko.util import Sole, SoleError, setup_logging
import os

setup_logging('my_program')

pid_path =  os.path.join(os.environ['HOME'], '.my_program_lock')
try:
    sole = Sole(pid_path)

except SoleError, e:
    logger.fatal( str(e) )
    sys.exit(1)

escapequotes(s)

class TableWriter(f)

Write table of space separated values to a file.

Parameters:f – file like object

Strings containing spaces are quoted on output.

writerow(row, minfieldwidths=None)

Write one row of values to underlying file.

Parameters:

• row – iterable of values
• minfieldwidths – minimum field widths for the values

Each value in in row is converted to a string and optionally padded with blanks. The resulting strings are output separated with blanks. If any values given are strings and if they contain whitespace, they are quoted with single quotes, and any internal single quotes are backslash-escaped.

class TableReader(f)

Read table of space separated values from a file.

Parameters:f – file-like object

This uses Pythons shlex module to tokenize lines. Should deal correctly with quoted strings.

readrow()

Read one row from the underlying file, tokenize it with shlex.

Returns:tokenized line as a list of strings.

gform(number, significant_digits=3)

Pretty print floating point numbers.

Align floating point numbers at the decimal dot.

|  -d.dde+xxx|
|  -d.dde+xx |
|-ddd.       |
| -dd.d      |
|  -d.dd     |
|  -0.ddd    |
|  -0.0ddd   |
|  -0.00ddd  |
|  -d.dde-xx |
|  -d.dde-xxx|
|         nan|

The formatted string has length significant_digits * 2 + 6.

human_bytesize(value)

pf_is_old(fn)

pf_upgrade(fn)

read_leap_seconds(tzfile='/usr/share/zoneinfo/right/UTC')

Extract leap second information from tzdata.

Based on example at http://stackoverflow.com/questions/19332902/ extract-historic-leap-seconds-from-tzdata

See also ‘man 5 tzfile’.

exception LeapSecondsError

exception LeapSecondsOutdated

exception InvalidLeapSecondsFile

parse_leap_seconds_list(fn)

read_leap_seconds2()

gps_utc_offset(t_utc)

Time offset t_gps - t_utc for a given t_utc.

utc_gps_offset(t_gps)

Time offset t_utc - t_gps for a given t_gps.

make_iload_family(iload_fh, doc_fmt='FMT', doc_yielded_objects='FMT')

exception Inconsistency

consistency_check(list_of_tuples, message='values differ:')

Check for inconsistencies.

Given a list of tuples, check that all tuple elements except for first one match. E.g. [('STA.N', 55.3, 103.2), ('STA.E', 55.3, 103.2)] would be valid because the coordinates at the two channels are the same.

class defaultzerodict

mostfrequent(x)

consistency_merge(list_of_tuples, message='values differ:', error='raise', merge=<function mostfrequent>)

parse_md(f)

mpl_show(plt)