yayaml package

yayaml: A package providing tools for working with YAML.

Specifically, it makes it simpler to add custom constructors or representers, and adds a whole suite of constructors for basic python functions.

Submodules

yayaml.constructors module

Defines and registers YAML constructors.

scalar_node_to_object(loader: ruamel.yaml.loader.BaseLoader, node: ruamel.yaml.nodes.Node)

Attempts to convert the given scalar node to a null (Python None), a bool, an int, or a float object using the corresponding YAML constructor. If those conversions fail, constructs a scalar (which will typically result in a string being returned).

construct_from_func(loader: ruamel.yaml.loader.BaseLoader, node: ruamel.yaml.nodes.Node, *, func: Callable, unpack: bool = True) → Any

A constructor that constructs a scalar, mapping, or sequence from the given node and subsequently applies the given function on it.

Parameters

• loader – The selected YAML loader

• node – The node from which to construct a Python object

• func (Callable) – The callable to invoke on the resulting

• unpack (bool, optional) – Whether to unpack sequences or mappings into the func call

getenv(loader: ruamel.yaml.loader.BaseLoader, node: ruamel.yaml.nodes.Node)

Retrieves an environment variable by name, optionally with fallback

expression(loader: ruamel.yaml.loader.BaseLoader, node: ruamel.yaml.nodes.Node)

Constructor that evaluates strings of simple mathematical expressions

listgen(loader: ruamel.yaml.loader.BaseLoader, node: ruamel.yaml.nodes.Node)

Constructor for lists, where node can be a mapping or sequence

yayaml.exceptions module

Custom exceptions and exception handling

raise_improved_exception(exc: Exception, *, hints: List[Tuple[Callable[[Exception], bool], str]] = []) → None

Improves the given exception by appending one or multiple hint messages.

The hints argument should be a list of 2-tuples, consisting of a unary matching function, expecting the exception as only argument, and a hint that is part of the new error message.

YAML_ERROR_HINTS: List[Tuple[Callable[[Exception], bool], str]] = [(<function <lambda>>, 'Did you include a space after the YAML tag defined in that line?'), (<function add_constructor.<locals>.<lambda>>, 'Check the expression syntax'), (<function add_constructor.<locals>.<lambda>>, 'Check the expression syntax'), (<function add_constructor.<locals>.<lambda>>, 'Check the expression syntax'), (<function <lambda>>, 'Read the error message above for details about the error location.')]

These are evaluated by raise_improved_exception() and from within load_yml().

Entries are of the form (match function, hint string) and can also be added via add_yaml_error_hint().

add_yaml_error_hint(match_func: Callable[[Exception], bool], hint: str)

Adds an error hint for YAML error messages.

yayaml.io module

Reading and writing YAML files

load_yml(path: str, *, mode: str = 'r', improve_errors: bool = True, _yaml: ruamel.yaml.main.YAML = <ruamel.yaml.main.YAML object>) → Any

Deserializes a YAML file into an object.

Uses the dantro-internal ruamel.yaml.YAML object for loading and thus supports all registered constructors.

Parameters

• path (str) – The path to the YAML file that should be loaded. A ~ in the path will be expanded to the current user’s directory.

• mode (str, optional) – Read mode for the file at path

• improve_errors (bool, optional) – Whether to improve error messages that come from the call to yaml.load. If true, the error message is inspected and hints are appended.

Returns

The result of the data loading. Typically, this will be a dict,

but depending on the structure of the file, it may be some other type, including None.

Return type

Any

write_yml(d: typing.Union[dict, typing.Any], *, path: str, mode: str = 'w', _yaml: ruamel.yaml.main.YAML = <ruamel.yaml.main.YAML object>)

Serialize an object using YAML and store it in a file.

Uses the dantro-internal ruamel.yaml.YAML object for dumping and thus supports all registered representers.

Parameters

• d (dict) – The object to serialize and write to file

• path (str) – The path to write the YAML output to. A ~ in the path will be expanded to the current user’s directory.

• mode (str, optional) – Write mode of the file

yaml_dumps(obj: typing.Any, *, register_classes: tuple = (), _yaml: ruamel.yaml.main.YAML = <ruamel.yaml.main.YAML object>, **dump_params) → str

Serializes the given object using (by default) yayaml’s YAML object.

Parameters

• obj (Any) – The object to dump

• register_classes (tuple, optional) – Additional classes to register

• _yaml (ruamel.yaml.YAML, optional) – Which YAML object to use for dumping

• **dump_params – Dumping parameters, which will be used to set the attributes of the given YAML object.

Returns

The output of serialization

Return type

str

Raises

RepresenterError – On failure to serialize the given object

yaml_dumps_plain(*args, **kwargs)

Like yaml_dumps() but will create a completely new ruamel.yaml.YAML object for dumping, thus not having any of the special constructors and representers registered.

The aim of this function is to provide YAML dumping that is not dependent on any package configuration; all parameters can be passed here.

In other words, this function does _not_ use the yayaml’s YAML object for dumping but each time creates a new dumper with fixed settings. This reduces the chance of interference from elsewhere. Compared to the time needed for serialization in itself, the extra time needed to create the new ruamel.yaml.YAML object and register the classes is negligible.

Hint

To use yayaml’s YAML object, use yaml_dumps() instead.

Note

The new YAML object will not have _any_ additional representers available!

yayaml.representers module

This module implements custom YAML representer functions

represent_by_type(representer: ruamel.yaml.representer.BaseRepresenter, obj: Union[list, tuple, dict, Any], *, tag: str) → ruamel.yaml.nodes.Node

A representer for simple types: sequence-like, mapping-like or scalar

build_representer(simplify_type: Callable[[Any], Union[list, tuple, dict, Any]]) → Callable[[ruamel.yaml.representer.BaseRepresenter, Any, str], ruamel.yaml.nodes.Node]

Builds a representer that

yayaml.tools module

Various tools

eval_simple_math_expr(s: str) → Union[int, float]

Evalautes simple mathematical expressions, given by strings.

Supports: +, -, , *, /, e-X, eX, inf, nan

listgen(*, from_range: Optional[list] = None, unique: bool = False, sort: bool = True, append: Optional[list] = None, remove: Optional[list] = None) → List[int]

Generates a list of integer elements.

Parameters

• from_range (list, optional) – range arguments to use as the basis of the list

• unique (bool, optional) – Whether to ascertain uniqueness of elements

• sort (bool, optional) – Whether to sort the list before returning

• append (list, optional) – Additional elements to append to the list

• remove (list, optional) – Elements to remove all occurrences of

Returns

The generated list

Return type

List[int]

yayaml.yaml module

This module registers various YAML constructors and representers, notably those for ParamSpace and ParamDim.

Furthermore, it defines a shared ruamel.yaml.YAML object that can be imported and used for loading and storing YAML files using the representers and constructors.

yaml_safe: ruamel.yaml.main.YAML = <ruamel.yaml.main.YAML object>

An explicitly ‘safe’ YAML object

yaml_unsafe = <ruamel.yaml.main.YAML object>

An explicitly ‘unsafe’ YAML object

yaml: ruamel.yaml.main.YAML = <ruamel.yaml.main.YAML object>

The default YAML object of yayaml

_REPRESENTERS: Dict[type, Callable[[ruamel.yaml.representer.BaseRepresenter, Any], ruamel.yaml.nodes.Node]] = {<class 'slice'>: functools.partial(<function build_representer.<locals>.representer_func>, tag='!slice'), <class 'range'>: functools.partial(<function build_representer.<locals>.representer_func>, tag='!range')}

All representers that have been added by yayaml

_CONSTRUCTORS: Dict[str, Callable[[ruamel.yaml.loader.BaseLoader, ruamel.yaml.nodes.Node], Any]] = {'!abs': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=False), '!add': functools.partial(<function construct_from_func>, func=<built-in function add>, unpack=True), '!all': functools.partial(<function construct_from_func>, func=<built-in function all>, unpack=False), '!and': functools.partial(<function construct_from_func>, func=<built-in function and_>, unpack=True), '!any': functools.partial(<function construct_from_func>, func=<built-in function any>, unpack=False), '!arange': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=True), '!array': functools.partial(<function construct_from_func>, func=<built-in function array>, unpack=False), '!compute': <function expression>, '!concat': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=True), '!contains': functools.partial(<function construct_from_func>, func=<built-in function contains>, unpack=True), '!deepcopy': functools.partial(<function construct_from_func>, func=<function deepcopy>, unpack=False), '!eq': functools.partial(<function construct_from_func>, func=<built-in function eq>, unpack=True), '!expanduser': functools.partial(<function construct_from_func>, func=<function expanduser>, unpack=False), '!expr': <function expression>, '!expression': <function expression>, '!floordiv': functools.partial(<function construct_from_func>, func=<built-in function floordiv>, unpack=True), '!format': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=True), '!ge': functools.partial(<function construct_from_func>, func=<built-in function ge>, unpack=True), '!getenv': <function getenv>, '!gt': functools.partial(<function construct_from_func>, func=<built-in function gt>, unpack=True), '!int': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=False), '!invert': functools.partial(<function construct_from_func>, func=<built-in function invert>, unpack=True), '!isorted': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=False), '!join': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=True), '!joinpath': functools.partial(<function construct_from_func>, func=<function join>, unpack=True), '!le': functools.partial(<function construct_from_func>, func=<built-in function le>, unpack=True), '!linspace': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=True), '!listgen': <function listgen>, '!logspace': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=True), '!lt': functools.partial(<function construct_from_func>, func=<built-in function lt>, unpack=True), '!max': functools.partial(<function construct_from_func>, func=<built-in function max>, unpack=False), '!min': functools.partial(<function construct_from_func>, func=<built-in function min>, unpack=False), '!mod': functools.partial(<function construct_from_func>, func=<built-in function mod>, unpack=True), '!mul': functools.partial(<function construct_from_func>, func=<built-in function mul>, unpack=True), '!ne': functools.partial(<function construct_from_func>, func=<built-in function ne>, unpack=True), '!negate': functools.partial(<function construct_from_func>, func=<built-in function neg>, unpack=True), '!not': functools.partial(<function construct_from_func>, func=<built-in function not_>, unpack=True), '!or': functools.partial(<function construct_from_func>, func=<built-in function or_>, unpack=True), '!pow': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=True), '!prod': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=False), '!range': functools.partial(<function construct_from_func>, func=<class 'range'>, unpack=True), '!round': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=False), '!slice': functools.partial(<function construct_from_func>, func=<class 'slice'>, unpack=True), '!sorted': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=False), '!split': functools.partial(<function construct_from_func>, func=<function <lambda>>, unpack=True), '!sub': functools.partial(<function construct_from_func>, func=<built-in function sub>, unpack=True), '!sum': functools.partial(<function construct_from_func>, func=<built-in function sum>, unpack=False), '!truediv': functools.partial(<function construct_from_func>, func=<built-in function truediv>, unpack=True), '!xor': functools.partial(<function construct_from_func>, func=<built-in function xor>, unpack=True)}

All constructors that have been added by yayaml

add_representer(t: type, representer: Callable[[ruamel.yaml.representer.BaseRepresenter, Any, str], ruamel.yaml.nodes.Node], *, tag: Optional[str] = None, _yaml: Optional[ruamel.yaml.main.YAML] = None)

Adds a representer function for the given type

add_constructor(tag: str, constructor: Callable[[ruamel.yaml.loader.BaseLoader, ruamel.yaml.nodes.Node], Any], *, aliases: Optional[List[str]] = None, hint: Optional[Union[str, Tuple[Callable[[Exception], bool], str]]] = None, _yaml: Optional[ruamel.yaml.main.YAML] = None)

Adds a constructor function for the given tag and optional aliases.

is_representer(t: type, *, tag: Optional[str] = None, _yaml: Optional[ruamel.yaml.main.YAML] = None)

Decorator to mark a function as the representer for a certain type

is_constructor(tag: str, *, aliases: Optional[List[str]] = None, hint: Optional[Union[str, Tuple[Callable[[Exception], bool], str]]] = None, _yaml: Optional[ruamel.yaml.main.YAML] = None)

Decorator to mark a function as being a constructor for the given tag