imaginglss.utils.npyquery module

npyquery, Query mini-language for numpy arrays.

We reuse the python expression parser to construct the AST. This is inspired by TinyDB, and may have been inspired by other python database implementations as well.

Override Column.apply for customized access to objects (e.g.

Pandas dataframe or imaginglss’s ColumnStore )

Simple manipulation of the AST is allowed. For example, node.assume(a, b) allows one to replace ‘a’ with ‘b’ in the expression.

Examples

>>> d = dtype([
        ('BlackholeMass', 'f4'), 
        ('Position', ('f4', 3))])

>>> data = numpy.zeros(4, dtype=d)
>>> data['BlackholeMass'][:] = numpy.arange(4)
>>> data['Position'][:] = numpy.arange(12).reshape(3, 1)
>>> query  = (Column('BlackholeMass') > 2.0)
>>> query &= (Column('BlackholeMass') > 4.0)
>>> query &= (Column('Position')[2] > 1.0)
>>> print(query.apply(data))

>>> query = query.assume(Column('BlackholeMass'), 1.0)
>>> print(query.apply(data))

class imaginglss.utils.npyquery.AssumptionVisitor(node, literal)

Bases: imaginglss.utils.npyquery.Visitor

visit_node(node)

class imaginglss.utils.npyquery.Column(name)

Bases: imaginglss.utils.npyquery.Node

Represents accessing a column from the data array

apply(array, s)

names

class imaginglss.utils.npyquery.Expr(operator, function, operands)

Bases: imaginglss.utils.npyquery.Node

Represents an expression.

e.g. comparing a column with a number.

An expression can have multiple operands.

>>> for o in expr.operands:
>>>   .....
>>> print expr[0], expr[1]

flatten(operands)

Flattens operands of associative operators.

e.g. (a + b) + (c + d) becomes a + b + c + d

operands

class imaginglss.utils.npyquery.GetItem(obj, index)

Bases: imaginglss.utils.npyquery.Node

Represents accesing an item in a vector column.

Astonishing that pandas can’t do this.

obj

class imaginglss.utils.npyquery.Literal(value)

Bases: imaginglss.utils.npyquery.Node

Represents a literal constant.

imaginglss.utils.npyquery.Max(*args)

imaginglss.utils.npyquery.Min(*args)

class imaginglss.utils.npyquery.Node

Bases: object

A node in the query expression.

Parameters:

array : ndarray or alike

applying the query node to the array and returns items satisfying the array.

T

apply(array)

assume(node, literal)

Replace all subexpression ‘node’ with ‘literal’.

cos()

equals(other)

log()

log10()

max()

min()

names

returns a list of column names used in this node.

This function is recursive

sin()

tan()

class imaginglss.utils.npyquery.QueryVisitor(array, s)

Bases: imaginglss.utils.npyquery.Visitor

visit_column(node)

visit_expr(node)

visit_getitem(node)

visit_literal(node)

visit_transpose(node)

class imaginglss.utils.npyquery.Transpose(node)

Bases: imaginglss.utils.npyquery.Node

Represents a transpose. (.T attribute)

obj

class imaginglss.utils.npyquery.Visitor

Bases: object

reg(attr, type)

visit(node)

visit_node(node)

imaginglss.utils.npyquery.repr_slice(s)

imaginglss.utils.npyquery.test()