CollectionExpression
- class hail.expr.CollectionExpression[source]
Expression of type
tarrayortset>>> a = hl.literal([1, 2, 3, 4, 5])
>>> s3 = hl.literal({'Alice', 'Bob', 'Charlie'})
Attributes
The data type of the expression.
Methods
Returns
Trueif f returnsTruefor every element.Returns
Trueif f returnsTruefor any element.Returns a new collection containing elements where f returns
True.Returns the first element where f returns
True.Map each element of the collection to a new collection, and flatten the results.
Reduces the collection with the given function f, provided the initial value zero.
Group elements into a dict according to a lambda function.
Returns the size of a collection.
Transform each element of a collection.
Returns the size of a collection.
Transform each element of a collection of tuples.
- __eq__(other)
Returns
Trueif the two expressions are equal.Examples
>>> x = hl.literal(5) >>> y = hl.literal(5) >>> z = hl.literal(1)
>>> hl.eval(x == y) True
>>> hl.eval(x == z) False
Notes
This method will fail with an error if the two expressions are not of comparable types.
- Parameters:
other (
Expression) – Expression for equality comparison.- Returns:
BooleanExpression–Trueif the two expressions are equal.
- __ge__(other)
Return self>=value.
- __gt__(other)
Return self>value.
- __le__(other)
Return self<=value.
- __lt__(other)
Return self<value.
- __ne__(other)
Returns
Trueif the two expressions are not equal.Examples
>>> x = hl.literal(5) >>> y = hl.literal(5) >>> z = hl.literal(1)
>>> hl.eval(x != y) False
>>> hl.eval(x != z) True
Notes
This method will fail with an error if the two expressions are not of comparable types.
- Parameters:
other (
Expression) – Expression for inequality comparison.- Returns:
BooleanExpression–Trueif the two expressions are not equal.
- all(f)[source]
Returns
Trueif f returnsTruefor every element.Examples
>>> hl.eval(a.all(lambda x: x < 10)) True
Notes
This method returns
Trueif the collection is empty.- Parameters:
f (function ( (arg) ->
BooleanExpression)) – Function to evaluate for each element of the collection. Must return aBooleanExpression.- Returns:
BooleanExpression. –Trueif f returnsTruefor every element,Falseotherwise.
- any(f)[source]
Returns
Trueif f returnsTruefor any element.Examples
>>> hl.eval(a.any(lambda x: x % 2 == 0)) True
>>> hl.eval(s3.any(lambda x: x[0] == 'D')) False
Notes
This method always returns
Falsefor empty collections.- Parameters:
f (function ( (arg) ->
BooleanExpression)) – Function to evaluate for each element of the collection. Must return aBooleanExpression.- Returns:
BooleanExpression. –Trueif f returnsTruefor any element,Falseotherwise.
- collect(_localize=True)
Collect all records of an expression into a local list.
Examples
Collect all the values from C1:
>>> table1.C1.collect() [2, 2, 10, 11]
Warning
Extremely experimental.
Warning
The list of records may be very large.
- Returns:
- describe(handler=<built-in function print>)
Print information about type, index, and dependencies.
- export(path, delimiter='\t', missing='NA', header=True)
Export a field to a text file.
Examples
>>> small_mt.GT.export('output/gt.tsv') >>> with open('output/gt.tsv', 'r') as f: ... for line in f: ... print(line, end='') locus alleles 0 1 2 3 1:1 ["A","C"] 0/1 0/0 0/1 0/0 1:2 ["A","C"] 1/1 0/1 0/1 0/1 1:3 ["A","C"] 0/0 0/1 0/0 0/0 1:4 ["A","C"] 0/1 1/1 0/1 0/1
>>> small_mt.GT.export('output/gt-no-header.tsv', header=False) >>> with open('output/gt-no-header.tsv', 'r') as f: ... for line in f: ... print(line, end='') 1:1 ["A","C"] 0/1 0/0 0/1 0/0 1:2 ["A","C"] 1/1 0/1 0/1 0/1 1:3 ["A","C"] 0/0 0/1 0/0 0/0 1:4 ["A","C"] 0/1 1/1 0/1 0/1
>>> small_mt.pop.export('output/pops.tsv') >>> with open('output/pops.tsv', 'r') as f: ... for line in f: ... print(line, end='') sample_idx pop 0 1 1 2 2 2 3 2
>>> small_mt.ancestral_af.export('output/ancestral_af.tsv') >>> with open('output/ancestral_af.tsv', 'r') as f: ... for line in f: ... print(line, end='') locus alleles ancestral_af 1:1 ["A","C"] 3.8152e-01 1:2 ["A","C"] 7.0588e-01 1:3 ["A","C"] 4.9991e-01 1:4 ["A","C"] 3.9616e-01
>>> small_mt.bn.export('output/bn.tsv') >>> with open('output/bn.tsv', 'r') as f: ... for line in f: ... print(line, end='') bn {"n_populations":3,"n_samples":4,"n_variants":4,"n_partitions":4,"pop_dist":[1,1,1],"fst":[0.1,0.1,0.1],"mixture":false}
Notes
For entry-indexed expressions, if there is one column key field, the result of calling
str()on that field is used as the column header. Otherwise, each compound column key is converted to JSON and used as a column header. For example:>>> small_mt = small_mt.key_cols_by(s=small_mt.sample_idx, family='fam1') >>> small_mt.GT.export('output/gt-no-header.tsv') >>> with open('output/gt-no-header.tsv', 'r') as f: ... for line in f: ... print(line, end='') locus alleles {"s":0,"family":"fam1"} {"s":1,"family":"fam1"} {"s":2,"family":"fam1"} {"s":3,"family":"fam1"} 1:1 ["A","C"] 0/1 0/0 0/1 0/0 1:2 ["A","C"] 1/1 0/1 0/1 0/1 1:3 ["A","C"] 0/0 0/1 0/0 0/0 1:4 ["A","C"] 0/1 1/1 0/1 0/1
- filter(f)[source]
Returns a new collection containing elements where f returns
True.Examples
>>> hl.eval(a.filter(lambda x: x % 2 == 0)) [2, 4]
>>> hl.eval(s3.filter(lambda x: ~(x[-1] == 'e'))) {'Bob'}
Notes
Returns a same-type expression; evaluated on a
SetExpression, returns aSetExpression. Evaluated on anArrayExpression, returns anArrayExpression.- Parameters:
f (function ( (arg) ->
BooleanExpression)) – Function to evaluate for each element of the collection. Must return aBooleanExpression.- Returns:
CollectionExpression– Expression of the same type as the callee.
- find(f)[source]
Returns the first element where f returns
True.Examples
>>> hl.eval(a.find(lambda x: x ** 2 > 20)) 5
>>> hl.eval(s3.find(lambda x: x[0] == 'D')) None
Notes
If f returns
Falsefor every element, then the result is missing.- Parameters:
f (function ( (arg) ->
BooleanExpression)) – Function to evaluate for each element of the collection. Must return aBooleanExpression.- Returns:
Expression– Expression whose type is the element type of the collection.
- flatmap(f)[source]
Map each element of the collection to a new collection, and flatten the results.
Examples
>>> hl.eval(a.flatmap(lambda x: hl.range(0, x))) [0, 0, 1, 0, 1, 2, 0, 1, 2, 3, 0, 1, 2, 3, 4]
>>> hl.eval(s3.flatmap(lambda x: hl.set(hl.range(0, x.length()).map(lambda i: x[i])))) {'A', 'B', 'C', 'a', 'b', 'c', 'e', 'h', 'i', 'l', 'o', 'r'}
- Parameters:
f (function ( (arg) ->
CollectionExpression)) – Function from the element type of the collection to the type of the collection. For instance, flatmap on aset<str>should take astrand return aset.- Returns:
- fold(f, zero)[source]
Reduces the collection with the given function f, provided the initial value zero.
Examples
>>> a = [0, 1, 2]
>>> hl.eval(hl.fold(lambda i, j: i + j, 0, a)) 3
- Parameters:
f (function ( (
Expression,Expression) ->Expression)) – Function which takes the cumulative value and the next element, and returns a new value.zero (
Expression) – Initial value to pass in as left argument of f.
- Returns:
- group_by(f)[source]
Group elements into a dict according to a lambda function.
Examples
>>> hl.eval(a.group_by(lambda x: x % 2 == 0)) {False: [1, 3, 5], True: [2, 4]}
>>> hl.eval(s3.group_by(lambda x: x.length())) {3: {'Bob'}, 5: {'Alice'}, 7: {'Charlie'}}
- Parameters:
f (function ( (arg) ->
Expression)) – Function to evaluate for each element of the collection to produce a key for the resulting dictionary.- Returns:
DictExpression. – Dictionary keyed by results of f.
- length()[source]
Returns the size of a collection.
Examples
>>> hl.eval(a.length()) 5
>>> hl.eval(s3.length()) 3
- Returns:
Expressionof typetint32– The number of elements in the collection.
- map(f)[source]
Transform each element of a collection.
Examples
>>> hl.eval(a.map(lambda x: x ** 3)) [1.0, 8.0, 27.0, 64.0, 125.0]
>>> hl.eval(s3.map(lambda x: x.length())) {3, 5, 7}
- Parameters:
f (function ( (arg) ->
Expression)) – Function to transform each element of the collection.- Returns:
CollectionExpression. – Collection where each element has been transformed according to f.
- show(n=None, width=None, truncate=None, types=True, handler=None, n_rows=None, n_cols=None)
Print the first few records of the expression to the console.
If the expression refers to a value on a keyed axis of a table or matrix table, then the accompanying keys will be shown along with the records.
Examples
>>> table1.SEX.show() +-------+-----+ | ID | SEX | +-------+-----+ | int32 | str | +-------+-----+ | 1 | "M" | | 2 | "M" | | 3 | "F" | | 4 | "F" | +-------+-----+
>>> hl.literal(123).show() +--------+ | <expr> | +--------+ | int32 | +--------+ | 123 | +--------+
Notes
The output can be passed piped to another output source using the handler argument:
>>> ht.foo.show(handler=lambda x: logging.info(x))
- Parameters:
- size()[source]
Returns the size of a collection.
Examples
>>> hl.eval(a.size()) 5
>>> hl.eval(s3.size()) 3
- Returns:
Expressionof typetint32– The number of elements in the collection.
- starmap(f)[source]
Transform each element of a collection of tuples.
Examples
>>> hl.eval(hl.array([(1, 2), (2, 3)]).starmap(lambda x, y: x+y)) [3, 5]
- Parameters:
f (function ( (*args) ->
Expression)) – Function to transform each element of the collection.- Returns:
CollectionExpression. – Collection where each element has been transformed according to f.
- summarize(handler=None)
Compute and print summary information about the expression.
Danger
This functionality is experimental. It may not be tested as well as other parts of Hail and the interface is subject to change.