Updated script that can be controled by Nodejs web app

This commit is contained in:
mac OS
2024-11-25 12:24:18 +07:00
parent c440eda1f4
commit 8b0ab2bd3a
8662 changed files with 1803808 additions and 34 deletions

View File

@ -0,0 +1,135 @@
.. -*- rest -*-
==================================================
API changes in the new masked array implementation
==================================================
Masked arrays are subclasses of ndarray
---------------------------------------
Contrary to the original implementation, masked arrays are now regular
ndarrays::
>>> x = masked_array([1,2,3],mask=[0,0,1])
>>> print isinstance(x, numpy.ndarray)
True
``_data`` returns a view of the masked array
--------------------------------------------
Masked arrays are composed of a ``_data`` part and a ``_mask``. Accessing the
``_data`` part will return a regular ndarray or any of its subclass, depending
on the initial data::
>>> x = masked_array(numpy.matrix([[1,2],[3,4]]),mask=[[0,0],[0,1]])
>>> print x._data
[[1 2]
[3 4]]
>>> print type(x._data)
<class 'numpy.matrixlib.defmatrix.matrix'>
In practice, ``_data`` is implemented as a property, not as an attribute.
Therefore, you cannot access it directly, and some simple tests such as the
following one will fail::
>>>x._data is x._data
False
``filled(x)`` can return a subclass of ndarray
----------------------------------------------
The function ``filled(a)`` returns an array of the same type as ``a._data``::
>>> x = masked_array(numpy.matrix([[1,2],[3,4]]),mask=[[0,0],[0,1]])
>>> y = filled(x)
>>> print type(y)
<class 'numpy.matrixlib.defmatrix.matrix'>
>>> print y
matrix([[ 1, 2],
[ 3, 999999]])
``put``, ``putmask`` behave like their ndarray counterparts
-----------------------------------------------------------
Previously, ``putmask`` was used like this::
mask = [False,True,True]
x = array([1,4,7],mask=mask)
putmask(x,mask,[3])
which translated to::
x[~mask] = [3]
(Note that a ``True``-value in a mask suppresses a value.)
In other words, the mask had the same length as ``x``, whereas
``values`` had ``sum(~mask)`` elements.
Now, the behaviour is similar to that of ``ndarray.putmask``, where
the mask and the values are both the same length as ``x``, i.e.
::
putmask(x,mask,[3,0,0])
``fill_value`` is a property
----------------------------
``fill_value`` is no longer a method, but a property::
>>> print x.fill_value
999999
``cumsum`` and ``cumprod`` ignore missing values
------------------------------------------------
Missing values are assumed to be the identity element, i.e. 0 for
``cumsum`` and 1 for ``cumprod``::
>>> x = N.ma.array([1,2,3,4],mask=[False,True,False,False])
>>> print x
[1 -- 3 4]
>>> print x.cumsum()
[1 -- 4 8]
>> print x.cumprod()
[1 -- 3 12]
``bool(x)`` raises a ValueError
-------------------------------
Masked arrays now behave like regular ``ndarrays``, in that they cannot be
converted to booleans:
::
>>> x = N.ma.array([1,2,3])
>>> bool(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
==================================
New features (non exhaustive list)
==================================
``mr_``
-------
``mr_`` mimics the behavior of ``r_`` for masked arrays::
>>> np.ma.mr_[3,4,5]
masked_array(data = [3 4 5],
mask = False,
fill_value=999999)
``anom``
--------
The ``anom`` method returns the deviations from the average (anomalies).

View File

@ -0,0 +1,24 @@
* Copyright (c) 2006, University of Georgia and Pierre G.F. Gerard-Marchant
* All rights reserved.
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the University of Georgia nor the
* names of its contributors may be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY
* DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
* ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@ -0,0 +1,236 @@
==================================
A guide to masked arrays in NumPy
==================================
.. Contents::
See http://www.scipy.org/scipy/numpy/wiki/MaskedArray (dead link)
for updates of this document.
History
-------
As a regular user of MaskedArray, I (Pierre G.F. Gerard-Marchant) became
increasingly frustrated with the subclassing of masked arrays (even if
I can only blame my inexperience). I needed to develop a class of arrays
that could store some additional information along with numerical values,
while keeping the possibility for missing data (picture storing a series
of dates along with measurements, what would later become the `TimeSeries
Scikit <http://projects.scipy.org/scipy/scikits/wiki/TimeSeries>`__
(dead link).
I started to implement such a class, but then quickly realized that
any additional information disappeared when processing these subarrays
(for example, adding a constant value to a subarray would erase its
dates). I ended up writing the equivalent of *numpy.core.ma* for my
particular class, ufuncs included. Everything went fine until I needed to
subclass my new class, when more problems showed up: some attributes of
the new subclass were lost during processing. I identified the culprit as
MaskedArray, which returns masked ndarrays when I expected masked
arrays of my class. I was preparing myself to rewrite *numpy.core.ma*
when I forced myself to learn how to subclass ndarrays. As I became more
familiar with the *__new__* and *__array_finalize__* methods,
I started to wonder why masked arrays were objects, and not ndarrays,
and whether it wouldn't be more convenient for subclassing if they did
behave like regular ndarrays.
The new *maskedarray* is what I eventually come up with. The
main differences with the initial *numpy.core.ma* package are
that MaskedArray is now a subclass of *ndarray* and that the
*_data* section can now be any subclass of *ndarray*. Apart from a
couple of issues listed below, the behavior of the new MaskedArray
class reproduces the old one. Initially the *maskedarray*
implementation was marginally slower than *numpy.ma* in some areas,
but work is underway to speed it up; the expectation is that it can be
made substantially faster than the present *numpy.ma*.
Note that if the subclass has some special methods and
attributes, they are not propagated to the masked version:
this would require a modification of the *__getattribute__*
method (first trying *ndarray.__getattribute__*, then trying
*self._data.__getattribute__* if an exception is raised in the first
place), which really slows things down.
Main differences
----------------
* The *_data* part of the masked array can be any subclass of ndarray (but not recarray, cf below).
* *fill_value* is now a property, not a function.
* in the majority of cases, the mask is forced to *nomask* when no value is actually masked. A notable exception is when a masked array (with no masked values) has just been unpickled.
* I got rid of the *share_mask* flag, I never understood its purpose.
* *put*, *putmask* and *take* now mimic the ndarray methods, to avoid unpleasant surprises. Moreover, *put* and *putmask* both update the mask when needed. * if *a* is a masked array, *bool(a)* raises a *ValueError*, as it does with ndarrays.
* in the same way, the comparison of two masked arrays is a masked array, not a boolean
* *filled(a)* returns an array of the same subclass as *a._data*, and no test is performed on whether it is contiguous or not.
* the mask is always printed, even if it's *nomask*, which makes things easy (for me at least) to remember that a masked array is used.
* *cumsum* works as if the *_data* array was filled with 0. The mask is preserved, but not updated.
* *cumprod* works as if the *_data* array was filled with 1. The mask is preserved, but not updated.
New features
------------
This list is non-exhaustive...
* the *mr_* function mimics *r_* for masked arrays.
* the *anom* method returns the anomalies (deviations from the average)
Using the new package with numpy.core.ma
----------------------------------------
I tried to make sure that the new package can understand old masked
arrays. Unfortunately, there's no upward compatibility.
For example:
>>> import numpy.core.ma as old_ma
>>> import maskedarray as new_ma
>>> x = old_ma.array([1,2,3,4,5], mask=[0,0,1,0,0])
>>> x
array(data =
[ 1 2 999999 4 5],
mask =
[False False True False False],
fill_value=999999)
>>> y = new_ma.array([1,2,3,4,5], mask=[0,0,1,0,0])
>>> y
array(data = [1 2 -- 4 5],
mask = [False False True False False],
fill_value=999999)
>>> x==y
array(data =
[True True True True True],
mask =
[False False True False False],
fill_value=?)
>>> old_ma.getmask(x) == new_ma.getmask(x)
array([True, True, True, True, True])
>>> old_ma.getmask(y) == new_ma.getmask(y)
array([True, True, False, True, True])
>>> old_ma.getmask(y)
False
Using maskedarray with matplotlib
---------------------------------
Starting with matplotlib 0.91.2, the masked array importing will work with
the maskedarray branch) as well as with earlier versions.
By default matplotlib still uses numpy.ma, but there is an rcParams setting
that you can use to select maskedarray instead. In the matplotlibrc file
you will find::
#maskedarray : False # True to use external maskedarray module
# instead of numpy.ma; this is a temporary #
setting for testing maskedarray.
Uncomment and set to True to select maskedarray everywhere.
Alternatively, you can test a script with maskedarray by using a
command-line option, e.g.::
python simple_plot.py --maskedarray
Masked records
--------------
Like *numpy.ma.core*, the *ndarray*-based implementation
of MaskedArray is limited when working with records: you can
mask any record of the array, but not a field in a record. If you
need this feature, you may want to give the *mrecords* package
a try (available in the *maskedarray* directory in the scipy
sandbox). This module defines a new class, *MaskedRecord*. An
instance of this class accepts a *recarray* as data, and uses two
masks: the *fieldmask* has as many entries as records in the array,
each entry with the same fields as a record, but of boolean types:
they indicate whether the field is masked or not; a record entry
is flagged as masked in the *mask* array if all the fields are
masked. A few examples in the file should give you an idea of what
can be done. Note that *mrecords* is still experimental...
Optimizing maskedarray
----------------------
Should masked arrays be filled before processing or not?
--------------------------------------------------------
In the current implementation, most operations on masked arrays involve
the following steps:
* the input arrays are filled
* the operation is performed on the filled arrays
* the mask is set for the results, from the combination of the input masks and the mask corresponding to the domain of the operation.
For example, consider the division of two masked arrays::
import numpy
import maskedarray as ma
x = ma.array([1,2,3,4],mask=[1,0,0,0], dtype=numpy.float64)
y = ma.array([-1,0,1,2], mask=[0,0,0,1], dtype=numpy.float64)
The division of x by y is then computed as::
d1 = x.filled(0) # d1 = array([0., 2., 3., 4.])
d2 = y.filled(1) # array([-1., 0., 1., 1.])
m = ma.mask_or(ma.getmask(x), ma.getmask(y)) # m =
array([True,False,False,True])
dm = ma.divide.domain(d1,d2) # array([False, True, False, False])
result = (d1/d2).view(MaskedArray) # masked_array([-0. inf, 3., 4.])
result._mask = logical_or(m, dm)
Note that a division by zero takes place. To avoid it, we can consider
to fill the input arrays, taking the domain mask into account, so that::
d1 = x._data.copy() # d1 = array([1., 2., 3., 4.])
d2 = y._data.copy() # array([-1., 0., 1., 2.])
dm = ma.divide.domain(d1,d2) # array([False, True, False, False])
numpy.putmask(d2, dm, 1) # d2 = array([-1., 1., 1., 2.])
m = ma.mask_or(ma.getmask(x), ma.getmask(y)) # m =
array([True,False,False,True])
result = (d1/d2).view(MaskedArray) # masked_array([-1. 0., 3., 2.])
result._mask = logical_or(m, dm)
Note that the *.copy()* is required to avoid updating the inputs with
*putmask*. The *.filled()* method also involves a *.copy()*.
A third possibility consists in avoid filling the arrays::
d1 = x._data # d1 = array([1., 2., 3., 4.])
d2 = y._data # array([-1., 0., 1., 2.])
dm = ma.divide.domain(d1,d2) # array([False, True, False, False])
m = ma.mask_or(ma.getmask(x), ma.getmask(y)) # m =
array([True,False,False,True])
result = (d1/d2).view(MaskedArray) # masked_array([-1. inf, 3., 2.])
result._mask = logical_or(m, dm)
Note that here again the division by zero takes place.
A quick benchmark gives the following results:
* *numpy.ma.divide* : 2.69 ms per loop
* classical division : 2.21 ms per loop
* division w/ prefilling : 2.34 ms per loop
* division w/o filling : 1.55 ms per loop
So, is it worth filling the arrays beforehand ? Yes, if we are interested
in avoiding floating-point exceptions that may fill the result with infs
and nans. No, if we are only interested into speed...
Thanks
------
I'd like to thank Paul Dubois, Travis Oliphant and Sasha for the
original masked array package: without you, I would never have started
that (it might be argued that I shouldn't have anyway, but that's
another story...). I also wish to extend these thanks to Reggie Dugard
and Eric Firing for their suggestions and numerous improvements.
Revision notes
--------------
* 08/25/2007 : Creation of this page
* 01/23/2007 : The package has been moved to the SciPy sandbox, and is regularly updated: please check out your SVN version!

View File

@ -0,0 +1,54 @@
"""
=============
Masked Arrays
=============
Arrays sometimes contain invalid or missing data. When doing operations
on such arrays, we wish to suppress invalid values, which is the purpose masked
arrays fulfill (an example of typical use is given below).
For example, examine the following array:
>>> x = np.array([2, 1, 3, np.nan, 5, 2, 3, np.nan])
When we try to calculate the mean of the data, the result is undetermined:
>>> np.mean(x)
nan
The mean is calculated using roughly ``np.sum(x)/len(x)``, but since
any number added to ``NaN`` [1]_ produces ``NaN``, this doesn't work. Enter
masked arrays:
>>> m = np.ma.masked_array(x, np.isnan(x))
>>> m
masked_array(data=[2.0, 1.0, 3.0, --, 5.0, 2.0, 3.0, --],
mask=[False, False, False, True, False, False, False, True],
fill_value=1e+20)
Here, we construct a masked array that suppress all ``NaN`` values. We
may now proceed to calculate the mean of the other values:
>>> np.mean(m)
2.6666666666666665
.. [1] Not-a-Number, a floating point value that is the result of an
invalid operation.
.. moduleauthor:: Pierre Gerard-Marchant
.. moduleauthor:: Jarrod Millman
"""
from . import core
from .core import *
from . import extras
from .extras import *
__all__ = ['core', 'extras']
__all__ += core.__all__
__all__ += extras.__all__
from numpy._pytesttester import PytestTester
test = PytestTester(__name__)
del PytestTester

View File

@ -0,0 +1,233 @@
from numpy._pytesttester import PytestTester
from numpy.ma import extras as extras
from numpy.ma.core import (
MAError as MAError,
MaskError as MaskError,
MaskType as MaskType,
MaskedArray as MaskedArray,
abs as abs,
absolute as absolute,
add as add,
all as all,
allclose as allclose,
allequal as allequal,
alltrue as alltrue,
amax as amax,
amin as amin,
angle as angle,
anom as anom,
anomalies as anomalies,
any as any,
append as append,
arange as arange,
arccos as arccos,
arccosh as arccosh,
arcsin as arcsin,
arcsinh as arcsinh,
arctan as arctan,
arctan2 as arctan2,
arctanh as arctanh,
argmax as argmax,
argmin as argmin,
argsort as argsort,
around as around,
array as array,
asanyarray as asanyarray,
asarray as asarray,
bitwise_and as bitwise_and,
bitwise_or as bitwise_or,
bitwise_xor as bitwise_xor,
bool as bool,
ceil as ceil,
choose as choose,
clip as clip,
common_fill_value as common_fill_value,
compress as compress,
compressed as compressed,
concatenate as concatenate,
conjugate as conjugate,
convolve as convolve,
copy as copy,
correlate as correlate,
cos as cos,
cosh as cosh,
count as count,
cumprod as cumprod,
cumsum as cumsum,
default_fill_value as default_fill_value,
diag as diag,
diagonal as diagonal,
diff as diff,
divide as divide,
empty as empty,
empty_like as empty_like,
equal as equal,
exp as exp,
expand_dims as expand_dims,
fabs as fabs,
filled as filled,
fix_invalid as fix_invalid,
flatten_mask as flatten_mask,
flatten_structured_array as flatten_structured_array,
floor as floor,
floor_divide as floor_divide,
fmod as fmod,
frombuffer as frombuffer,
fromflex as fromflex,
fromfunction as fromfunction,
getdata as getdata,
getmask as getmask,
getmaskarray as getmaskarray,
greater as greater,
greater_equal as greater_equal,
harden_mask as harden_mask,
hypot as hypot,
identity as identity,
ids as ids,
indices as indices,
inner as inner,
innerproduct as innerproduct,
isMA as isMA,
isMaskedArray as isMaskedArray,
is_mask as is_mask,
is_masked as is_masked,
isarray as isarray,
left_shift as left_shift,
less as less,
less_equal as less_equal,
log as log,
log10 as log10,
log2 as log2,
logical_and as logical_and,
logical_not as logical_not,
logical_or as logical_or,
logical_xor as logical_xor,
make_mask as make_mask,
make_mask_descr as make_mask_descr,
make_mask_none as make_mask_none,
mask_or as mask_or,
masked as masked,
masked_array as masked_array,
masked_equal as masked_equal,
masked_greater as masked_greater,
masked_greater_equal as masked_greater_equal,
masked_inside as masked_inside,
masked_invalid as masked_invalid,
masked_less as masked_less,
masked_less_equal as masked_less_equal,
masked_not_equal as masked_not_equal,
masked_object as masked_object,
masked_outside as masked_outside,
masked_print_option as masked_print_option,
masked_singleton as masked_singleton,
masked_values as masked_values,
masked_where as masked_where,
max as max,
maximum as maximum,
maximum_fill_value as maximum_fill_value,
mean as mean,
min as min,
minimum as minimum,
minimum_fill_value as minimum_fill_value,
mod as mod,
multiply as multiply,
mvoid as mvoid,
ndim as ndim,
negative as negative,
nomask as nomask,
nonzero as nonzero,
not_equal as not_equal,
ones as ones,
outer as outer,
outerproduct as outerproduct,
power as power,
prod as prod,
product as product,
ptp as ptp,
put as put,
putmask as putmask,
ravel as ravel,
remainder as remainder,
repeat as repeat,
reshape as reshape,
resize as resize,
right_shift as right_shift,
round as round,
set_fill_value as set_fill_value,
shape as shape,
sin as sin,
sinh as sinh,
size as size,
soften_mask as soften_mask,
sometrue as sometrue,
sort as sort,
sqrt as sqrt,
squeeze as squeeze,
std as std,
subtract as subtract,
sum as sum,
swapaxes as swapaxes,
take as take,
tan as tan,
tanh as tanh,
trace as trace,
transpose as transpose,
true_divide as true_divide,
var as var,
where as where,
zeros as zeros,
)
from numpy.ma.extras import (
apply_along_axis as apply_along_axis,
apply_over_axes as apply_over_axes,
atleast_1d as atleast_1d,
atleast_2d as atleast_2d,
atleast_3d as atleast_3d,
average as average,
clump_masked as clump_masked,
clump_unmasked as clump_unmasked,
column_stack as column_stack,
compress_cols as compress_cols,
compress_nd as compress_nd,
compress_rowcols as compress_rowcols,
compress_rows as compress_rows,
count_masked as count_masked,
corrcoef as corrcoef,
cov as cov,
diagflat as diagflat,
dot as dot,
dstack as dstack,
ediff1d as ediff1d,
flatnotmasked_contiguous as flatnotmasked_contiguous,
flatnotmasked_edges as flatnotmasked_edges,
hsplit as hsplit,
hstack as hstack,
isin as isin,
in1d as in1d,
intersect1d as intersect1d,
mask_cols as mask_cols,
mask_rowcols as mask_rowcols,
mask_rows as mask_rows,
masked_all as masked_all,
masked_all_like as masked_all_like,
median as median,
mr_ as mr_,
ndenumerate as ndenumerate,
notmasked_contiguous as notmasked_contiguous,
notmasked_edges as notmasked_edges,
polyfit as polyfit,
row_stack as row_stack,
setdiff1d as setdiff1d,
setxor1d as setxor1d,
stack as stack,
unique as unique,
union1d as union1d,
vander as vander,
vstack as vstack,
)
__all__: list[str]
test: PytestTester

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,467 @@
from collections.abc import Callable
from typing import Any, TypeVar
from numpy import ndarray, dtype, float64
from numpy import (
amax as amax,
amin as amin,
bool as bool,
expand_dims as expand_dims,
clip as clip,
indices as indices,
ones_like as ones_like,
squeeze as squeeze,
zeros_like as zeros_like,
angle as angle
)
_ShapeType_co = TypeVar("_ShapeType_co", bound=tuple[int, ...], covariant=True)
_DType_co = TypeVar("_DType_co", bound=dtype[Any], covariant=True)
__all__: list[str]
MaskType = bool
nomask: bool
class MaskedArrayFutureWarning(FutureWarning): ...
class MAError(Exception): ...
class MaskError(MAError): ...
def default_fill_value(obj): ...
def minimum_fill_value(obj): ...
def maximum_fill_value(obj): ...
def set_fill_value(a, fill_value): ...
def common_fill_value(a, b): ...
def filled(a, fill_value=...): ...
def getdata(a, subok=...): ...
get_data = getdata
def fix_invalid(a, mask=..., copy=..., fill_value=...): ...
class _MaskedUFunc:
f: Any
__doc__: Any
__name__: Any
def __init__(self, ufunc): ...
class _MaskedUnaryOperation(_MaskedUFunc):
fill: Any
domain: Any
def __init__(self, mufunc, fill=..., domain=...): ...
def __call__(self, a, *args, **kwargs): ...
class _MaskedBinaryOperation(_MaskedUFunc):
fillx: Any
filly: Any
def __init__(self, mbfunc, fillx=..., filly=...): ...
def __call__(self, a, b, *args, **kwargs): ...
def reduce(self, target, axis=..., dtype=...): ...
def outer(self, a, b): ...
def accumulate(self, target, axis=...): ...
class _DomainedBinaryOperation(_MaskedUFunc):
domain: Any
fillx: Any
filly: Any
def __init__(self, dbfunc, domain, fillx=..., filly=...): ...
def __call__(self, a, b, *args, **kwargs): ...
exp: _MaskedUnaryOperation
conjugate: _MaskedUnaryOperation
sin: _MaskedUnaryOperation
cos: _MaskedUnaryOperation
arctan: _MaskedUnaryOperation
arcsinh: _MaskedUnaryOperation
sinh: _MaskedUnaryOperation
cosh: _MaskedUnaryOperation
tanh: _MaskedUnaryOperation
abs: _MaskedUnaryOperation
absolute: _MaskedUnaryOperation
fabs: _MaskedUnaryOperation
negative: _MaskedUnaryOperation
floor: _MaskedUnaryOperation
ceil: _MaskedUnaryOperation
around: _MaskedUnaryOperation
logical_not: _MaskedUnaryOperation
sqrt: _MaskedUnaryOperation
log: _MaskedUnaryOperation
log2: _MaskedUnaryOperation
log10: _MaskedUnaryOperation
tan: _MaskedUnaryOperation
arcsin: _MaskedUnaryOperation
arccos: _MaskedUnaryOperation
arccosh: _MaskedUnaryOperation
arctanh: _MaskedUnaryOperation
add: _MaskedBinaryOperation
subtract: _MaskedBinaryOperation
multiply: _MaskedBinaryOperation
arctan2: _MaskedBinaryOperation
equal: _MaskedBinaryOperation
not_equal: _MaskedBinaryOperation
less_equal: _MaskedBinaryOperation
greater_equal: _MaskedBinaryOperation
less: _MaskedBinaryOperation
greater: _MaskedBinaryOperation
logical_and: _MaskedBinaryOperation
alltrue: _MaskedBinaryOperation
logical_or: _MaskedBinaryOperation
sometrue: Callable[..., Any]
logical_xor: _MaskedBinaryOperation
bitwise_and: _MaskedBinaryOperation
bitwise_or: _MaskedBinaryOperation
bitwise_xor: _MaskedBinaryOperation
hypot: _MaskedBinaryOperation
divide: _MaskedBinaryOperation
true_divide: _MaskedBinaryOperation
floor_divide: _MaskedBinaryOperation
remainder: _MaskedBinaryOperation
fmod: _MaskedBinaryOperation
mod: _MaskedBinaryOperation
def make_mask_descr(ndtype): ...
def getmask(a): ...
get_mask = getmask
def getmaskarray(arr): ...
def is_mask(m): ...
def make_mask(m, copy=..., shrink=..., dtype=...): ...
def make_mask_none(newshape, dtype=...): ...
def mask_or(m1, m2, copy=..., shrink=...): ...
def flatten_mask(mask): ...
def masked_where(condition, a, copy=...): ...
def masked_greater(x, value, copy=...): ...
def masked_greater_equal(x, value, copy=...): ...
def masked_less(x, value, copy=...): ...
def masked_less_equal(x, value, copy=...): ...
def masked_not_equal(x, value, copy=...): ...
def masked_equal(x, value, copy=...): ...
def masked_inside(x, v1, v2, copy=...): ...
def masked_outside(x, v1, v2, copy=...): ...
def masked_object(x, value, copy=..., shrink=...): ...
def masked_values(x, value, rtol=..., atol=..., copy=..., shrink=...): ...
def masked_invalid(a, copy=...): ...
class _MaskedPrintOption:
def __init__(self, display): ...
def display(self): ...
def set_display(self, s): ...
def enabled(self): ...
def enable(self, shrink=...): ...
masked_print_option: _MaskedPrintOption
def flatten_structured_array(a): ...
class MaskedIterator:
ma: Any
dataiter: Any
maskiter: Any
def __init__(self, ma): ...
def __iter__(self): ...
def __getitem__(self, indx): ...
def __setitem__(self, index, value): ...
def __next__(self): ...
class MaskedArray(ndarray[_ShapeType_co, _DType_co]):
__array_priority__: Any
def __new__(cls, data=..., mask=..., dtype=..., copy=..., subok=..., ndmin=..., fill_value=..., keep_mask=..., hard_mask=..., shrink=..., order=...): ...
def __array_finalize__(self, obj): ...
def __array_wrap__(self, obj, context=..., return_scalar=...): ...
def view(self, dtype=..., type=..., fill_value=...): ...
def __getitem__(self, indx): ...
def __setitem__(self, indx, value): ...
@property
def dtype(self): ...
@dtype.setter
def dtype(self, dtype): ...
@property
def shape(self): ...
@shape.setter
def shape(self, shape): ...
def __setmask__(self, mask, copy=...): ...
@property
def mask(self): ...
@mask.setter
def mask(self, value): ...
@property
def recordmask(self): ...
@recordmask.setter
def recordmask(self, mask): ...
def harden_mask(self): ...
def soften_mask(self): ...
@property
def hardmask(self): ...
def unshare_mask(self): ...
@property
def sharedmask(self): ...
def shrink_mask(self): ...
@property
def baseclass(self): ...
data: Any
@property
def flat(self): ...
@flat.setter
def flat(self, value): ...
@property
def fill_value(self): ...
@fill_value.setter
def fill_value(self, value=...): ...
get_fill_value: Any
set_fill_value: Any
def filled(self, fill_value=...): ...
def compressed(self): ...
def compress(self, condition, axis=..., out=...): ...
def __eq__(self, other): ...
def __ne__(self, other): ...
def __ge__(self, other): ...
def __gt__(self, other): ...
def __le__(self, other): ...
def __lt__(self, other): ...
def __add__(self, other): ...
def __radd__(self, other): ...
def __sub__(self, other): ...
def __rsub__(self, other): ...
def __mul__(self, other): ...
def __rmul__(self, other): ...
def __div__(self, other): ...
def __truediv__(self, other): ...
def __rtruediv__(self, other): ...
def __floordiv__(self, other): ...
def __rfloordiv__(self, other): ...
def __pow__(self, other): ...
def __rpow__(self, other): ...
def __iadd__(self, other): ...
def __isub__(self, other): ...
def __imul__(self, other): ...
def __idiv__(self, other): ...
def __ifloordiv__(self, other): ...
def __itruediv__(self, other): ...
def __ipow__(self, other): ...
def __float__(self): ...
def __int__(self): ...
@property # type: ignore[misc]
def imag(self): ...
get_imag: Any
@property # type: ignore[misc]
def real(self): ...
get_real: Any
def count(self, axis=..., keepdims=...): ...
def ravel(self, order=...): ...
def reshape(self, *s, **kwargs): ...
def resize(self, newshape, refcheck=..., order=...): ...
def put(self, indices, values, mode=...): ...
def ids(self): ...
def iscontiguous(self): ...
def all(self, axis=..., out=..., keepdims=...): ...
def any(self, axis=..., out=..., keepdims=...): ...
def nonzero(self): ...
def trace(self, offset=..., axis1=..., axis2=..., dtype=..., out=...): ...
def dot(self, b, out=..., strict=...): ...
def sum(self, axis=..., dtype=..., out=..., keepdims=...): ...
def cumsum(self, axis=..., dtype=..., out=...): ...
def prod(self, axis=..., dtype=..., out=..., keepdims=...): ...
product: Any
def cumprod(self, axis=..., dtype=..., out=...): ...
def mean(self, axis=..., dtype=..., out=..., keepdims=...): ...
def anom(self, axis=..., dtype=...): ...
def var(self, axis=..., dtype=..., out=..., ddof=..., keepdims=...): ...
def std(self, axis=..., dtype=..., out=..., ddof=..., keepdims=...): ...
def round(self, decimals=..., out=...): ...
def argsort(self, axis=..., kind=..., order=..., endwith=..., fill_value=..., stable=...): ...
def argmin(self, axis=..., fill_value=..., out=..., *, keepdims=...): ...
def argmax(self, axis=..., fill_value=..., out=..., *, keepdims=...): ...
def sort(self, axis=..., kind=..., order=..., endwith=..., fill_value=..., stable=...): ...
def min(self, axis=..., out=..., fill_value=..., keepdims=...): ...
# NOTE: deprecated
# def tostring(self, fill_value=..., order=...): ...
def max(self, axis=..., out=..., fill_value=..., keepdims=...): ...
def ptp(self, axis=..., out=..., fill_value=..., keepdims=...): ...
def partition(self, *args, **kwargs): ...
def argpartition(self, *args, **kwargs): ...
def take(self, indices, axis=..., out=..., mode=...): ...
copy: Any
diagonal: Any
flatten: Any
repeat: Any
squeeze: Any
swapaxes: Any
T: Any
transpose: Any
@property # type: ignore[misc]
def mT(self): ...
def tolist(self, fill_value=...): ...
def tobytes(self, fill_value=..., order=...): ...
def tofile(self, fid, sep=..., format=...): ...
def toflex(self): ...
torecords: Any
def __reduce__(self): ...
def __deepcopy__(self, memo=...): ...
class mvoid(MaskedArray[_ShapeType_co, _DType_co]):
def __new__(
self,
data,
mask=...,
dtype=...,
fill_value=...,
hardmask=...,
copy=...,
subok=...,
): ...
def __getitem__(self, indx): ...
def __setitem__(self, indx, value): ...
def __iter__(self): ...
def __len__(self): ...
def filled(self, fill_value=...): ...
def tolist(self): ...
def isMaskedArray(x): ...
isarray = isMaskedArray
isMA = isMaskedArray
# 0D float64 array
class MaskedConstant(MaskedArray[Any, dtype[float64]]):
def __new__(cls): ...
__class__: Any
def __array_finalize__(self, obj): ...
def __array_wrap__(self, obj, context=..., return_scalar=...): ...
def __format__(self, format_spec): ...
def __reduce__(self): ...
def __iop__(self, other): ...
__iadd__: Any
__isub__: Any
__imul__: Any
__ifloordiv__: Any
__itruediv__: Any
__ipow__: Any
def copy(self, *args, **kwargs): ...
def __copy__(self): ...
def __deepcopy__(self, memo): ...
def __setattr__(self, attr, value): ...
masked: MaskedConstant
masked_singleton: MaskedConstant
masked_array = MaskedArray
def array(
data,
dtype=...,
copy=...,
order=...,
mask=...,
fill_value=...,
keep_mask=...,
hard_mask=...,
shrink=...,
subok=...,
ndmin=...,
): ...
def is_masked(x): ...
class _extrema_operation(_MaskedUFunc):
compare: Any
fill_value_func: Any
def __init__(self, ufunc, compare, fill_value): ...
# NOTE: in practice `b` has a default value, but users should
# explicitly provide a value here as the default is deprecated
def __call__(self, a, b): ...
def reduce(self, target, axis=...): ...
def outer(self, a, b): ...
def min(obj, axis=..., out=..., fill_value=..., keepdims=...): ...
def max(obj, axis=..., out=..., fill_value=..., keepdims=...): ...
def ptp(obj, axis=..., out=..., fill_value=..., keepdims=...): ...
class _frommethod:
__name__: Any
__doc__: Any
reversed: Any
def __init__(self, methodname, reversed=...): ...
def getdoc(self): ...
def __call__(self, a, *args, **params): ...
all: _frommethod
anomalies: _frommethod
anom: _frommethod
any: _frommethod
compress: _frommethod
cumprod: _frommethod
cumsum: _frommethod
copy: _frommethod
diagonal: _frommethod
harden_mask: _frommethod
ids: _frommethod
mean: _frommethod
nonzero: _frommethod
prod: _frommethod
product: _frommethod
ravel: _frommethod
repeat: _frommethod
soften_mask: _frommethod
std: _frommethod
sum: _frommethod
swapaxes: _frommethod
trace: _frommethod
var: _frommethod
count: _frommethod
argmin: _frommethod
argmax: _frommethod
minimum: _extrema_operation
maximum: _extrema_operation
def take(a, indices, axis=..., out=..., mode=...): ...
def power(a, b, third=...): ...
def argsort(a, axis=..., kind=..., order=..., endwith=..., fill_value=..., stable=...): ...
def sort(a, axis=..., kind=..., order=..., endwith=..., fill_value=..., stable=...): ...
def compressed(x): ...
def concatenate(arrays, axis=...): ...
def diag(v, k=...): ...
def left_shift(a, n): ...
def right_shift(a, n): ...
def put(a, indices, values, mode=...): ...
def putmask(a, mask, values): ...
def transpose(a, axes=...): ...
def reshape(a, new_shape, order=...): ...
def resize(x, new_shape): ...
def ndim(obj): ...
def shape(obj): ...
def size(obj, axis=...): ...
def diff(a, /, n=..., axis=..., prepend=..., append=...): ...
def where(condition, x=..., y=...): ...
def choose(indices, choices, out=..., mode=...): ...
def round(a, decimals=..., out=...): ...
def inner(a, b): ...
innerproduct = inner
def outer(a, b): ...
outerproduct = outer
def correlate(a, v, mode=..., propagate_mask=...): ...
def convolve(a, v, mode=..., propagate_mask=...): ...
def allequal(a, b, fill_value=...): ...
def allclose(a, b, masked_equal=..., rtol=..., atol=...): ...
def asarray(a, dtype=..., order=...): ...
def asanyarray(a, dtype=...): ...
def fromflex(fxarray): ...
class _convert2ma:
__doc__: Any
def __init__(self, funcname, params=...): ...
def getdoc(self): ...
def __call__(self, *args, **params): ...
arange: _convert2ma
empty: _convert2ma
empty_like: _convert2ma
frombuffer: _convert2ma
fromfunction: _convert2ma
identity: _convert2ma
ones: _convert2ma
zeros: _convert2ma
def append(a, b, axis=...): ...
def dot(a, b, strict=..., out=...): ...
def mask_rowcols(a, axis=...): ...

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,86 @@
from typing import Any
from numpy.lib._index_tricks_impl import AxisConcatenator
from numpy.ma.core import (
dot as dot,
mask_rowcols as mask_rowcols,
)
__all__: list[str]
def count_masked(arr, axis=...): ...
def masked_all(shape, dtype = ...): ...
def masked_all_like(arr): ...
class _fromnxfunction:
__name__: Any
__doc__: Any
def __init__(self, funcname): ...
def getdoc(self): ...
def __call__(self, *args, **params): ...
class _fromnxfunction_single(_fromnxfunction):
def __call__(self, x, *args, **params): ...
class _fromnxfunction_seq(_fromnxfunction):
def __call__(self, x, *args, **params): ...
class _fromnxfunction_allargs(_fromnxfunction):
def __call__(self, *args, **params): ...
atleast_1d: _fromnxfunction_allargs
atleast_2d: _fromnxfunction_allargs
atleast_3d: _fromnxfunction_allargs
vstack: _fromnxfunction_seq
row_stack: _fromnxfunction_seq
hstack: _fromnxfunction_seq
column_stack: _fromnxfunction_seq
dstack: _fromnxfunction_seq
stack: _fromnxfunction_seq
hsplit: _fromnxfunction_single
diagflat: _fromnxfunction_single
def apply_along_axis(func1d, axis, arr, *args, **kwargs): ...
def apply_over_axes(func, a, axes): ...
def average(a, axis=..., weights=..., returned=..., keepdims=...): ...
def median(a, axis=..., out=..., overwrite_input=..., keepdims=...): ...
def compress_nd(x, axis=...): ...
def compress_rowcols(x, axis=...): ...
def compress_rows(a): ...
def compress_cols(a): ...
def mask_rows(a, axis = ...): ...
def mask_cols(a, axis = ...): ...
def ediff1d(arr, to_end=..., to_begin=...): ...
def unique(ar1, return_index=..., return_inverse=...): ...
def intersect1d(ar1, ar2, assume_unique=...): ...
def setxor1d(ar1, ar2, assume_unique=...): ...
def in1d(ar1, ar2, assume_unique=..., invert=...): ...
def isin(element, test_elements, assume_unique=..., invert=...): ...
def union1d(ar1, ar2): ...
def setdiff1d(ar1, ar2, assume_unique=...): ...
def cov(x, y=..., rowvar=..., bias=..., allow_masked=..., ddof=...): ...
def corrcoef(x, y=..., rowvar=..., bias = ..., allow_masked=..., ddof = ...): ...
class MAxisConcatenator(AxisConcatenator):
concatenate: Any
@classmethod
def makemat(cls, arr): ...
def __getitem__(self, key): ...
class mr_class(MAxisConcatenator):
def __init__(self): ...
mr_: mr_class
def ndenumerate(a, compressed=...): ...
def flatnotmasked_edges(a): ...
def notmasked_edges(a, axis=...): ...
def flatnotmasked_contiguous(a): ...
def notmasked_contiguous(a, axis=...): ...
def clump_unmasked(a): ...
def clump_masked(a): ...
def vander(x, n=...): ...
def polyfit(x, y, deg, rcond=..., full=..., w=..., cov=...): ...

View File

@ -0,0 +1,782 @@
""":mod:`numpy.ma..mrecords`
Defines the equivalent of :class:`numpy.recarrays` for masked arrays,
where fields can be accessed as attributes.
Note that :class:`numpy.ma.MaskedArray` already supports structured datatypes
and the masking of individual fields.
.. moduleauthor:: Pierre Gerard-Marchant
"""
# We should make sure that no field is called '_mask','mask','_fieldmask',
# or whatever restricted keywords. An idea would be to no bother in the
# first place, and then rename the invalid fields with a trailing
# underscore. Maybe we could just overload the parser function ?
from numpy.ma import (
MAError, MaskedArray, masked, nomask, masked_array, getdata,
getmaskarray, filled
)
import numpy.ma as ma
import warnings
import numpy as np
from numpy import dtype, ndarray, array as narray
from numpy._core.records import (
recarray, fromarrays as recfromarrays, fromrecords as recfromrecords
)
_byteorderconv = np._core.records._byteorderconv
_check_fill_value = ma.core._check_fill_value
__all__ = [
'MaskedRecords', 'mrecarray', 'fromarrays', 'fromrecords',
'fromtextfile', 'addfield',
]
reserved_fields = ['_data', '_mask', '_fieldmask', 'dtype']
def _checknames(descr, names=None):
"""
Checks that field names ``descr`` are not reserved keywords.
If this is the case, a default 'f%i' is substituted. If the argument
`names` is not None, updates the field names to valid names.
"""
ndescr = len(descr)
default_names = ['f%i' % i for i in range(ndescr)]
if names is None:
new_names = default_names
else:
if isinstance(names, (tuple, list)):
new_names = names
elif isinstance(names, str):
new_names = names.split(',')
else:
raise NameError(f'illegal input names {names!r}')
nnames = len(new_names)
if nnames < ndescr:
new_names += default_names[nnames:]
ndescr = []
for (n, d, t) in zip(new_names, default_names, descr.descr):
if n in reserved_fields:
if t[0] in reserved_fields:
ndescr.append((d, t[1]))
else:
ndescr.append(t)
else:
ndescr.append((n, t[1]))
return np.dtype(ndescr)
def _get_fieldmask(self):
mdescr = [(n, '|b1') for n in self.dtype.names]
fdmask = np.empty(self.shape, dtype=mdescr)
fdmask.flat = tuple([False] * len(mdescr))
return fdmask
class MaskedRecords(MaskedArray):
"""
Attributes
----------
_data : recarray
Underlying data, as a record array.
_mask : boolean array
Mask of the records. A record is masked when all its fields are
masked.
_fieldmask : boolean recarray
Record array of booleans, setting the mask of each individual field
of each record.
_fill_value : record
Filling values for each field.
"""
def __new__(cls, shape, dtype=None, buf=None, offset=0, strides=None,
formats=None, names=None, titles=None,
byteorder=None, aligned=False,
mask=nomask, hard_mask=False, fill_value=None, keep_mask=True,
copy=False,
**options):
self = recarray.__new__(cls, shape, dtype=dtype, buf=buf, offset=offset,
strides=strides, formats=formats, names=names,
titles=titles, byteorder=byteorder,
aligned=aligned,)
mdtype = ma.make_mask_descr(self.dtype)
if mask is nomask or not np.size(mask):
if not keep_mask:
self._mask = tuple([False] * len(mdtype))
else:
mask = np.array(mask, copy=copy)
if mask.shape != self.shape:
(nd, nm) = (self.size, mask.size)
if nm == 1:
mask = np.resize(mask, self.shape)
elif nm == nd:
mask = np.reshape(mask, self.shape)
else:
msg = "Mask and data not compatible: data size is %i, " + \
"mask size is %i."
raise MAError(msg % (nd, nm))
if not keep_mask:
self.__setmask__(mask)
self._sharedmask = True
else:
if mask.dtype == mdtype:
_mask = mask
else:
_mask = np.array([tuple([m] * len(mdtype)) for m in mask],
dtype=mdtype)
self._mask = _mask
return self
def __array_finalize__(self, obj):
# Make sure we have a _fieldmask by default
_mask = getattr(obj, '_mask', None)
if _mask is None:
objmask = getattr(obj, '_mask', nomask)
_dtype = ndarray.__getattribute__(self, 'dtype')
if objmask is nomask:
_mask = ma.make_mask_none(self.shape, dtype=_dtype)
else:
mdescr = ma.make_mask_descr(_dtype)
_mask = narray([tuple([m] * len(mdescr)) for m in objmask],
dtype=mdescr).view(recarray)
# Update some of the attributes
_dict = self.__dict__
_dict.update(_mask=_mask)
self._update_from(obj)
if _dict['_baseclass'] == ndarray:
_dict['_baseclass'] = recarray
return
@property
def _data(self):
"""
Returns the data as a recarray.
"""
return ndarray.view(self, recarray)
@property
def _fieldmask(self):
"""
Alias to mask.
"""
return self._mask
def __len__(self):
"""
Returns the length
"""
# We have more than one record
if self.ndim:
return len(self._data)
# We have only one record: return the nb of fields
return len(self.dtype)
def __getattribute__(self, attr):
try:
return object.__getattribute__(self, attr)
except AttributeError:
# attr must be a fieldname
pass
fielddict = ndarray.__getattribute__(self, 'dtype').fields
try:
res = fielddict[attr][:2]
except (TypeError, KeyError) as e:
raise AttributeError(
f'record array has no attribute {attr}') from e
# So far, so good
_localdict = ndarray.__getattribute__(self, '__dict__')
_data = ndarray.view(self, _localdict['_baseclass'])
obj = _data.getfield(*res)
if obj.dtype.names is not None:
raise NotImplementedError("MaskedRecords is currently limited to"
"simple records.")
# Get some special attributes
# Reset the object's mask
hasmasked = False
_mask = _localdict.get('_mask', None)
if _mask is not None:
try:
_mask = _mask[attr]
except IndexError:
# Couldn't find a mask: use the default (nomask)
pass
tp_len = len(_mask.dtype)
hasmasked = _mask.view((bool, ((tp_len,) if tp_len else ()))).any()
if (obj.shape or hasmasked):
obj = obj.view(MaskedArray)
obj._baseclass = ndarray
obj._isfield = True
obj._mask = _mask
# Reset the field values
_fill_value = _localdict.get('_fill_value', None)
if _fill_value is not None:
try:
obj._fill_value = _fill_value[attr]
except ValueError:
obj._fill_value = None
else:
obj = obj.item()
return obj
def __setattr__(self, attr, val):
"""
Sets the attribute attr to the value val.
"""
# Should we call __setmask__ first ?
if attr in ['mask', 'fieldmask']:
self.__setmask__(val)
return
# Create a shortcut (so that we don't have to call getattr all the time)
_localdict = object.__getattribute__(self, '__dict__')
# Check whether we're creating a new field
newattr = attr not in _localdict
try:
# Is attr a generic attribute ?
ret = object.__setattr__(self, attr, val)
except Exception:
# Not a generic attribute: exit if it's not a valid field
fielddict = ndarray.__getattribute__(self, 'dtype').fields or {}
optinfo = ndarray.__getattribute__(self, '_optinfo') or {}
if not (attr in fielddict or attr in optinfo):
raise
else:
# Get the list of names
fielddict = ndarray.__getattribute__(self, 'dtype').fields or {}
# Check the attribute
if attr not in fielddict:
return ret
if newattr:
# We just added this one or this setattr worked on an
# internal attribute.
try:
object.__delattr__(self, attr)
except Exception:
return ret
# Let's try to set the field
try:
res = fielddict[attr][:2]
except (TypeError, KeyError) as e:
raise AttributeError(
f'record array has no attribute {attr}') from e
if val is masked:
_fill_value = _localdict['_fill_value']
if _fill_value is not None:
dval = _localdict['_fill_value'][attr]
else:
dval = val
mval = True
else:
dval = filled(val)
mval = getmaskarray(val)
obj = ndarray.__getattribute__(self, '_data').setfield(dval, *res)
_localdict['_mask'].__setitem__(attr, mval)
return obj
def __getitem__(self, indx):
"""
Returns all the fields sharing the same fieldname base.
The fieldname base is either `_data` or `_mask`.
"""
_localdict = self.__dict__
_mask = ndarray.__getattribute__(self, '_mask')
_data = ndarray.view(self, _localdict['_baseclass'])
# We want a field
if isinstance(indx, str):
# Make sure _sharedmask is True to propagate back to _fieldmask
# Don't use _set_mask, there are some copies being made that
# break propagation Don't force the mask to nomask, that wreaks
# easy masking
obj = _data[indx].view(MaskedArray)
obj._mask = _mask[indx]
obj._sharedmask = True
fval = _localdict['_fill_value']
if fval is not None:
obj._fill_value = fval[indx]
# Force to masked if the mask is True
if not obj.ndim and obj._mask:
return masked
return obj
# We want some elements.
# First, the data.
obj = np.asarray(_data[indx]).view(mrecarray)
obj._mask = np.asarray(_mask[indx]).view(recarray)
return obj
def __setitem__(self, indx, value):
"""
Sets the given record to value.
"""
MaskedArray.__setitem__(self, indx, value)
if isinstance(indx, str):
self._mask[indx] = ma.getmaskarray(value)
def __str__(self):
"""
Calculates the string representation.
"""
if self.size > 1:
mstr = [f"({','.join([str(i) for i in s])})"
for s in zip(*[getattr(self, f) for f in self.dtype.names])]
return f"[{', '.join(mstr)}]"
else:
mstr = [f"{','.join([str(i) for i in s])}"
for s in zip([getattr(self, f) for f in self.dtype.names])]
return f"({', '.join(mstr)})"
def __repr__(self):
"""
Calculates the repr representation.
"""
_names = self.dtype.names
fmt = "%%%is : %%s" % (max([len(n) for n in _names]) + 4,)
reprstr = [fmt % (f, getattr(self, f)) for f in self.dtype.names]
reprstr.insert(0, 'masked_records(')
reprstr.extend([fmt % (' fill_value', self.fill_value),
' )'])
return str("\n".join(reprstr))
def view(self, dtype=None, type=None):
"""
Returns a view of the mrecarray.
"""
# OK, basic copy-paste from MaskedArray.view.
if dtype is None:
if type is None:
output = ndarray.view(self)
else:
output = ndarray.view(self, type)
# Here again.
elif type is None:
try:
if issubclass(dtype, ndarray):
output = ndarray.view(self, dtype)
else:
output = ndarray.view(self, dtype)
# OK, there's the change
except TypeError:
dtype = np.dtype(dtype)
# we need to revert to MaskedArray, but keeping the possibility
# of subclasses (eg, TimeSeriesRecords), so we'll force a type
# set to the first parent
if dtype.fields is None:
basetype = self.__class__.__bases__[0]
output = self.__array__().view(dtype, basetype)
output._update_from(self)
else:
output = ndarray.view(self, dtype)
output._fill_value = None
else:
output = ndarray.view(self, dtype, type)
# Update the mask, just like in MaskedArray.view
if (getattr(output, '_mask', nomask) is not nomask):
mdtype = ma.make_mask_descr(output.dtype)
output._mask = self._mask.view(mdtype, ndarray)
output._mask.shape = output.shape
return output
def harden_mask(self):
"""
Forces the mask to hard.
"""
self._hardmask = True
def soften_mask(self):
"""
Forces the mask to soft
"""
self._hardmask = False
def copy(self):
"""
Returns a copy of the masked record.
"""
copied = self._data.copy().view(type(self))
copied._mask = self._mask.copy()
return copied
def tolist(self, fill_value=None):
"""
Return the data portion of the array as a list.
Data items are converted to the nearest compatible Python type.
Masked values are converted to fill_value. If fill_value is None,
the corresponding entries in the output list will be ``None``.
"""
if fill_value is not None:
return self.filled(fill_value).tolist()
result = narray(self.filled().tolist(), dtype=object)
mask = narray(self._mask.tolist())
result[mask] = None
return result.tolist()
def __getstate__(self):
"""Return the internal state of the masked array.
This is for pickling.
"""
state = (1,
self.shape,
self.dtype,
self.flags.fnc,
self._data.tobytes(),
self._mask.tobytes(),
self._fill_value,
)
return state
def __setstate__(self, state):
"""
Restore the internal state of the masked array.
This is for pickling. ``state`` is typically the output of the
``__getstate__`` output, and is a 5-tuple:
- class name
- a tuple giving the shape of the data
- a typecode for the data
- a binary string for the data
- a binary string for the mask.
"""
(ver, shp, typ, isf, raw, msk, flv) = state
ndarray.__setstate__(self, (shp, typ, isf, raw))
mdtype = dtype([(k, np.bool) for (k, _) in self.dtype.descr])
self.__dict__['_mask'].__setstate__((shp, mdtype, isf, msk))
self.fill_value = flv
def __reduce__(self):
"""
Return a 3-tuple for pickling a MaskedArray.
"""
return (_mrreconstruct,
(self.__class__, self._baseclass, (0,), 'b',),
self.__getstate__())
def _mrreconstruct(subtype, baseclass, baseshape, basetype,):
"""
Build a new MaskedArray from the information stored in a pickle.
"""
_data = ndarray.__new__(baseclass, baseshape, basetype).view(subtype)
_mask = ndarray.__new__(ndarray, baseshape, 'b1')
return subtype.__new__(subtype, _data, mask=_mask, dtype=basetype,)
mrecarray = MaskedRecords
###############################################################################
# Constructors #
###############################################################################
def fromarrays(arraylist, dtype=None, shape=None, formats=None,
names=None, titles=None, aligned=False, byteorder=None,
fill_value=None):
"""
Creates a mrecarray from a (flat) list of masked arrays.
Parameters
----------
arraylist : sequence
A list of (masked) arrays. Each element of the sequence is first converted
to a masked array if needed. If a 2D array is passed as argument, it is
processed line by line
dtype : {None, dtype}, optional
Data type descriptor.
shape : {None, integer}, optional
Number of records. If None, shape is defined from the shape of the
first array in the list.
formats : {None, sequence}, optional
Sequence of formats for each individual field. If None, the formats will
be autodetected by inspecting the fields and selecting the highest dtype
possible.
names : {None, sequence}, optional
Sequence of the names of each field.
fill_value : {None, sequence}, optional
Sequence of data to be used as filling values.
Notes
-----
Lists of tuples should be preferred over lists of lists for faster processing.
"""
datalist = [getdata(x) for x in arraylist]
masklist = [np.atleast_1d(getmaskarray(x)) for x in arraylist]
_array = recfromarrays(datalist,
dtype=dtype, shape=shape, formats=formats,
names=names, titles=titles, aligned=aligned,
byteorder=byteorder).view(mrecarray)
_array._mask.flat = list(zip(*masklist))
if fill_value is not None:
_array.fill_value = fill_value
return _array
def fromrecords(reclist, dtype=None, shape=None, formats=None, names=None,
titles=None, aligned=False, byteorder=None,
fill_value=None, mask=nomask):
"""
Creates a MaskedRecords from a list of records.
Parameters
----------
reclist : sequence
A list of records. Each element of the sequence is first converted
to a masked array if needed. If a 2D array is passed as argument, it is
processed line by line
dtype : {None, dtype}, optional
Data type descriptor.
shape : {None,int}, optional
Number of records. If None, ``shape`` is defined from the shape of the
first array in the list.
formats : {None, sequence}, optional
Sequence of formats for each individual field. If None, the formats will
be autodetected by inspecting the fields and selecting the highest dtype
possible.
names : {None, sequence}, optional
Sequence of the names of each field.
fill_value : {None, sequence}, optional
Sequence of data to be used as filling values.
mask : {nomask, sequence}, optional.
External mask to apply on the data.
Notes
-----
Lists of tuples should be preferred over lists of lists for faster processing.
"""
# Grab the initial _fieldmask, if needed:
_mask = getattr(reclist, '_mask', None)
# Get the list of records.
if isinstance(reclist, ndarray):
# Make sure we don't have some hidden mask
if isinstance(reclist, MaskedArray):
reclist = reclist.filled().view(ndarray)
# Grab the initial dtype, just in case
if dtype is None:
dtype = reclist.dtype
reclist = reclist.tolist()
mrec = recfromrecords(reclist, dtype=dtype, shape=shape, formats=formats,
names=names, titles=titles,
aligned=aligned, byteorder=byteorder).view(mrecarray)
# Set the fill_value if needed
if fill_value is not None:
mrec.fill_value = fill_value
# Now, let's deal w/ the mask
if mask is not nomask:
mask = np.asarray(mask)
maskrecordlength = len(mask.dtype)
if maskrecordlength:
mrec._mask.flat = mask
elif mask.ndim == 2:
mrec._mask.flat = [tuple(m) for m in mask]
else:
mrec.__setmask__(mask)
if _mask is not None:
mrec._mask[:] = _mask
return mrec
def _guessvartypes(arr):
"""
Tries to guess the dtypes of the str_ ndarray `arr`.
Guesses by testing element-wise conversion. Returns a list of dtypes.
The array is first converted to ndarray. If the array is 2D, the test
is performed on the first line. An exception is raised if the file is
3D or more.
"""
vartypes = []
arr = np.asarray(arr)
if arr.ndim == 2:
arr = arr[0]
elif arr.ndim > 2:
raise ValueError("The array should be 2D at most!")
# Start the conversion loop.
for f in arr:
try:
int(f)
except (ValueError, TypeError):
try:
float(f)
except (ValueError, TypeError):
try:
complex(f)
except (ValueError, TypeError):
vartypes.append(arr.dtype)
else:
vartypes.append(np.dtype(complex))
else:
vartypes.append(np.dtype(float))
else:
vartypes.append(np.dtype(int))
return vartypes
def openfile(fname):
"""
Opens the file handle of file `fname`.
"""
# A file handle
if hasattr(fname, 'readline'):
return fname
# Try to open the file and guess its type
try:
f = open(fname)
except FileNotFoundError as e:
raise FileNotFoundError(f"No such file: '{fname}'") from e
if f.readline()[:2] != "\\x":
f.seek(0, 0)
return f
f.close()
raise NotImplementedError("Wow, binary file")
def fromtextfile(fname, delimiter=None, commentchar='#', missingchar='',
varnames=None, vartypes=None,
*, delimitor=np._NoValue): # backwards compatibility
"""
Creates a mrecarray from data stored in the file `filename`.
Parameters
----------
fname : {file name/handle}
Handle of an opened file.
delimiter : {None, string}, optional
Alphanumeric character used to separate columns in the file.
If None, any (group of) white spacestring(s) will be used.
commentchar : {'#', string}, optional
Alphanumeric character used to mark the start of a comment.
missingchar : {'', string}, optional
String indicating missing data, and used to create the masks.
varnames : {None, sequence}, optional
Sequence of the variable names. If None, a list will be created from
the first non empty line of the file.
vartypes : {None, sequence}, optional
Sequence of the variables dtypes. If None, it will be estimated from
the first non-commented line.
Ultra simple: the varnames are in the header, one line"""
if delimitor is not np._NoValue:
if delimiter is not None:
raise TypeError("fromtextfile() got multiple values for argument "
"'delimiter'")
# NumPy 1.22.0, 2021-09-23
warnings.warn("The 'delimitor' keyword argument of "
"numpy.ma.mrecords.fromtextfile() is deprecated "
"since NumPy 1.22.0, use 'delimiter' instead.",
DeprecationWarning, stacklevel=2)
delimiter = delimitor
# Try to open the file.
ftext = openfile(fname)
# Get the first non-empty line as the varnames
while True:
line = ftext.readline()
firstline = line[:line.find(commentchar)].strip()
_varnames = firstline.split(delimiter)
if len(_varnames) > 1:
break
if varnames is None:
varnames = _varnames
# Get the data.
_variables = masked_array([line.strip().split(delimiter) for line in ftext
if line[0] != commentchar and len(line) > 1])
(_, nfields) = _variables.shape
ftext.close()
# Try to guess the dtype.
if vartypes is None:
vartypes = _guessvartypes(_variables[0])
else:
vartypes = [np.dtype(v) for v in vartypes]
if len(vartypes) != nfields:
msg = "Attempting to %i dtypes for %i fields!"
msg += " Reverting to default."
warnings.warn(msg % (len(vartypes), nfields), stacklevel=2)
vartypes = _guessvartypes(_variables[0])
# Construct the descriptor.
mdescr = [(n, f) for (n, f) in zip(varnames, vartypes)]
mfillv = [ma.default_fill_value(f) for f in vartypes]
# Get the data and the mask.
# We just need a list of masked_arrays. It's easier to create it like that:
_mask = (_variables.T == missingchar)
_datalist = [masked_array(a, mask=m, dtype=t, fill_value=f)
for (a, m, t, f) in zip(_variables.T, _mask, vartypes, mfillv)]
return fromarrays(_datalist, dtype=mdescr)
def addfield(mrecord, newfield, newfieldname=None):
"""Adds a new field to the masked record array
Uses `newfield` as data and `newfieldname` as name. If `newfieldname`
is None, the new field name is set to 'fi', where `i` is the number of
existing fields.
"""
_data = mrecord._data
_mask = mrecord._mask
if newfieldname is None or newfieldname in reserved_fields:
newfieldname = 'f%i' % len(_data.dtype)
newfield = ma.array(newfield)
# Get the new data.
# Create a new empty recarray
newdtype = np.dtype(_data.dtype.descr + [(newfieldname, newfield.dtype)])
newdata = recarray(_data.shape, newdtype)
# Add the existing field
[newdata.setfield(_data.getfield(*f), *f)
for f in _data.dtype.fields.values()]
# Add the new field
newdata.setfield(newfield._data, *newdata.dtype.fields[newfieldname])
newdata = newdata.view(MaskedRecords)
# Get the new mask
# Create a new empty recarray
newmdtype = np.dtype([(n, np.bool) for n in newdtype.names])
newmask = recarray(_data.shape, newmdtype)
# Add the old masks
[newmask.setfield(_mask.getfield(*f), *f)
for f in _mask.dtype.fields.values()]
# Add the mask of the new field
newmask.setfield(getmaskarray(newfield),
*newmask.dtype.fields[newfieldname])
newdata._mask = newmask
return newdata

View File

@ -0,0 +1,88 @@
from typing import Any, TypeVar
from numpy import dtype
from numpy.ma import MaskedArray
__all__: list[str]
_ShapeType_co = TypeVar("_ShapeType_co", covariant=True, bound=tuple[int, ...])
_DType_co = TypeVar("_DType_co", bound=dtype[Any], covariant=True)
class MaskedRecords(MaskedArray[_ShapeType_co, _DType_co]):
def __new__(
cls,
shape,
dtype=...,
buf=...,
offset=...,
strides=...,
formats=...,
names=...,
titles=...,
byteorder=...,
aligned=...,
mask=...,
hard_mask=...,
fill_value=...,
keep_mask=...,
copy=...,
**options,
): ...
_mask: Any
_fill_value: Any
@property
def _data(self): ...
@property
def _fieldmask(self): ...
def __array_finalize__(self, obj): ...
def __len__(self): ...
def __getattribute__(self, attr): ...
def __setattr__(self, attr, val): ...
def __getitem__(self, indx): ...
def __setitem__(self, indx, value): ...
def view(self, dtype=..., type=...): ...
def harden_mask(self): ...
def soften_mask(self): ...
def copy(self): ...
def tolist(self, fill_value=...): ...
def __reduce__(self): ...
mrecarray = MaskedRecords
def fromarrays(
arraylist,
dtype=...,
shape=...,
formats=...,
names=...,
titles=...,
aligned=...,
byteorder=...,
fill_value=...,
): ...
def fromrecords(
reclist,
dtype=...,
shape=...,
formats=...,
names=...,
titles=...,
aligned=...,
byteorder=...,
fill_value=...,
mask=...,
): ...
def fromtextfile(
fname,
delimiter=...,
commentchar=...,
missingchar=...,
varnames=...,
vartypes=...,
# NOTE: deprecated: NumPy 1.22.0, 2021-09-23
# delimitor=...,
): ...
def addfield(mrecord, newfield, newfieldname=...): ...

View File

@ -0,0 +1,40 @@
import pytest
import numpy as np
from numpy.ma import masked_array
from numpy.testing import assert_array_equal
def test_matrix_transpose_raises_error_for_1d():
msg = "matrix transpose with ndim < 2 is undefined"
ma_arr = masked_array(data=[1, 2, 3, 4, 5, 6],
mask=[1, 0, 1, 1, 1, 0])
with pytest.raises(ValueError, match=msg):
ma_arr.mT
def test_matrix_transpose_equals_transpose_2d():
ma_arr = masked_array(data=[[1, 2, 3], [4, 5, 6]],
mask=[[1, 0, 1], [1, 1, 0]])
assert_array_equal(ma_arr.T, ma_arr.mT)
ARRAY_SHAPES_TO_TEST = (
(5, 2),
(5, 2, 3),
(5, 2, 3, 4),
)
@pytest.mark.parametrize("shape", ARRAY_SHAPES_TO_TEST)
def test_matrix_transpose_equals_swapaxes(shape):
num_of_axes = len(shape)
vec = np.arange(shape[-1])
arr = np.broadcast_to(vec, shape)
rng = np.random.default_rng(42)
mask = rng.choice([0, 1], size=shape)
ma_arr = masked_array(data=arr, mask=mask)
tgt = np.swapaxes(arr, num_of_axes - 2, num_of_axes - 1)
assert_array_equal(tgt, ma_arr.mT)

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,84 @@
"""Test deprecation and future warnings.
"""
import pytest
import numpy as np
from numpy.testing import assert_warns
from numpy.ma.testutils import assert_equal
from numpy.ma.core import MaskedArrayFutureWarning
import io
import textwrap
class TestArgsort:
""" gh-8701 """
def _test_base(self, argsort, cls):
arr_0d = np.array(1).view(cls)
argsort(arr_0d)
arr_1d = np.array([1, 2, 3]).view(cls)
argsort(arr_1d)
# argsort has a bad default for >1d arrays
arr_2d = np.array([[1, 2], [3, 4]]).view(cls)
result = assert_warns(
np.ma.core.MaskedArrayFutureWarning, argsort, arr_2d)
assert_equal(result, argsort(arr_2d, axis=None))
# should be no warnings for explicitly specifying it
argsort(arr_2d, axis=None)
argsort(arr_2d, axis=-1)
def test_function_ndarray(self):
return self._test_base(np.ma.argsort, np.ndarray)
def test_function_maskedarray(self):
return self._test_base(np.ma.argsort, np.ma.MaskedArray)
def test_method(self):
return self._test_base(np.ma.MaskedArray.argsort, np.ma.MaskedArray)
class TestMinimumMaximum:
def test_axis_default(self):
# NumPy 1.13, 2017-05-06
data1d = np.ma.arange(6)
data2d = data1d.reshape(2, 3)
ma_min = np.ma.minimum.reduce
ma_max = np.ma.maximum.reduce
# check that the default axis is still None, but warns on 2d arrays
result = assert_warns(MaskedArrayFutureWarning, ma_max, data2d)
assert_equal(result, ma_max(data2d, axis=None))
result = assert_warns(MaskedArrayFutureWarning, ma_min, data2d)
assert_equal(result, ma_min(data2d, axis=None))
# no warnings on 1d, as both new and old defaults are equivalent
result = ma_min(data1d)
assert_equal(result, ma_min(data1d, axis=None))
assert_equal(result, ma_min(data1d, axis=0))
result = ma_max(data1d)
assert_equal(result, ma_max(data1d, axis=None))
assert_equal(result, ma_max(data1d, axis=0))
class TestFromtextfile:
def test_fromtextfile_delimitor(self):
# NumPy 1.22.0, 2021-09-23
textfile = io.StringIO(textwrap.dedent(
"""
A,B,C,D
'string 1';1;1.0;'mixed column'
'string 2';2;2.0;
'string 3';3;3.0;123
'string 4';4;4.0;3.14
"""
))
with pytest.warns(DeprecationWarning):
result = np.ma.mrecords.fromtextfile(textfile, delimitor=';')

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,493 @@
# pylint: disable-msg=W0611, W0612, W0511,R0201
"""Tests suite for mrecords.
:author: Pierre Gerard-Marchant
:contact: pierregm_at_uga_dot_edu
"""
import pickle
import numpy as np
import numpy.ma as ma
from numpy.ma import masked, nomask
from numpy.testing import temppath
from numpy._core.records import (
recarray, fromrecords as recfromrecords, fromarrays as recfromarrays
)
from numpy.ma.mrecords import (
MaskedRecords, mrecarray, fromarrays, fromtextfile, fromrecords,
addfield
)
from numpy.ma.testutils import (
assert_, assert_equal,
assert_equal_records,
)
class TestMRecords:
ilist = [1, 2, 3, 4, 5]
flist = [1.1, 2.2, 3.3, 4.4, 5.5]
slist = [b'one', b'two', b'three', b'four', b'five']
ddtype = [('a', int), ('b', float), ('c', '|S8')]
mask = [0, 1, 0, 0, 1]
base = ma.array(list(zip(ilist, flist, slist)), mask=mask, dtype=ddtype)
def test_byview(self):
# Test creation by view
base = self.base
mbase = base.view(mrecarray)
assert_equal(mbase.recordmask, base.recordmask)
assert_equal_records(mbase._mask, base._mask)
assert_(isinstance(mbase._data, recarray))
assert_equal_records(mbase._data, base._data.view(recarray))
for field in ('a', 'b', 'c'):
assert_equal(base[field], mbase[field])
assert_equal_records(mbase.view(mrecarray), mbase)
def test_get(self):
# Tests fields retrieval
base = self.base.copy()
mbase = base.view(mrecarray)
# As fields..........
for field in ('a', 'b', 'c'):
assert_equal(getattr(mbase, field), mbase[field])
assert_equal(base[field], mbase[field])
# as elements .......
mbase_first = mbase[0]
assert_(isinstance(mbase_first, mrecarray))
assert_equal(mbase_first.dtype, mbase.dtype)
assert_equal(mbase_first.tolist(), (1, 1.1, b'one'))
# Used to be mask, now it's recordmask
assert_equal(mbase_first.recordmask, nomask)
assert_equal(mbase_first._mask.item(), (False, False, False))
assert_equal(mbase_first['a'], mbase['a'][0])
mbase_last = mbase[-1]
assert_(isinstance(mbase_last, mrecarray))
assert_equal(mbase_last.dtype, mbase.dtype)
assert_equal(mbase_last.tolist(), (None, None, None))
# Used to be mask, now it's recordmask
assert_equal(mbase_last.recordmask, True)
assert_equal(mbase_last._mask.item(), (True, True, True))
assert_equal(mbase_last['a'], mbase['a'][-1])
assert_(mbase_last['a'] is masked)
# as slice ..........
mbase_sl = mbase[:2]
assert_(isinstance(mbase_sl, mrecarray))
assert_equal(mbase_sl.dtype, mbase.dtype)
# Used to be mask, now it's recordmask
assert_equal(mbase_sl.recordmask, [0, 1])
assert_equal_records(mbase_sl.mask,
np.array([(False, False, False),
(True, True, True)],
dtype=mbase._mask.dtype))
assert_equal_records(mbase_sl, base[:2].view(mrecarray))
for field in ('a', 'b', 'c'):
assert_equal(getattr(mbase_sl, field), base[:2][field])
def test_set_fields(self):
# Tests setting fields.
base = self.base.copy()
mbase = base.view(mrecarray)
mbase = mbase.copy()
mbase.fill_value = (999999, 1e20, 'N/A')
# Change the data, the mask should be conserved
mbase.a._data[:] = 5
assert_equal(mbase['a']._data, [5, 5, 5, 5, 5])
assert_equal(mbase['a']._mask, [0, 1, 0, 0, 1])
# Change the elements, and the mask will follow
mbase.a = 1
assert_equal(mbase['a']._data, [1]*5)
assert_equal(ma.getmaskarray(mbase['a']), [0]*5)
# Use to be _mask, now it's recordmask
assert_equal(mbase.recordmask, [False]*5)
assert_equal(mbase._mask.tolist(),
np.array([(0, 0, 0),
(0, 1, 1),
(0, 0, 0),
(0, 0, 0),
(0, 1, 1)],
dtype=bool))
# Set a field to mask ........................
mbase.c = masked
# Use to be mask, and now it's still mask !
assert_equal(mbase.c.mask, [1]*5)
assert_equal(mbase.c.recordmask, [1]*5)
assert_equal(ma.getmaskarray(mbase['c']), [1]*5)
assert_equal(ma.getdata(mbase['c']), [b'N/A']*5)
assert_equal(mbase._mask.tolist(),
np.array([(0, 0, 1),
(0, 1, 1),
(0, 0, 1),
(0, 0, 1),
(0, 1, 1)],
dtype=bool))
# Set fields by slices .......................
mbase = base.view(mrecarray).copy()
mbase.a[3:] = 5
assert_equal(mbase.a, [1, 2, 3, 5, 5])
assert_equal(mbase.a._mask, [0, 1, 0, 0, 0])
mbase.b[3:] = masked
assert_equal(mbase.b, base['b'])
assert_equal(mbase.b._mask, [0, 1, 0, 1, 1])
# Set fields globally..........................
ndtype = [('alpha', '|S1'), ('num', int)]
data = ma.array([('a', 1), ('b', 2), ('c', 3)], dtype=ndtype)
rdata = data.view(MaskedRecords)
val = ma.array([10, 20, 30], mask=[1, 0, 0])
rdata['num'] = val
assert_equal(rdata.num, val)
assert_equal(rdata.num.mask, [1, 0, 0])
def test_set_fields_mask(self):
# Tests setting the mask of a field.
base = self.base.copy()
# This one has already a mask....
mbase = base.view(mrecarray)
mbase['a'][-2] = masked
assert_equal(mbase.a, [1, 2, 3, 4, 5])
assert_equal(mbase.a._mask, [0, 1, 0, 1, 1])
# This one has not yet
mbase = fromarrays([np.arange(5), np.random.rand(5)],
dtype=[('a', int), ('b', float)])
mbase['a'][-2] = masked
assert_equal(mbase.a, [0, 1, 2, 3, 4])
assert_equal(mbase.a._mask, [0, 0, 0, 1, 0])
def test_set_mask(self):
base = self.base.copy()
mbase = base.view(mrecarray)
# Set the mask to True .......................
mbase.mask = masked
assert_equal(ma.getmaskarray(mbase['b']), [1]*5)
assert_equal(mbase['a']._mask, mbase['b']._mask)
assert_equal(mbase['a']._mask, mbase['c']._mask)
assert_equal(mbase._mask.tolist(),
np.array([(1, 1, 1)]*5, dtype=bool))
# Delete the mask ............................
mbase.mask = nomask
assert_equal(ma.getmaskarray(mbase['c']), [0]*5)
assert_equal(mbase._mask.tolist(),
np.array([(0, 0, 0)]*5, dtype=bool))
def test_set_mask_fromarray(self):
base = self.base.copy()
mbase = base.view(mrecarray)
# Sets the mask w/ an array
mbase.mask = [1, 0, 0, 0, 1]
assert_equal(mbase.a.mask, [1, 0, 0, 0, 1])
assert_equal(mbase.b.mask, [1, 0, 0, 0, 1])
assert_equal(mbase.c.mask, [1, 0, 0, 0, 1])
# Yay, once more !
mbase.mask = [0, 0, 0, 0, 1]
assert_equal(mbase.a.mask, [0, 0, 0, 0, 1])
assert_equal(mbase.b.mask, [0, 0, 0, 0, 1])
assert_equal(mbase.c.mask, [0, 0, 0, 0, 1])
def test_set_mask_fromfields(self):
mbase = self.base.copy().view(mrecarray)
nmask = np.array(
[(0, 1, 0), (0, 1, 0), (1, 0, 1), (1, 0, 1), (0, 0, 0)],
dtype=[('a', bool), ('b', bool), ('c', bool)])
mbase.mask = nmask
assert_equal(mbase.a.mask, [0, 0, 1, 1, 0])
assert_equal(mbase.b.mask, [1, 1, 0, 0, 0])
assert_equal(mbase.c.mask, [0, 0, 1, 1, 0])
# Reinitialize and redo
mbase.mask = False
mbase.fieldmask = nmask
assert_equal(mbase.a.mask, [0, 0, 1, 1, 0])
assert_equal(mbase.b.mask, [1, 1, 0, 0, 0])
assert_equal(mbase.c.mask, [0, 0, 1, 1, 0])
def test_set_elements(self):
base = self.base.copy()
# Set an element to mask .....................
mbase = base.view(mrecarray).copy()
mbase[-2] = masked
assert_equal(
mbase._mask.tolist(),
np.array([(0, 0, 0), (1, 1, 1), (0, 0, 0), (1, 1, 1), (1, 1, 1)],
dtype=bool))
# Used to be mask, now it's recordmask!
assert_equal(mbase.recordmask, [0, 1, 0, 1, 1])
# Set slices .................................
mbase = base.view(mrecarray).copy()
mbase[:2] = (5, 5, 5)
assert_equal(mbase.a._data, [5, 5, 3, 4, 5])
assert_equal(mbase.a._mask, [0, 0, 0, 0, 1])
assert_equal(mbase.b._data, [5., 5., 3.3, 4.4, 5.5])
assert_equal(mbase.b._mask, [0, 0, 0, 0, 1])
assert_equal(mbase.c._data,
[b'5', b'5', b'three', b'four', b'five'])
assert_equal(mbase.b._mask, [0, 0, 0, 0, 1])
mbase = base.view(mrecarray).copy()
mbase[:2] = masked
assert_equal(mbase.a._data, [1, 2, 3, 4, 5])
assert_equal(mbase.a._mask, [1, 1, 0, 0, 1])
assert_equal(mbase.b._data, [1.1, 2.2, 3.3, 4.4, 5.5])
assert_equal(mbase.b._mask, [1, 1, 0, 0, 1])
assert_equal(mbase.c._data,
[b'one', b'two', b'three', b'four', b'five'])
assert_equal(mbase.b._mask, [1, 1, 0, 0, 1])
def test_setslices_hardmask(self):
# Tests setting slices w/ hardmask.
base = self.base.copy()
mbase = base.view(mrecarray)
mbase.harden_mask()
try:
mbase[-2:] = (5, 5, 5)
assert_equal(mbase.a._data, [1, 2, 3, 5, 5])
assert_equal(mbase.b._data, [1.1, 2.2, 3.3, 5, 5.5])
assert_equal(mbase.c._data,
[b'one', b'two', b'three', b'5', b'five'])
assert_equal(mbase.a._mask, [0, 1, 0, 0, 1])
assert_equal(mbase.b._mask, mbase.a._mask)
assert_equal(mbase.b._mask, mbase.c._mask)
except NotImplementedError:
# OK, not implemented yet...
pass
except AssertionError:
raise
else:
raise Exception("Flexible hard masks should be supported !")
# Not using a tuple should crash
try:
mbase[-2:] = 3
except (NotImplementedError, TypeError):
pass
else:
raise TypeError("Should have expected a readable buffer object!")
def test_hardmask(self):
# Test hardmask
base = self.base.copy()
mbase = base.view(mrecarray)
mbase.harden_mask()
assert_(mbase._hardmask)
mbase.mask = nomask
assert_equal_records(mbase._mask, base._mask)
mbase.soften_mask()
assert_(not mbase._hardmask)
mbase.mask = nomask
# So, the mask of a field is no longer set to nomask...
assert_equal_records(mbase._mask,
ma.make_mask_none(base.shape, base.dtype))
assert_(ma.make_mask(mbase['b']._mask) is nomask)
assert_equal(mbase['a']._mask, mbase['b']._mask)
def test_pickling(self):
# Test pickling
base = self.base.copy()
mrec = base.view(mrecarray)
for proto in range(2, pickle.HIGHEST_PROTOCOL + 1):
_ = pickle.dumps(mrec, protocol=proto)
mrec_ = pickle.loads(_)
assert_equal(mrec_.dtype, mrec.dtype)
assert_equal_records(mrec_._data, mrec._data)
assert_equal(mrec_._mask, mrec._mask)
assert_equal_records(mrec_._mask, mrec._mask)
def test_filled(self):
# Test filling the array
_a = ma.array([1, 2, 3], mask=[0, 0, 1], dtype=int)
_b = ma.array([1.1, 2.2, 3.3], mask=[0, 0, 1], dtype=float)
_c = ma.array(['one', 'two', 'three'], mask=[0, 0, 1], dtype='|S8')
ddtype = [('a', int), ('b', float), ('c', '|S8')]
mrec = fromarrays([_a, _b, _c], dtype=ddtype,
fill_value=(99999, 99999., 'N/A'))
mrecfilled = mrec.filled()
assert_equal(mrecfilled['a'], np.array((1, 2, 99999), dtype=int))
assert_equal(mrecfilled['b'], np.array((1.1, 2.2, 99999.),
dtype=float))
assert_equal(mrecfilled['c'], np.array(('one', 'two', 'N/A'),
dtype='|S8'))
def test_tolist(self):
# Test tolist.
_a = ma.array([1, 2, 3], mask=[0, 0, 1], dtype=int)
_b = ma.array([1.1, 2.2, 3.3], mask=[0, 0, 1], dtype=float)
_c = ma.array(['one', 'two', 'three'], mask=[1, 0, 0], dtype='|S8')
ddtype = [('a', int), ('b', float), ('c', '|S8')]
mrec = fromarrays([_a, _b, _c], dtype=ddtype,
fill_value=(99999, 99999., 'N/A'))
assert_equal(mrec.tolist(),
[(1, 1.1, None), (2, 2.2, b'two'),
(None, None, b'three')])
def test_withnames(self):
# Test the creation w/ format and names
x = mrecarray(1, formats=float, names='base')
x[0]['base'] = 10
assert_equal(x['base'][0], 10)
def test_exotic_formats(self):
# Test that 'exotic' formats are processed properly
easy = mrecarray(1, dtype=[('i', int), ('s', '|S8'), ('f', float)])
easy[0] = masked
assert_equal(easy.filled(1).item(), (1, b'1', 1.))
solo = mrecarray(1, dtype=[('f0', '<f8', (2, 2))])
solo[0] = masked
assert_equal(solo.filled(1).item(),
np.array((1,), dtype=solo.dtype).item())
mult = mrecarray(2, dtype="i4, (2,3)float, float")
mult[0] = masked
mult[1] = (1, 1, 1)
mult.filled(0)
assert_equal_records(mult.filled(0),
np.array([(0, 0, 0), (1, 1, 1)],
dtype=mult.dtype))
class TestView:
def setup_method(self):
(a, b) = (np.arange(10), np.random.rand(10))
ndtype = [('a', float), ('b', float)]
arr = np.array(list(zip(a, b)), dtype=ndtype)
mrec = fromarrays([a, b], dtype=ndtype, fill_value=(-9., -99.))
mrec.mask[3] = (False, True)
self.data = (mrec, a, b, arr)
def test_view_by_itself(self):
(mrec, a, b, arr) = self.data
test = mrec.view()
assert_(isinstance(test, MaskedRecords))
assert_equal_records(test, mrec)
assert_equal_records(test._mask, mrec._mask)
def test_view_simple_dtype(self):
(mrec, a, b, arr) = self.data
ntype = (float, 2)
test = mrec.view(ntype)
assert_(isinstance(test, ma.MaskedArray))
assert_equal(test, np.array(list(zip(a, b)), dtype=float))
assert_(test[3, 1] is ma.masked)
def test_view_flexible_type(self):
(mrec, a, b, arr) = self.data
alttype = [('A', float), ('B', float)]
test = mrec.view(alttype)
assert_(isinstance(test, MaskedRecords))
assert_equal_records(test, arr.view(alttype))
assert_(test['B'][3] is masked)
assert_equal(test.dtype, np.dtype(alttype))
assert_(test._fill_value is None)
##############################################################################
class TestMRecordsImport:
_a = ma.array([1, 2, 3], mask=[0, 0, 1], dtype=int)
_b = ma.array([1.1, 2.2, 3.3], mask=[0, 0, 1], dtype=float)
_c = ma.array([b'one', b'two', b'three'],
mask=[0, 0, 1], dtype='|S8')
ddtype = [('a', int), ('b', float), ('c', '|S8')]
mrec = fromarrays([_a, _b, _c], dtype=ddtype,
fill_value=(b'99999', b'99999.',
b'N/A'))
nrec = recfromarrays((_a._data, _b._data, _c._data), dtype=ddtype)
data = (mrec, nrec, ddtype)
def test_fromarrays(self):
_a = ma.array([1, 2, 3], mask=[0, 0, 1], dtype=int)
_b = ma.array([1.1, 2.2, 3.3], mask=[0, 0, 1], dtype=float)
_c = ma.array(['one', 'two', 'three'], mask=[0, 0, 1], dtype='|S8')
(mrec, nrec, _) = self.data
for (f, l) in zip(('a', 'b', 'c'), (_a, _b, _c)):
assert_equal(getattr(mrec, f)._mask, l._mask)
# One record only
_x = ma.array([1, 1.1, 'one'], mask=[1, 0, 0], dtype=object)
assert_equal_records(fromarrays(_x, dtype=mrec.dtype), mrec[0])
def test_fromrecords(self):
# Test construction from records.
(mrec, nrec, ddtype) = self.data
#......
palist = [(1, 'abc', 3.7000002861022949, 0),
(2, 'xy', 6.6999998092651367, 1),
(0, ' ', 0.40000000596046448, 0)]
pa = recfromrecords(palist, names='c1, c2, c3, c4')
mpa = fromrecords(palist, names='c1, c2, c3, c4')
assert_equal_records(pa, mpa)
#.....
_mrec = fromrecords(nrec)
assert_equal(_mrec.dtype, mrec.dtype)
for field in _mrec.dtype.names:
assert_equal(getattr(_mrec, field), getattr(mrec._data, field))
_mrec = fromrecords(nrec.tolist(), names='c1,c2,c3')
assert_equal(_mrec.dtype, [('c1', int), ('c2', float), ('c3', '|S5')])
for (f, n) in zip(('c1', 'c2', 'c3'), ('a', 'b', 'c')):
assert_equal(getattr(_mrec, f), getattr(mrec._data, n))
_mrec = fromrecords(mrec)
assert_equal(_mrec.dtype, mrec.dtype)
assert_equal_records(_mrec._data, mrec.filled())
assert_equal_records(_mrec._mask, mrec._mask)
def test_fromrecords_wmask(self):
# Tests construction from records w/ mask.
(mrec, nrec, ddtype) = self.data
_mrec = fromrecords(nrec.tolist(), dtype=ddtype, mask=[0, 1, 0,])
assert_equal_records(_mrec._data, mrec._data)
assert_equal(_mrec._mask.tolist(), [(0, 0, 0), (1, 1, 1), (0, 0, 0)])
_mrec = fromrecords(nrec.tolist(), dtype=ddtype, mask=True)
assert_equal_records(_mrec._data, mrec._data)
assert_equal(_mrec._mask.tolist(), [(1, 1, 1), (1, 1, 1), (1, 1, 1)])
_mrec = fromrecords(nrec.tolist(), dtype=ddtype, mask=mrec._mask)
assert_equal_records(_mrec._data, mrec._data)
assert_equal(_mrec._mask.tolist(), mrec._mask.tolist())
_mrec = fromrecords(nrec.tolist(), dtype=ddtype,
mask=mrec._mask.tolist())
assert_equal_records(_mrec._data, mrec._data)
assert_equal(_mrec._mask.tolist(), mrec._mask.tolist())
def test_fromtextfile(self):
# Tests reading from a text file.
fcontent = (
"""#
'One (S)','Two (I)','Three (F)','Four (M)','Five (-)','Six (C)'
'strings',1,1.0,'mixed column',,1
'with embedded "double quotes"',2,2.0,1.0,,1
'strings',3,3.0E5,3,,1
'strings',4,-1e-10,,,1
""")
with temppath() as path:
with open(path, 'w') as f:
f.write(fcontent)
mrectxt = fromtextfile(path, delimiter=',', varnames='ABCDEFG')
assert_(isinstance(mrectxt, MaskedRecords))
assert_equal(mrectxt.F, [1, 1, 1, 1])
assert_equal(mrectxt.E._mask, [1, 1, 1, 1])
assert_equal(mrectxt.C, [1, 2, 3.e+5, -1e-10])
def test_addfield(self):
# Tests addfield
(mrec, nrec, ddtype) = self.data
(d, m) = ([100, 200, 300], [1, 0, 0])
mrec = addfield(mrec, ma.array(d, mask=m))
assert_equal(mrec.f3, d)
assert_equal(mrec.f3._mask, m)
def test_record_array_with_object_field():
# Trac #1839
y = ma.masked_array(
[(1, '2'), (3, '4')],
mask=[(0, 0), (0, 1)],
dtype=[('a', int), ('b', object)])
# getting an item used to fail
y[1]

View File

@ -0,0 +1,876 @@
from functools import reduce
import pickle
import pytest
import numpy as np
import numpy._core.umath as umath
import numpy._core.fromnumeric as fromnumeric
from numpy.testing import (
assert_, assert_raises, assert_equal,
)
from numpy.ma import (
MaskType, MaskedArray, absolute, add, all, allclose, allequal, alltrue,
arange, arccos, arcsin, arctan, arctan2, array, average, choose,
concatenate, conjugate, cos, cosh, count, divide, equal, exp, filled,
getmask, greater, greater_equal, inner, isMaskedArray, less,
less_equal, log, log10, make_mask, masked, masked_array, masked_equal,
masked_greater, masked_greater_equal, masked_inside, masked_less,
masked_less_equal, masked_not_equal, masked_outside,
masked_print_option, masked_values, masked_where, maximum, minimum,
multiply, nomask, nonzero, not_equal, ones, outer, product, put, ravel,
repeat, resize, shape, sin, sinh, sometrue, sort, sqrt, subtract, sum,
take, tan, tanh, transpose, where, zeros,
)
pi = np.pi
def eq(v, w, msg=''):
result = allclose(v, w)
if not result:
print(f'Not eq:{msg}\n{v}\n----{w}')
return result
class TestMa:
def setup_method(self):
x = np.array([1., 1., 1., -2., pi/2.0, 4., 5., -10., 10., 1., 2., 3.])
y = np.array([5., 0., 3., 2., -1., -4., 0., -10., 10., 1., 0., 3.])
a10 = 10.
m1 = [1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]
m2 = [0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1]
xm = array(x, mask=m1)
ym = array(y, mask=m2)
z = np.array([-.5, 0., .5, .8])
zm = array(z, mask=[0, 1, 0, 0])
xf = np.where(m1, 1e+20, x)
s = x.shape
xm.set_fill_value(1e+20)
self.d = (x, y, a10, m1, m2, xm, ym, z, zm, xf, s)
def test_testBasic1d(self):
# Test of basic array creation and properties in 1 dimension.
(x, y, a10, m1, m2, xm, ym, z, zm, xf, s) = self.d
assert_(not isMaskedArray(x))
assert_(isMaskedArray(xm))
assert_equal(shape(xm), s)
assert_equal(xm.shape, s)
assert_equal(xm.dtype, x.dtype)
assert_equal(xm.size, reduce(lambda x, y:x * y, s))
assert_equal(count(xm), len(m1) - reduce(lambda x, y:x + y, m1))
assert_(eq(xm, xf))
assert_(eq(filled(xm, 1.e20), xf))
assert_(eq(x, xm))
@pytest.mark.parametrize("s", [(4, 3), (6, 2)])
def test_testBasic2d(self, s):
# Test of basic array creation and properties in 2 dimensions.
(x, y, a10, m1, m2, xm, ym, z, zm, xf, s) = self.d
x.shape = s
y.shape = s
xm.shape = s
ym.shape = s
xf.shape = s
assert_(not isMaskedArray(x))
assert_(isMaskedArray(xm))
assert_equal(shape(xm), s)
assert_equal(xm.shape, s)
assert_equal(xm.size, reduce(lambda x, y: x * y, s))
assert_equal(count(xm), len(m1) - reduce(lambda x, y: x + y, m1))
assert_(eq(xm, xf))
assert_(eq(filled(xm, 1.e20), xf))
assert_(eq(x, xm))
def test_testArithmetic(self):
# Test of basic arithmetic.
(x, y, a10, m1, m2, xm, ym, z, zm, xf, s) = self.d
a2d = array([[1, 2], [0, 4]])
a2dm = masked_array(a2d, [[0, 0], [1, 0]])
assert_(eq(a2d * a2d, a2d * a2dm))
assert_(eq(a2d + a2d, a2d + a2dm))
assert_(eq(a2d - a2d, a2d - a2dm))
for s in [(12,), (4, 3), (2, 6)]:
x = x.reshape(s)
y = y.reshape(s)
xm = xm.reshape(s)
ym = ym.reshape(s)
xf = xf.reshape(s)
assert_(eq(-x, -xm))
assert_(eq(x + y, xm + ym))
assert_(eq(x - y, xm - ym))
assert_(eq(x * y, xm * ym))
with np.errstate(divide='ignore', invalid='ignore'):
assert_(eq(x / y, xm / ym))
assert_(eq(a10 + y, a10 + ym))
assert_(eq(a10 - y, a10 - ym))
assert_(eq(a10 * y, a10 * ym))
with np.errstate(divide='ignore', invalid='ignore'):
assert_(eq(a10 / y, a10 / ym))
assert_(eq(x + a10, xm + a10))
assert_(eq(x - a10, xm - a10))
assert_(eq(x * a10, xm * a10))
assert_(eq(x / a10, xm / a10))
assert_(eq(x ** 2, xm ** 2))
assert_(eq(abs(x) ** 2.5, abs(xm) ** 2.5))
assert_(eq(x ** y, xm ** ym))
assert_(eq(np.add(x, y), add(xm, ym)))
assert_(eq(np.subtract(x, y), subtract(xm, ym)))
assert_(eq(np.multiply(x, y), multiply(xm, ym)))
with np.errstate(divide='ignore', invalid='ignore'):
assert_(eq(np.divide(x, y), divide(xm, ym)))
def test_testMixedArithmetic(self):
na = np.array([1])
ma = array([1])
assert_(isinstance(na + ma, MaskedArray))
assert_(isinstance(ma + na, MaskedArray))
def test_testUfuncs1(self):
# Test various functions such as sin, cos.
(x, y, a10, m1, m2, xm, ym, z, zm, xf, s) = self.d
assert_(eq(np.cos(x), cos(xm)))
assert_(eq(np.cosh(x), cosh(xm)))
assert_(eq(np.sin(x), sin(xm)))
assert_(eq(np.sinh(x), sinh(xm)))
assert_(eq(np.tan(x), tan(xm)))
assert_(eq(np.tanh(x), tanh(xm)))
with np.errstate(divide='ignore', invalid='ignore'):
assert_(eq(np.sqrt(abs(x)), sqrt(xm)))
assert_(eq(np.log(abs(x)), log(xm)))
assert_(eq(np.log10(abs(x)), log10(xm)))
assert_(eq(np.exp(x), exp(xm)))
assert_(eq(np.arcsin(z), arcsin(zm)))
assert_(eq(np.arccos(z), arccos(zm)))
assert_(eq(np.arctan(z), arctan(zm)))
assert_(eq(np.arctan2(x, y), arctan2(xm, ym)))
assert_(eq(np.absolute(x), absolute(xm)))
assert_(eq(np.equal(x, y), equal(xm, ym)))
assert_(eq(np.not_equal(x, y), not_equal(xm, ym)))
assert_(eq(np.less(x, y), less(xm, ym)))
assert_(eq(np.greater(x, y), greater(xm, ym)))
assert_(eq(np.less_equal(x, y), less_equal(xm, ym)))
assert_(eq(np.greater_equal(x, y), greater_equal(xm, ym)))
assert_(eq(np.conjugate(x), conjugate(xm)))
assert_(eq(np.concatenate((x, y)), concatenate((xm, ym))))
assert_(eq(np.concatenate((x, y)), concatenate((x, y))))
assert_(eq(np.concatenate((x, y)), concatenate((xm, y))))
assert_(eq(np.concatenate((x, y, x)), concatenate((x, ym, x))))
def test_xtestCount(self):
# Test count
ott = array([0., 1., 2., 3.], mask=[1, 0, 0, 0])
assert_(count(ott).dtype.type is np.intp)
assert_equal(3, count(ott))
assert_equal(1, count(1))
assert_(eq(0, array(1, mask=[1])))
ott = ott.reshape((2, 2))
assert_(count(ott).dtype.type is np.intp)
assert_(isinstance(count(ott, 0), np.ndarray))
assert_(count(ott).dtype.type is np.intp)
assert_(eq(3, count(ott)))
assert_(getmask(count(ott, 0)) is nomask)
assert_(eq([1, 2], count(ott, 0)))
def test_testMinMax(self):
# Test minimum and maximum.
(x, y, a10, m1, m2, xm, ym, z, zm, xf, s) = self.d
xr = np.ravel(x) # max doesn't work if shaped
xmr = ravel(xm)
# true because of careful selection of data
assert_(eq(max(xr), maximum.reduce(xmr)))
assert_(eq(min(xr), minimum.reduce(xmr)))
def test_testAddSumProd(self):
# Test add, sum, product.
(x, y, a10, m1, m2, xm, ym, z, zm, xf, s) = self.d
assert_(eq(np.add.reduce(x), add.reduce(x)))
assert_(eq(np.add.accumulate(x), add.accumulate(x)))
assert_(eq(4, sum(array(4), axis=0)))
assert_(eq(4, sum(array(4), axis=0)))
assert_(eq(np.sum(x, axis=0), sum(x, axis=0)))
assert_(eq(np.sum(filled(xm, 0), axis=0), sum(xm, axis=0)))
assert_(eq(np.sum(x, 0), sum(x, 0)))
assert_(eq(np.prod(x, axis=0), product(x, axis=0)))
assert_(eq(np.prod(x, 0), product(x, 0)))
assert_(eq(np.prod(filled(xm, 1), axis=0),
product(xm, axis=0)))
if len(s) > 1:
assert_(eq(np.concatenate((x, y), 1),
concatenate((xm, ym), 1)))
assert_(eq(np.add.reduce(x, 1), add.reduce(x, 1)))
assert_(eq(np.sum(x, 1), sum(x, 1)))
assert_(eq(np.prod(x, 1), product(x, 1)))
def test_testCI(self):
# Test of conversions and indexing
x1 = np.array([1, 2, 4, 3])
x2 = array(x1, mask=[1, 0, 0, 0])
x3 = array(x1, mask=[0, 1, 0, 1])
x4 = array(x1)
# test conversion to strings
str(x2) # raises?
repr(x2) # raises?
assert_(eq(np.sort(x1), sort(x2, fill_value=0)))
# tests of indexing
assert_(type(x2[1]) is type(x1[1]))
assert_(x1[1] == x2[1])
assert_(x2[0] is masked)
assert_(eq(x1[2], x2[2]))
assert_(eq(x1[2:5], x2[2:5]))
assert_(eq(x1[:], x2[:]))
assert_(eq(x1[1:], x3[1:]))
x1[2] = 9
x2[2] = 9
assert_(eq(x1, x2))
x1[1:3] = 99
x2[1:3] = 99
assert_(eq(x1, x2))
x2[1] = masked
assert_(eq(x1, x2))
x2[1:3] = masked
assert_(eq(x1, x2))
x2[:] = x1
x2[1] = masked
assert_(allequal(getmask(x2), array([0, 1, 0, 0])))
x3[:] = masked_array([1, 2, 3, 4], [0, 1, 1, 0])
assert_(allequal(getmask(x3), array([0, 1, 1, 0])))
x4[:] = masked_array([1, 2, 3, 4], [0, 1, 1, 0])
assert_(allequal(getmask(x4), array([0, 1, 1, 0])))
assert_(allequal(x4, array([1, 2, 3, 4])))
x1 = np.arange(5) * 1.0
x2 = masked_values(x1, 3.0)
assert_(eq(x1, x2))
assert_(allequal(array([0, 0, 0, 1, 0], MaskType), x2.mask))
assert_(eq(3.0, x2.fill_value))
x1 = array([1, 'hello', 2, 3], object)
x2 = np.array([1, 'hello', 2, 3], object)
s1 = x1[1]
s2 = x2[1]
assert_equal(type(s2), str)
assert_equal(type(s1), str)
assert_equal(s1, s2)
assert_(x1[1:1].shape == (0,))
def test_testCopySize(self):
# Tests of some subtle points of copying and sizing.
n = [0, 0, 1, 0, 0]
m = make_mask(n)
m2 = make_mask(m)
assert_(m is m2)
m3 = make_mask(m, copy=True)
assert_(m is not m3)
x1 = np.arange(5)
y1 = array(x1, mask=m)
assert_(y1._data is not x1)
assert_(allequal(x1, y1._data))
assert_(y1._mask is m)
y1a = array(y1, copy=0)
# For copy=False, one might expect that the array would just
# passed on, i.e., that it would be "is" instead of "==".
# See gh-4043 for discussion.
assert_(y1a._mask.__array_interface__ ==
y1._mask.__array_interface__)
y2 = array(x1, mask=m3, copy=0)
assert_(y2._mask is m3)
assert_(y2[2] is masked)
y2[2] = 9
assert_(y2[2] is not masked)
assert_(y2._mask is m3)
assert_(allequal(y2.mask, 0))
y2a = array(x1, mask=m, copy=1)
assert_(y2a._mask is not m)
assert_(y2a[2] is masked)
y2a[2] = 9
assert_(y2a[2] is not masked)
assert_(y2a._mask is not m)
assert_(allequal(y2a.mask, 0))
y3 = array(x1 * 1.0, mask=m)
assert_(filled(y3).dtype is (x1 * 1.0).dtype)
x4 = arange(4)
x4[2] = masked
y4 = resize(x4, (8,))
assert_(eq(concatenate([x4, x4]), y4))
assert_(eq(getmask(y4), [0, 0, 1, 0, 0, 0, 1, 0]))
y5 = repeat(x4, (2, 2, 2, 2), axis=0)
assert_(eq(y5, [0, 0, 1, 1, 2, 2, 3, 3]))
y6 = repeat(x4, 2, axis=0)
assert_(eq(y5, y6))
def test_testPut(self):
# Test of put
d = arange(5)
n = [0, 0, 0, 1, 1]
m = make_mask(n)
m2 = m.copy()
x = array(d, mask=m)
assert_(x[3] is masked)
assert_(x[4] is masked)
x[[1, 4]] = [10, 40]
assert_(x._mask is m)
assert_(x[3] is masked)
assert_(x[4] is not masked)
assert_(eq(x, [0, 10, 2, -1, 40]))
x = array(d, mask=m2, copy=True)
x.put([0, 1, 2], [-1, 100, 200])
assert_(x._mask is not m2)
assert_(x[3] is masked)
assert_(x[4] is masked)
assert_(eq(x, [-1, 100, 200, 0, 0]))
def test_testPut2(self):
# Test of put
d = arange(5)
x = array(d, mask=[0, 0, 0, 0, 0])
z = array([10, 40], mask=[1, 0])
assert_(x[2] is not masked)
assert_(x[3] is not masked)
x[2:4] = z
assert_(x[2] is masked)
assert_(x[3] is not masked)
assert_(eq(x, [0, 1, 10, 40, 4]))
d = arange(5)
x = array(d, mask=[0, 0, 0, 0, 0])
y = x[2:4]
z = array([10, 40], mask=[1, 0])
assert_(x[2] is not masked)
assert_(x[3] is not masked)
y[:] = z
assert_(y[0] is masked)
assert_(y[1] is not masked)
assert_(eq(y, [10, 40]))
assert_(x[2] is masked)
assert_(x[3] is not masked)
assert_(eq(x, [0, 1, 10, 40, 4]))
def test_testMaPut(self):
(x, y, a10, m1, m2, xm, ym, z, zm, xf, s) = self.d
m = [1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1]
i = np.nonzero(m)[0]
put(ym, i, zm)
assert_(all(take(ym, i, axis=0) == zm))
def test_testOddFeatures(self):
# Test of other odd features
x = arange(20)
x = x.reshape(4, 5)
x.flat[5] = 12
assert_(x[1, 0] == 12)
z = x + 10j * x
assert_(eq(z.real, x))
assert_(eq(z.imag, 10 * x))
assert_(eq((z * conjugate(z)).real, 101 * x * x))
z.imag[...] = 0.0
x = arange(10)
x[3] = masked
assert_(str(x[3]) == str(masked))
c = x >= 8
assert_(count(where(c, masked, masked)) == 0)
assert_(shape(where(c, masked, masked)) == c.shape)
z = where(c, x, masked)
assert_(z.dtype is x.dtype)
assert_(z[3] is masked)
assert_(z[4] is masked)
assert_(z[7] is masked)
assert_(z[8] is not masked)
assert_(z[9] is not masked)
assert_(eq(x, z))
z = where(c, masked, x)
assert_(z.dtype is x.dtype)
assert_(z[3] is masked)
assert_(z[4] is not masked)
assert_(z[7] is not masked)
assert_(z[8] is masked)
assert_(z[9] is masked)
z = masked_where(c, x)
assert_(z.dtype is x.dtype)
assert_(z[3] is masked)
assert_(z[4] is not masked)
assert_(z[7] is not masked)
assert_(z[8] is masked)
assert_(z[9] is masked)
assert_(eq(x, z))
x = array([1., 2., 3., 4., 5.])
c = array([1, 1, 1, 0, 0])
x[2] = masked
z = where(c, x, -x)
assert_(eq(z, [1., 2., 0., -4., -5]))
c[0] = masked
z = where(c, x, -x)
assert_(eq(z, [1., 2., 0., -4., -5]))
assert_(z[0] is masked)
assert_(z[1] is not masked)
assert_(z[2] is masked)
assert_(eq(masked_where(greater(x, 2), x), masked_greater(x, 2)))
assert_(eq(masked_where(greater_equal(x, 2), x),
masked_greater_equal(x, 2)))
assert_(eq(masked_where(less(x, 2), x), masked_less(x, 2)))
assert_(eq(masked_where(less_equal(x, 2), x), masked_less_equal(x, 2)))
assert_(eq(masked_where(not_equal(x, 2), x), masked_not_equal(x, 2)))
assert_(eq(masked_where(equal(x, 2), x), masked_equal(x, 2)))
assert_(eq(masked_where(not_equal(x, 2), x), masked_not_equal(x, 2)))
assert_(eq(masked_inside(list(range(5)), 1, 3), [0, 199, 199, 199, 4]))
assert_(eq(masked_outside(list(range(5)), 1, 3), [199, 1, 2, 3, 199]))
assert_(eq(masked_inside(array(list(range(5)),
mask=[1, 0, 0, 0, 0]), 1, 3).mask,
[1, 1, 1, 1, 0]))
assert_(eq(masked_outside(array(list(range(5)),
mask=[0, 1, 0, 0, 0]), 1, 3).mask,
[1, 1, 0, 0, 1]))
assert_(eq(masked_equal(array(list(range(5)),
mask=[1, 0, 0, 0, 0]), 2).mask,
[1, 0, 1, 0, 0]))
assert_(eq(masked_not_equal(array([2, 2, 1, 2, 1],
mask=[1, 0, 0, 0, 0]), 2).mask,
[1, 0, 1, 0, 1]))
assert_(eq(masked_where([1, 1, 0, 0, 0], [1, 2, 3, 4, 5]),
[99, 99, 3, 4, 5]))
atest = ones((10, 10, 10), dtype=np.float32)
btest = zeros(atest.shape, MaskType)
ctest = masked_where(btest, atest)
assert_(eq(atest, ctest))
z = choose(c, (-x, x))
assert_(eq(z, [1., 2., 0., -4., -5]))
assert_(z[0] is masked)
assert_(z[1] is not masked)
assert_(z[2] is masked)
x = arange(6)
x[5] = masked
y = arange(6) * 10
y[2] = masked
c = array([1, 1, 1, 0, 0, 0], mask=[1, 0, 0, 0, 0, 0])
cm = c.filled(1)
z = where(c, x, y)
zm = where(cm, x, y)
assert_(eq(z, zm))
assert_(getmask(zm) is nomask)
assert_(eq(zm, [0, 1, 2, 30, 40, 50]))
z = where(c, masked, 1)
assert_(eq(z, [99, 99, 99, 1, 1, 1]))
z = where(c, 1, masked)
assert_(eq(z, [99, 1, 1, 99, 99, 99]))
def test_testMinMax2(self):
# Test of minimum, maximum.
assert_(eq(minimum([1, 2, 3], [4, 0, 9]), [1, 0, 3]))
assert_(eq(maximum([1, 2, 3], [4, 0, 9]), [4, 2, 9]))
x = arange(5)
y = arange(5) - 2
x[3] = masked
y[0] = masked
assert_(eq(minimum(x, y), where(less(x, y), x, y)))
assert_(eq(maximum(x, y), where(greater(x, y), x, y)))
assert_(minimum.reduce(x) == 0)
assert_(maximum.reduce(x) == 4)
def test_testTakeTransposeInnerOuter(self):
# Test of take, transpose, inner, outer products
x = arange(24)
y = np.arange(24)
x[5:6] = masked
x = x.reshape(2, 3, 4)
y = y.reshape(2, 3, 4)
assert_(eq(np.transpose(y, (2, 0, 1)), transpose(x, (2, 0, 1))))
assert_(eq(np.take(y, (2, 0, 1), 1), take(x, (2, 0, 1), 1)))
assert_(eq(np.inner(filled(x, 0), filled(y, 0)),
inner(x, y)))
assert_(eq(np.outer(filled(x, 0), filled(y, 0)),
outer(x, y)))
y = array(['abc', 1, 'def', 2, 3], object)
y[2] = masked
t = take(y, [0, 3, 4])
assert_(t[0] == 'abc')
assert_(t[1] == 2)
assert_(t[2] == 3)
def test_testInplace(self):
# Test of inplace operations and rich comparisons
y = arange(10)
x = arange(10)
xm = arange(10)
xm[2] = masked
x += 1
assert_(eq(x, y + 1))
xm += 1
assert_(eq(x, y + 1))
x = arange(10)
xm = arange(10)
xm[2] = masked
x -= 1
assert_(eq(x, y - 1))
xm -= 1
assert_(eq(xm, y - 1))
x = arange(10) * 1.0
xm = arange(10) * 1.0
xm[2] = masked
x *= 2.0
assert_(eq(x, y * 2))
xm *= 2.0
assert_(eq(xm, y * 2))
x = arange(10) * 2
xm = arange(10)
xm[2] = masked
x //= 2
assert_(eq(x, y))
xm //= 2
assert_(eq(x, y))
x = arange(10) * 1.0
xm = arange(10) * 1.0
xm[2] = masked
x /= 2.0
assert_(eq(x, y / 2.0))
xm /= arange(10)
assert_(eq(xm, ones((10,))))
x = arange(10).astype(np.float32)
xm = arange(10)
xm[2] = masked
x += 1.
assert_(eq(x, y + 1.))
def test_testPickle(self):
# Test of pickling
x = arange(12)
x[4:10:2] = masked
x = x.reshape(4, 3)
for proto in range(2, pickle.HIGHEST_PROTOCOL + 1):
s = pickle.dumps(x, protocol=proto)
y = pickle.loads(s)
assert_(eq(x, y))
def test_testMasked(self):
# Test of masked element
xx = arange(6)
xx[1] = masked
assert_(str(masked) == '--')
assert_(xx[1] is masked)
assert_equal(filled(xx[1], 0), 0)
def test_testAverage1(self):
# Test of average.
ott = array([0., 1., 2., 3.], mask=[1, 0, 0, 0])
assert_(eq(2.0, average(ott, axis=0)))
assert_(eq(2.0, average(ott, weights=[1., 1., 2., 1.])))
result, wts = average(ott, weights=[1., 1., 2., 1.], returned=True)
assert_(eq(2.0, result))
assert_(wts == 4.0)
ott[:] = masked
assert_(average(ott, axis=0) is masked)
ott = array([0., 1., 2., 3.], mask=[1, 0, 0, 0])
ott = ott.reshape(2, 2)
ott[:, 1] = masked
assert_(eq(average(ott, axis=0), [2.0, 0.0]))
assert_(average(ott, axis=1)[0] is masked)
assert_(eq([2., 0.], average(ott, axis=0)))
result, wts = average(ott, axis=0, returned=True)
assert_(eq(wts, [1., 0.]))
def test_testAverage2(self):
# More tests of average.
w1 = [0, 1, 1, 1, 1, 0]
w2 = [[0, 1, 1, 1, 1, 0], [1, 0, 0, 0, 0, 1]]
x = arange(6)
assert_(allclose(average(x, axis=0), 2.5))
assert_(allclose(average(x, axis=0, weights=w1), 2.5))
y = array([arange(6), 2.0 * arange(6)])
assert_(allclose(average(y, None),
np.add.reduce(np.arange(6)) * 3. / 12.))
assert_(allclose(average(y, axis=0), np.arange(6) * 3. / 2.))
assert_(allclose(average(y, axis=1),
[average(x, axis=0), average(x, axis=0)*2.0]))
assert_(allclose(average(y, None, weights=w2), 20. / 6.))
assert_(allclose(average(y, axis=0, weights=w2),
[0., 1., 2., 3., 4., 10.]))
assert_(allclose(average(y, axis=1),
[average(x, axis=0), average(x, axis=0)*2.0]))
m1 = zeros(6)
m2 = [0, 0, 1, 1, 0, 0]
m3 = [[0, 0, 1, 1, 0, 0], [0, 1, 1, 1, 1, 0]]
m4 = ones(6)
m5 = [0, 1, 1, 1, 1, 1]
assert_(allclose(average(masked_array(x, m1), axis=0), 2.5))
assert_(allclose(average(masked_array(x, m2), axis=0), 2.5))
assert_(average(masked_array(x, m4), axis=0) is masked)
assert_equal(average(masked_array(x, m5), axis=0), 0.0)
assert_equal(count(average(masked_array(x, m4), axis=0)), 0)
z = masked_array(y, m3)
assert_(allclose(average(z, None), 20. / 6.))
assert_(allclose(average(z, axis=0),
[0., 1., 99., 99., 4.0, 7.5]))
assert_(allclose(average(z, axis=1), [2.5, 5.0]))
assert_(allclose(average(z, axis=0, weights=w2),
[0., 1., 99., 99., 4.0, 10.0]))
a = arange(6)
b = arange(6) * 3
r1, w1 = average([[a, b], [b, a]], axis=1, returned=True)
assert_equal(shape(r1), shape(w1))
assert_equal(r1.shape, w1.shape)
r2, w2 = average(ones((2, 2, 3)), axis=0, weights=[3, 1], returned=True)
assert_equal(shape(w2), shape(r2))
r2, w2 = average(ones((2, 2, 3)), returned=True)
assert_equal(shape(w2), shape(r2))
r2, w2 = average(ones((2, 2, 3)), weights=ones((2, 2, 3)), returned=True)
assert_(shape(w2) == shape(r2))
a2d = array([[1, 2], [0, 4]], float)
a2dm = masked_array(a2d, [[0, 0], [1, 0]])
a2da = average(a2d, axis=0)
assert_(eq(a2da, [0.5, 3.0]))
a2dma = average(a2dm, axis=0)
assert_(eq(a2dma, [1.0, 3.0]))
a2dma = average(a2dm, axis=None)
assert_(eq(a2dma, 7. / 3.))
a2dma = average(a2dm, axis=1)
assert_(eq(a2dma, [1.5, 4.0]))
def test_testToPython(self):
assert_equal(1, int(array(1)))
assert_equal(1.0, float(array(1)))
assert_equal(1, int(array([[[1]]])))
assert_equal(1.0, float(array([[1]])))
assert_raises(TypeError, float, array([1, 1]))
assert_raises(ValueError, bool, array([0, 1]))
assert_raises(ValueError, bool, array([0, 0], mask=[0, 1]))
def test_testScalarArithmetic(self):
xm = array(0, mask=1)
#TODO FIXME: Find out what the following raises a warning in r8247
with np.errstate(divide='ignore'):
assert_((1 / array(0)).mask)
assert_((1 + xm).mask)
assert_((-xm).mask)
assert_((-xm).mask)
assert_(maximum(xm, xm).mask)
assert_(minimum(xm, xm).mask)
assert_(xm.filled().dtype is xm._data.dtype)
x = array(0, mask=0)
assert_(x.filled() == x._data)
assert_equal(str(xm), str(masked_print_option))
def test_testArrayMethods(self):
a = array([1, 3, 2])
assert_(eq(a.any(), a._data.any()))
assert_(eq(a.all(), a._data.all()))
assert_(eq(a.argmax(), a._data.argmax()))
assert_(eq(a.argmin(), a._data.argmin()))
assert_(eq(a.choose(0, 1, 2, 3, 4),
a._data.choose(0, 1, 2, 3, 4)))
assert_(eq(a.compress([1, 0, 1]), a._data.compress([1, 0, 1])))
assert_(eq(a.conj(), a._data.conj()))
assert_(eq(a.conjugate(), a._data.conjugate()))
m = array([[1, 2], [3, 4]])
assert_(eq(m.diagonal(), m._data.diagonal()))
assert_(eq(a.sum(), a._data.sum()))
assert_(eq(a.take([1, 2]), a._data.take([1, 2])))
assert_(eq(m.transpose(), m._data.transpose()))
def test_testArrayAttributes(self):
a = array([1, 3, 2])
assert_equal(a.ndim, 1)
def test_testAPI(self):
assert_(not [m for m in dir(np.ndarray)
if m not in dir(MaskedArray) and
not m.startswith('_')])
def test_testSingleElementSubscript(self):
a = array([1, 3, 2])
b = array([1, 3, 2], mask=[1, 0, 1])
assert_equal(a[0].shape, ())
assert_equal(b[0].shape, ())
assert_equal(b[1].shape, ())
def test_assignment_by_condition(self):
# Test for gh-18951
a = array([1, 2, 3, 4], mask=[1, 0, 1, 0])
c = a >= 3
a[c] = 5
assert_(a[2] is masked)
def test_assignment_by_condition_2(self):
# gh-19721
a = masked_array([0, 1], mask=[False, False])
b = masked_array([0, 1], mask=[True, True])
mask = a < 1
b[mask] = a[mask]
expected_mask = [False, True]
assert_equal(b.mask, expected_mask)
class TestUfuncs:
def setup_method(self):
self.d = (array([1.0, 0, -1, pi / 2] * 2, mask=[0, 1] + [0] * 6),
array([1.0, 0, -1, pi / 2] * 2, mask=[1, 0] + [0] * 6),)
def test_testUfuncRegression(self):
f_invalid_ignore = [
'sqrt', 'arctanh', 'arcsin', 'arccos',
'arccosh', 'arctanh', 'log', 'log10', 'divide',
'true_divide', 'floor_divide', 'remainder', 'fmod']
for f in ['sqrt', 'log', 'log10', 'exp', 'conjugate',
'sin', 'cos', 'tan',
'arcsin', 'arccos', 'arctan',
'sinh', 'cosh', 'tanh',
'arcsinh',
'arccosh',
'arctanh',
'absolute', 'fabs', 'negative',
'floor', 'ceil',
'logical_not',
'add', 'subtract', 'multiply',
'divide', 'true_divide', 'floor_divide',
'remainder', 'fmod', 'hypot', 'arctan2',
'equal', 'not_equal', 'less_equal', 'greater_equal',
'less', 'greater',
'logical_and', 'logical_or', 'logical_xor']:
try:
uf = getattr(umath, f)
except AttributeError:
uf = getattr(fromnumeric, f)
mf = getattr(np.ma, f)
args = self.d[:uf.nin]
with np.errstate():
if f in f_invalid_ignore:
np.seterr(invalid='ignore')
if f in ['arctanh', 'log', 'log10']:
np.seterr(divide='ignore')
ur = uf(*args)
mr = mf(*args)
assert_(eq(ur.filled(0), mr.filled(0), f))
assert_(eqmask(ur.mask, mr.mask))
def test_reduce(self):
a = self.d[0]
assert_(not alltrue(a, axis=0))
assert_(sometrue(a, axis=0))
assert_equal(sum(a[:3], axis=0), 0)
assert_equal(product(a, axis=0), 0)
def test_minmax(self):
a = arange(1, 13).reshape(3, 4)
amask = masked_where(a < 5, a)
assert_equal(amask.max(), a.max())
assert_equal(amask.min(), 5)
assert_((amask.max(0) == a.max(0)).all())
assert_((amask.min(0) == [5, 6, 7, 8]).all())
assert_(amask.max(1)[0].mask)
assert_(amask.min(1)[0].mask)
def test_nonzero(self):
for t in "?bhilqpBHILQPfdgFDGO":
x = array([1, 0, 2, 0], mask=[0, 0, 1, 1])
assert_(eq(nonzero(x), [0]))
class TestArrayMethods:
def setup_method(self):
x = np.array([8.375, 7.545, 8.828, 8.5, 1.757, 5.928,
8.43, 7.78, 9.865, 5.878, 8.979, 4.732,
3.012, 6.022, 5.095, 3.116, 5.238, 3.957,
6.04, 9.63, 7.712, 3.382, 4.489, 6.479,
7.189, 9.645, 5.395, 4.961, 9.894, 2.893,
7.357, 9.828, 6.272, 3.758, 6.693, 0.993])
X = x.reshape(6, 6)
XX = x.reshape(3, 2, 2, 3)
m = np.array([0, 1, 0, 1, 0, 0,
1, 0, 1, 1, 0, 1,
0, 0, 0, 1, 0, 1,
0, 0, 0, 1, 1, 1,
1, 0, 0, 1, 0, 0,
0, 0, 1, 0, 1, 0])
mx = array(data=x, mask=m)
mX = array(data=X, mask=m.reshape(X.shape))
mXX = array(data=XX, mask=m.reshape(XX.shape))
self.d = (x, X, XX, m, mx, mX, mXX)
def test_trace(self):
(x, X, XX, m, mx, mX, mXX,) = self.d
mXdiag = mX.diagonal()
assert_equal(mX.trace(), mX.diagonal().compressed().sum())
assert_(eq(mX.trace(),
X.trace() - sum(mXdiag.mask * X.diagonal(),
axis=0)))
def test_clip(self):
(x, X, XX, m, mx, mX, mXX,) = self.d
clipped = mx.clip(2, 8)
assert_(eq(clipped.mask, mx.mask))
assert_(eq(clipped._data, x.clip(2, 8)))
assert_(eq(clipped._data, mx._data.clip(2, 8)))
def test_ptp(self):
(x, X, XX, m, mx, mX, mXX,) = self.d
(n, m) = X.shape
# print(type(mx), mx.compressed())
# raise Exception()
assert_equal(mx.ptp(), np.ptp(mx.compressed()))
rows = np.zeros(n, np.float64)
cols = np.zeros(m, np.float64)
for k in range(m):
cols[k] = np.ptp(mX[:, k].compressed())
for k in range(n):
rows[k] = np.ptp(mX[k].compressed())
assert_(eq(mX.ptp(0), cols))
assert_(eq(mX.ptp(1), rows))
def test_swapaxes(self):
(x, X, XX, m, mx, mX, mXX,) = self.d
mXswapped = mX.swapaxes(0, 1)
assert_(eq(mXswapped[-1], mX[:, -1]))
mXXswapped = mXX.swapaxes(0, 2)
assert_equal(mXXswapped.shape, (2, 2, 3, 3))
def test_cumprod(self):
(x, X, XX, m, mx, mX, mXX,) = self.d
mXcp = mX.cumprod(0)
assert_(eq(mXcp._data, mX.filled(1).cumprod(0)))
mXcp = mX.cumprod(1)
assert_(eq(mXcp._data, mX.filled(1).cumprod(1)))
def test_cumsum(self):
(x, X, XX, m, mx, mX, mXX,) = self.d
mXcp = mX.cumsum(0)
assert_(eq(mXcp._data, mX.filled(0).cumsum(0)))
mXcp = mX.cumsum(1)
assert_(eq(mXcp._data, mX.filled(0).cumsum(1)))
def test_varstd(self):
(x, X, XX, m, mx, mX, mXX,) = self.d
assert_(eq(mX.var(axis=None), mX.compressed().var()))
assert_(eq(mX.std(axis=None), mX.compressed().std()))
assert_(eq(mXX.var(axis=3).shape, XX.var(axis=3).shape))
assert_(eq(mX.var().shape, X.var().shape))
(mXvar0, mXvar1) = (mX.var(axis=0), mX.var(axis=1))
for k in range(6):
assert_(eq(mXvar1[k], mX[k].compressed().var()))
assert_(eq(mXvar0[k], mX[:, k].compressed().var()))
assert_(eq(np.sqrt(mXvar0[k]),
mX[:, k].compressed().std()))
def eqmask(m1, m2):
if m1 is nomask:
return m2 is nomask
if m2 is nomask:
return m1 is nomask
return (m1 == m2).all()

View File

@ -0,0 +1,97 @@
import numpy as np
from numpy.testing import (
assert_, assert_array_equal, assert_allclose, suppress_warnings
)
class TestRegression:
def test_masked_array_create(self):
# Ticket #17
x = np.ma.masked_array([0, 1, 2, 3, 0, 4, 5, 6],
mask=[0, 0, 0, 1, 1, 1, 0, 0])
assert_array_equal(np.ma.nonzero(x), [[1, 2, 6, 7]])
def test_masked_array(self):
# Ticket #61
np.ma.array(1, mask=[1])
def test_mem_masked_where(self):
# Ticket #62
from numpy.ma import masked_where, MaskType
a = np.zeros((1, 1))
b = np.zeros(a.shape, MaskType)
c = masked_where(b, a)
a-c
def test_masked_array_multiply(self):
# Ticket #254
a = np.ma.zeros((4, 1))
a[2, 0] = np.ma.masked
b = np.zeros((4, 2))
a*b
b*a
def test_masked_array_repeat(self):
# Ticket #271
np.ma.array([1], mask=False).repeat(10)
def test_masked_array_repr_unicode(self):
# Ticket #1256
repr(np.ma.array("Unicode"))
def test_atleast_2d(self):
# Ticket #1559
a = np.ma.masked_array([0.0, 1.2, 3.5], mask=[False, True, False])
b = np.atleast_2d(a)
assert_(a.mask.ndim == 1)
assert_(b.mask.ndim == 2)
def test_set_fill_value_unicode_py3(self):
# Ticket #2733
a = np.ma.masked_array(['a', 'b', 'c'], mask=[1, 0, 0])
a.fill_value = 'X'
assert_(a.fill_value == 'X')
def test_var_sets_maskedarray_scalar(self):
# Issue gh-2757
a = np.ma.array(np.arange(5), mask=True)
mout = np.ma.array(-1, dtype=float)
a.var(out=mout)
assert_(mout._data == 0)
def test_ddof_corrcoef(self):
# See gh-3336
x = np.ma.masked_equal([1, 2, 3, 4, 5], 4)
y = np.array([2, 2.5, 3.1, 3, 5])
# this test can be removed after deprecation.
with suppress_warnings() as sup:
sup.filter(DeprecationWarning, "bias and ddof have no effect")
r0 = np.ma.corrcoef(x, y, ddof=0)
r1 = np.ma.corrcoef(x, y, ddof=1)
# ddof should not have an effect (it gets cancelled out)
assert_allclose(r0.data, r1.data)
def test_mask_not_backmangled(self):
# See gh-10314. Test case taken from gh-3140.
a = np.ma.MaskedArray([1., 2.], mask=[False, False])
assert_(a.mask.shape == (2,))
b = np.tile(a, (2, 1))
# Check that the above no longer changes a.shape to (1, 2)
assert_(a.mask.shape == (2,))
assert_(b.shape == (2, 2))
assert_(b.mask.shape == (2, 2))
def test_empty_list_on_structured(self):
# See gh-12464. Indexing with empty list should give empty result.
ma = np.ma.MaskedArray([(1, 1.), (2, 2.), (3, 3.)], dtype='i4,f4')
assert_array_equal(ma[[]], ma[:0])
def test_masked_array_tobytes_fortran(self):
ma = np.ma.arange(4).reshape((2,2))
assert_array_equal(ma.tobytes(order='F'), ma.T.tobytes())
def test_structured_array(self):
# see gh-22041
np.ma.array((1, (b"", b"")),
dtype=[("x", np.int_),
("y", [("i", np.void), ("j", np.void)])])

View File

@ -0,0 +1,460 @@
# pylint: disable-msg=W0611, W0612, W0511,R0201
"""Tests suite for MaskedArray & subclassing.
:author: Pierre Gerard-Marchant
:contact: pierregm_at_uga_dot_edu
:version: $Id: test_subclassing.py 3473 2007-10-29 15:18:13Z jarrod.millman $
"""
import numpy as np
from numpy.lib.mixins import NDArrayOperatorsMixin
from numpy.testing import assert_, assert_raises
from numpy.ma.testutils import assert_equal
from numpy.ma.core import (
array, arange, masked, MaskedArray, masked_array, log, add, hypot,
divide, asarray, asanyarray, nomask
)
# from numpy.ma.core import (
def assert_startswith(a, b):
# produces a better error message than assert_(a.startswith(b))
assert_equal(a[:len(b)], b)
class SubArray(np.ndarray):
# Defines a generic np.ndarray subclass, that stores some metadata
# in the dictionary `info`.
def __new__(cls,arr,info={}):
x = np.asanyarray(arr).view(cls)
x.info = info.copy()
return x
def __array_finalize__(self, obj):
super().__array_finalize__(obj)
self.info = getattr(obj, 'info', {}).copy()
return
def __add__(self, other):
result = super().__add__(other)
result.info['added'] = result.info.get('added', 0) + 1
return result
def __iadd__(self, other):
result = super().__iadd__(other)
result.info['iadded'] = result.info.get('iadded', 0) + 1
return result
subarray = SubArray
class SubMaskedArray(MaskedArray):
"""Pure subclass of MaskedArray, keeping some info on subclass."""
def __new__(cls, info=None, **kwargs):
obj = super().__new__(cls, **kwargs)
obj._optinfo['info'] = info
return obj
class MSubArray(SubArray, MaskedArray):
def __new__(cls, data, info={}, mask=nomask):
subarr = SubArray(data, info)
_data = MaskedArray.__new__(cls, data=subarr, mask=mask)
_data.info = subarr.info
return _data
@property
def _series(self):
_view = self.view(MaskedArray)
_view._sharedmask = False
return _view
msubarray = MSubArray
# Also a subclass that overrides __str__, __repr__ and __setitem__, disallowing
# setting to non-class values (and thus np.ma.core.masked_print_option)
# and overrides __array_wrap__, updating the info dict, to check that this
# doesn't get destroyed by MaskedArray._update_from. But this one also needs
# its own iterator...
class CSAIterator:
"""
Flat iterator object that uses its own setter/getter
(works around ndarray.flat not propagating subclass setters/getters
see https://github.com/numpy/numpy/issues/4564)
roughly following MaskedIterator
"""
def __init__(self, a):
self._original = a
self._dataiter = a.view(np.ndarray).flat
def __iter__(self):
return self
def __getitem__(self, indx):
out = self._dataiter.__getitem__(indx)
if not isinstance(out, np.ndarray):
out = out.__array__()
out = out.view(type(self._original))
return out
def __setitem__(self, index, value):
self._dataiter[index] = self._original._validate_input(value)
def __next__(self):
return next(self._dataiter).__array__().view(type(self._original))
class ComplicatedSubArray(SubArray):
def __str__(self):
return f'myprefix {self.view(SubArray)} mypostfix'
def __repr__(self):
# Return a repr that does not start with 'name('
return f'<{self.__class__.__name__} {self}>'
def _validate_input(self, value):
if not isinstance(value, ComplicatedSubArray):
raise ValueError("Can only set to MySubArray values")
return value
def __setitem__(self, item, value):
# validation ensures direct assignment with ndarray or
# masked_print_option will fail
super().__setitem__(item, self._validate_input(value))
def __getitem__(self, item):
# ensure getter returns our own class also for scalars
value = super().__getitem__(item)
if not isinstance(value, np.ndarray): # scalar
value = value.__array__().view(ComplicatedSubArray)
return value
@property
def flat(self):
return CSAIterator(self)
@flat.setter
def flat(self, value):
y = self.ravel()
y[:] = value
def __array_wrap__(self, obj, context=None, return_scalar=False):
obj = super().__array_wrap__(obj, context, return_scalar)
if context is not None and context[0] is np.multiply:
obj.info['multiplied'] = obj.info.get('multiplied', 0) + 1
return obj
class WrappedArray(NDArrayOperatorsMixin):
"""
Wrapping a MaskedArray rather than subclassing to test that
ufunc deferrals are commutative.
See: https://github.com/numpy/numpy/issues/15200)
"""
__slots__ = ('_array', 'attrs')
__array_priority__ = 20
def __init__(self, array, **attrs):
self._array = array
self.attrs = attrs
def __repr__(self):
return f"{self.__class__.__name__}(\n{self._array}\n{self.attrs}\n)"
def __array__(self, dtype=None, copy=None):
return np.asarray(self._array)
def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
if method == '__call__':
inputs = [arg._array if isinstance(arg, self.__class__) else arg
for arg in inputs]
return self.__class__(ufunc(*inputs, **kwargs), **self.attrs)
else:
return NotImplemented
class TestSubclassing:
# Test suite for masked subclasses of ndarray.
def setup_method(self):
x = np.arange(5, dtype='float')
mx = msubarray(x, mask=[0, 1, 0, 0, 0])
self.data = (x, mx)
def test_data_subclassing(self):
# Tests whether the subclass is kept.
x = np.arange(5)
m = [0, 0, 1, 0, 0]
xsub = SubArray(x)
xmsub = masked_array(xsub, mask=m)
assert_(isinstance(xmsub, MaskedArray))
assert_equal(xmsub._data, xsub)
assert_(isinstance(xmsub._data, SubArray))
def test_maskedarray_subclassing(self):
# Tests subclassing MaskedArray
(x, mx) = self.data
assert_(isinstance(mx._data, subarray))
def test_masked_unary_operations(self):
# Tests masked_unary_operation
(x, mx) = self.data
with np.errstate(divide='ignore'):
assert_(isinstance(log(mx), msubarray))
assert_equal(log(x), np.log(x))
def test_masked_binary_operations(self):
# Tests masked_binary_operation
(x, mx) = self.data
# Result should be a msubarray
assert_(isinstance(add(mx, mx), msubarray))
assert_(isinstance(add(mx, x), msubarray))
# Result should work
assert_equal(add(mx, x), mx+x)
assert_(isinstance(add(mx, mx)._data, subarray))
assert_(isinstance(add.outer(mx, mx), msubarray))
assert_(isinstance(hypot(mx, mx), msubarray))
assert_(isinstance(hypot(mx, x), msubarray))
def test_masked_binary_operations2(self):
# Tests domained_masked_binary_operation
(x, mx) = self.data
xmx = masked_array(mx.data.__array__(), mask=mx.mask)
assert_(isinstance(divide(mx, mx), msubarray))
assert_(isinstance(divide(mx, x), msubarray))
assert_equal(divide(mx, mx), divide(xmx, xmx))
def test_attributepropagation(self):
x = array(arange(5), mask=[0]+[1]*4)
my = masked_array(subarray(x))
ym = msubarray(x)
#
z = (my+1)
assert_(isinstance(z, MaskedArray))
assert_(not isinstance(z, MSubArray))
assert_(isinstance(z._data, SubArray))
assert_equal(z._data.info, {})
#
z = (ym+1)
assert_(isinstance(z, MaskedArray))
assert_(isinstance(z, MSubArray))
assert_(isinstance(z._data, SubArray))
assert_(z._data.info['added'] > 0)
# Test that inplace methods from data get used (gh-4617)
ym += 1
assert_(isinstance(ym, MaskedArray))
assert_(isinstance(ym, MSubArray))
assert_(isinstance(ym._data, SubArray))
assert_(ym._data.info['iadded'] > 0)
#
ym._set_mask([1, 0, 0, 0, 1])
assert_equal(ym._mask, [1, 0, 0, 0, 1])
ym._series._set_mask([0, 0, 0, 0, 1])
assert_equal(ym._mask, [0, 0, 0, 0, 1])
#
xsub = subarray(x, info={'name':'x'})
mxsub = masked_array(xsub)
assert_(hasattr(mxsub, 'info'))
assert_equal(mxsub.info, xsub.info)
def test_subclasspreservation(self):
# Checks that masked_array(...,subok=True) preserves the class.
x = np.arange(5)
m = [0, 0, 1, 0, 0]
xinfo = [(i, j) for (i, j) in zip(x, m)]
xsub = MSubArray(x, mask=m, info={'xsub':xinfo})
#
mxsub = masked_array(xsub, subok=False)
assert_(not isinstance(mxsub, MSubArray))
assert_(isinstance(mxsub, MaskedArray))
assert_equal(mxsub._mask, m)
#
mxsub = asarray(xsub)
assert_(not isinstance(mxsub, MSubArray))
assert_(isinstance(mxsub, MaskedArray))
assert_equal(mxsub._mask, m)
#
mxsub = masked_array(xsub, subok=True)
assert_(isinstance(mxsub, MSubArray))
assert_equal(mxsub.info, xsub.info)
assert_equal(mxsub._mask, xsub._mask)
#
mxsub = asanyarray(xsub)
assert_(isinstance(mxsub, MSubArray))
assert_equal(mxsub.info, xsub.info)
assert_equal(mxsub._mask, m)
def test_subclass_items(self):
"""test that getter and setter go via baseclass"""
x = np.arange(5)
xcsub = ComplicatedSubArray(x)
mxcsub = masked_array(xcsub, mask=[True, False, True, False, False])
# getter should return a ComplicatedSubArray, even for single item
# first check we wrote ComplicatedSubArray correctly
assert_(isinstance(xcsub[1], ComplicatedSubArray))
assert_(isinstance(xcsub[1,...], ComplicatedSubArray))
assert_(isinstance(xcsub[1:4], ComplicatedSubArray))
# now that it propagates inside the MaskedArray
assert_(isinstance(mxcsub[1], ComplicatedSubArray))
assert_(isinstance(mxcsub[1,...].data, ComplicatedSubArray))
assert_(mxcsub[0] is masked)
assert_(isinstance(mxcsub[0,...].data, ComplicatedSubArray))
assert_(isinstance(mxcsub[1:4].data, ComplicatedSubArray))
# also for flattened version (which goes via MaskedIterator)
assert_(isinstance(mxcsub.flat[1].data, ComplicatedSubArray))
assert_(mxcsub.flat[0] is masked)
assert_(isinstance(mxcsub.flat[1:4].base, ComplicatedSubArray))
# setter should only work with ComplicatedSubArray input
# first check we wrote ComplicatedSubArray correctly
assert_raises(ValueError, xcsub.__setitem__, 1, x[4])
# now that it propagates inside the MaskedArray
assert_raises(ValueError, mxcsub.__setitem__, 1, x[4])
assert_raises(ValueError, mxcsub.__setitem__, slice(1, 4), x[1:4])
mxcsub[1] = xcsub[4]
mxcsub[1:4] = xcsub[1:4]
# also for flattened version (which goes via MaskedIterator)
assert_raises(ValueError, mxcsub.flat.__setitem__, 1, x[4])
assert_raises(ValueError, mxcsub.flat.__setitem__, slice(1, 4), x[1:4])
mxcsub.flat[1] = xcsub[4]
mxcsub.flat[1:4] = xcsub[1:4]
def test_subclass_nomask_items(self):
x = np.arange(5)
xcsub = ComplicatedSubArray(x)
mxcsub_nomask = masked_array(xcsub)
assert_(isinstance(mxcsub_nomask[1,...].data, ComplicatedSubArray))
assert_(isinstance(mxcsub_nomask[0,...].data, ComplicatedSubArray))
assert_(isinstance(mxcsub_nomask[1], ComplicatedSubArray))
assert_(isinstance(mxcsub_nomask[0], ComplicatedSubArray))
def test_subclass_repr(self):
"""test that repr uses the name of the subclass
and 'array' for np.ndarray"""
x = np.arange(5)
mx = masked_array(x, mask=[True, False, True, False, False])
assert_startswith(repr(mx), 'masked_array')
xsub = SubArray(x)
mxsub = masked_array(xsub, mask=[True, False, True, False, False])
assert_startswith(repr(mxsub),
f'masked_{SubArray.__name__}(data=[--, 1, --, 3, 4]')
def test_subclass_str(self):
"""test str with subclass that has overridden str, setitem"""
# first without override
x = np.arange(5)
xsub = SubArray(x)
mxsub = masked_array(xsub, mask=[True, False, True, False, False])
assert_equal(str(mxsub), '[-- 1 -- 3 4]')
xcsub = ComplicatedSubArray(x)
assert_raises(ValueError, xcsub.__setitem__, 0,
np.ma.core.masked_print_option)
mxcsub = masked_array(xcsub, mask=[True, False, True, False, False])
assert_equal(str(mxcsub), 'myprefix [-- 1 -- 3 4] mypostfix')
def test_pure_subclass_info_preservation(self):
# Test that ufuncs and methods conserve extra information consistently;
# see gh-7122.
arr1 = SubMaskedArray('test', data=[1,2,3,4,5,6])
arr2 = SubMaskedArray(data=[0,1,2,3,4,5])
diff1 = np.subtract(arr1, arr2)
assert_('info' in diff1._optinfo)
assert_(diff1._optinfo['info'] == 'test')
diff2 = arr1 - arr2
assert_('info' in diff2._optinfo)
assert_(diff2._optinfo['info'] == 'test')
class ArrayNoInheritance:
"""Quantity-like class that does not inherit from ndarray"""
def __init__(self, data, units):
self.magnitude = data
self.units = units
def __getattr__(self, attr):
return getattr(self.magnitude, attr)
def test_array_no_inheritance():
data_masked = np.ma.array([1, 2, 3], mask=[True, False, True])
data_masked_units = ArrayNoInheritance(data_masked, 'meters')
# Get the masked representation of the Quantity-like class
new_array = np.ma.array(data_masked_units)
assert_equal(data_masked.data, new_array.data)
assert_equal(data_masked.mask, new_array.mask)
# Test sharing the mask
data_masked.mask = [True, False, False]
assert_equal(data_masked.mask, new_array.mask)
assert_(new_array.sharedmask)
# Get the masked representation of the Quantity-like class
new_array = np.ma.array(data_masked_units, copy=True)
assert_equal(data_masked.data, new_array.data)
assert_equal(data_masked.mask, new_array.mask)
# Test that the mask is not shared when copy=True
data_masked.mask = [True, False, True]
assert_equal([True, False, False], new_array.mask)
assert_(not new_array.sharedmask)
# Get the masked representation of the Quantity-like class
new_array = np.ma.array(data_masked_units, keep_mask=False)
assert_equal(data_masked.data, new_array.data)
# The change did not affect the original mask
assert_equal(data_masked.mask, [True, False, True])
# Test that the mask is False and not shared when keep_mask=False
assert_(not new_array.mask)
assert_(not new_array.sharedmask)
class TestClassWrapping:
# Test suite for classes that wrap MaskedArrays
def setup_method(self):
m = np.ma.masked_array([1, 3, 5], mask=[False, True, False])
wm = WrappedArray(m)
self.data = (m, wm)
def test_masked_unary_operations(self):
# Tests masked_unary_operation
(m, wm) = self.data
with np.errstate(divide='ignore'):
assert_(isinstance(np.log(wm), WrappedArray))
def test_masked_binary_operations(self):
# Tests masked_binary_operation
(m, wm) = self.data
# Result should be a WrappedArray
assert_(isinstance(np.add(wm, wm), WrappedArray))
assert_(isinstance(np.add(m, wm), WrappedArray))
assert_(isinstance(np.add(wm, m), WrappedArray))
# add and '+' should call the same ufunc
assert_equal(np.add(m, wm), m + wm)
assert_(isinstance(np.hypot(m, wm), WrappedArray))
assert_(isinstance(np.hypot(wm, m), WrappedArray))
# Test domained binary operations
assert_(isinstance(np.divide(wm, m), WrappedArray))
assert_(isinstance(np.divide(m, wm), WrappedArray))
assert_equal(np.divide(wm, m) * m, np.divide(m, m) * wm)
# Test broadcasting
m2 = np.stack([m, m])
assert_(isinstance(np.divide(wm, m2), WrappedArray))
assert_(isinstance(np.divide(m2, wm), WrappedArray))
assert_equal(np.divide(m2, wm), np.divide(wm, m2))
def test_mixins_have_slots(self):
mixin = NDArrayOperatorsMixin()
# Should raise an error
assert_raises(AttributeError, mixin.__setattr__, "not_a_real_attr", 1)
m = np.ma.masked_array([1, 3, 5], mask=[False, True, False])
wm = WrappedArray(m)
assert_raises(AttributeError, wm.__setattr__, "not_an_attr", 2)

View File

@ -0,0 +1,292 @@
"""Miscellaneous functions for testing masked arrays and subclasses
:author: Pierre Gerard-Marchant
:contact: pierregm_at_uga_dot_edu
:version: $Id: testutils.py 3529 2007-11-13 08:01:14Z jarrod.millman $
"""
import operator
import numpy as np
from numpy import ndarray
import numpy._core.umath as umath
import numpy.testing
from numpy.testing import (
assert_, assert_allclose, assert_array_almost_equal_nulp,
assert_raises, build_err_msg
)
from .core import mask_or, getmask, masked_array, nomask, masked, filled
__all__masked = [
'almost', 'approx', 'assert_almost_equal', 'assert_array_almost_equal',
'assert_array_approx_equal', 'assert_array_compare',
'assert_array_equal', 'assert_array_less', 'assert_close',
'assert_equal', 'assert_equal_records', 'assert_mask_equal',
'assert_not_equal', 'fail_if_array_equal',
]
# Include some normal test functions to avoid breaking other projects who
# have mistakenly included them from this file. SciPy is one. That is
# unfortunate, as some of these functions are not intended to work with
# masked arrays. But there was no way to tell before.
from unittest import TestCase
__some__from_testing = [
'TestCase', 'assert_', 'assert_allclose', 'assert_array_almost_equal_nulp',
'assert_raises'
]
__all__ = __all__masked + __some__from_testing
def approx(a, b, fill_value=True, rtol=1e-5, atol=1e-8):
"""
Returns true if all components of a and b are equal to given tolerances.
If fill_value is True, masked values considered equal. Otherwise,
masked values are considered unequal. The relative error rtol should
be positive and << 1.0 The absolute error atol comes into play for
those elements of b that are very small or zero; it says how small a
must be also.
"""
m = mask_or(getmask(a), getmask(b))
d1 = filled(a)
d2 = filled(b)
if d1.dtype.char == "O" or d2.dtype.char == "O":
return np.equal(d1, d2).ravel()
x = filled(
masked_array(d1, copy=False, mask=m), fill_value
).astype(np.float64)
y = filled(masked_array(d2, copy=False, mask=m), 1).astype(np.float64)
d = np.less_equal(umath.absolute(x - y), atol + rtol * umath.absolute(y))
return d.ravel()
def almost(a, b, decimal=6, fill_value=True):
"""
Returns True if a and b are equal up to decimal places.
If fill_value is True, masked values considered equal. Otherwise,
masked values are considered unequal.
"""
m = mask_or(getmask(a), getmask(b))
d1 = filled(a)
d2 = filled(b)
if d1.dtype.char == "O" or d2.dtype.char == "O":
return np.equal(d1, d2).ravel()
x = filled(
masked_array(d1, copy=False, mask=m), fill_value
).astype(np.float64)
y = filled(masked_array(d2, copy=False, mask=m), 1).astype(np.float64)
d = np.around(np.abs(x - y), decimal) <= 10.0 ** (-decimal)
return d.ravel()
def _assert_equal_on_sequences(actual, desired, err_msg=''):
"""
Asserts the equality of two non-array sequences.
"""
assert_equal(len(actual), len(desired), err_msg)
for k in range(len(desired)):
assert_equal(actual[k], desired[k], f'item={k!r}\n{err_msg}')
return
def assert_equal_records(a, b):
"""
Asserts that two records are equal.
Pretty crude for now.
"""
assert_equal(a.dtype, b.dtype)
for f in a.dtype.names:
(af, bf) = (operator.getitem(a, f), operator.getitem(b, f))
if not (af is masked) and not (bf is masked):
assert_equal(operator.getitem(a, f), operator.getitem(b, f))
return
def assert_equal(actual, desired, err_msg=''):
"""
Asserts that two items are equal.
"""
# Case #1: dictionary .....
if isinstance(desired, dict):
if not isinstance(actual, dict):
raise AssertionError(repr(type(actual)))
assert_equal(len(actual), len(desired), err_msg)
for k, i in desired.items():
if k not in actual:
raise AssertionError(f"{k} not in {actual}")
assert_equal(actual[k], desired[k], f'key={k!r}\n{err_msg}')
return
# Case #2: lists .....
if isinstance(desired, (list, tuple)) and isinstance(actual, (list, tuple)):
return _assert_equal_on_sequences(actual, desired, err_msg='')
if not (isinstance(actual, ndarray) or isinstance(desired, ndarray)):
msg = build_err_msg([actual, desired], err_msg,)
if not desired == actual:
raise AssertionError(msg)
return
# Case #4. arrays or equivalent
if ((actual is masked) and not (desired is masked)) or \
((desired is masked) and not (actual is masked)):
msg = build_err_msg([actual, desired],
err_msg, header='', names=('x', 'y'))
raise ValueError(msg)
actual = np.asanyarray(actual)
desired = np.asanyarray(desired)
(actual_dtype, desired_dtype) = (actual.dtype, desired.dtype)
if actual_dtype.char == "S" and desired_dtype.char == "S":
return _assert_equal_on_sequences(actual.tolist(),
desired.tolist(),
err_msg='')
return assert_array_equal(actual, desired, err_msg)
def fail_if_equal(actual, desired, err_msg='',):
"""
Raises an assertion error if two items are equal.
"""
if isinstance(desired, dict):
if not isinstance(actual, dict):
raise AssertionError(repr(type(actual)))
fail_if_equal(len(actual), len(desired), err_msg)
for k, i in desired.items():
if k not in actual:
raise AssertionError(repr(k))
fail_if_equal(actual[k], desired[k], f'key={k!r}\n{err_msg}')
return
if isinstance(desired, (list, tuple)) and isinstance(actual, (list, tuple)):
fail_if_equal(len(actual), len(desired), err_msg)
for k in range(len(desired)):
fail_if_equal(actual[k], desired[k], f'item={k!r}\n{err_msg}')
return
if isinstance(actual, np.ndarray) or isinstance(desired, np.ndarray):
return fail_if_array_equal(actual, desired, err_msg)
msg = build_err_msg([actual, desired], err_msg)
if not desired != actual:
raise AssertionError(msg)
assert_not_equal = fail_if_equal
def assert_almost_equal(actual, desired, decimal=7, err_msg='', verbose=True):
"""
Asserts that two items are almost equal.
The test is equivalent to abs(desired-actual) < 0.5 * 10**(-decimal).
"""
if isinstance(actual, np.ndarray) or isinstance(desired, np.ndarray):
return assert_array_almost_equal(actual, desired, decimal=decimal,
err_msg=err_msg, verbose=verbose)
msg = build_err_msg([actual, desired],
err_msg=err_msg, verbose=verbose)
if not round(abs(desired - actual), decimal) == 0:
raise AssertionError(msg)
assert_close = assert_almost_equal
def assert_array_compare(comparison, x, y, err_msg='', verbose=True, header='',
fill_value=True):
"""
Asserts that comparison between two masked arrays is satisfied.
The comparison is elementwise.
"""
# Allocate a common mask and refill
m = mask_or(getmask(x), getmask(y))
x = masked_array(x, copy=False, mask=m, keep_mask=False, subok=False)
y = masked_array(y, copy=False, mask=m, keep_mask=False, subok=False)
if ((x is masked) and not (y is masked)) or \
((y is masked) and not (x is masked)):
msg = build_err_msg([x, y], err_msg=err_msg, verbose=verbose,
header=header, names=('x', 'y'))
raise ValueError(msg)
# OK, now run the basic tests on filled versions
return np.testing.assert_array_compare(comparison,
x.filled(fill_value),
y.filled(fill_value),
err_msg=err_msg,
verbose=verbose, header=header)
def assert_array_equal(x, y, err_msg='', verbose=True):
"""
Checks the elementwise equality of two masked arrays.
"""
assert_array_compare(operator.__eq__, x, y,
err_msg=err_msg, verbose=verbose,
header='Arrays are not equal')
def fail_if_array_equal(x, y, err_msg='', verbose=True):
"""
Raises an assertion error if two masked arrays are not equal elementwise.
"""
def compare(x, y):
return (not np.all(approx(x, y)))
assert_array_compare(compare, x, y, err_msg=err_msg, verbose=verbose,
header='Arrays are not equal')
def assert_array_approx_equal(x, y, decimal=6, err_msg='', verbose=True):
"""
Checks the equality of two masked arrays, up to given number odecimals.
The equality is checked elementwise.
"""
def compare(x, y):
"Returns the result of the loose comparison between x and y)."
return approx(x, y, rtol=10. ** -decimal)
assert_array_compare(compare, x, y, err_msg=err_msg, verbose=verbose,
header='Arrays are not almost equal')
def assert_array_almost_equal(x, y, decimal=6, err_msg='', verbose=True):
"""
Checks the equality of two masked arrays, up to given number odecimals.
The equality is checked elementwise.
"""
def compare(x, y):
"Returns the result of the loose comparison between x and y)."
return almost(x, y, decimal)
assert_array_compare(compare, x, y, err_msg=err_msg, verbose=verbose,
header='Arrays are not almost equal')
def assert_array_less(x, y, err_msg='', verbose=True):
"""
Checks that x is smaller than y elementwise.
"""
assert_array_compare(operator.__lt__, x, y,
err_msg=err_msg, verbose=verbose,
header='Arrays are not less-ordered')
def assert_mask_equal(m1, m2, err_msg=''):
"""
Asserts the equality of two masks.
"""
if m1 is nomask:
assert_(m2 is nomask)
if m2 is nomask:
assert_(m1 is nomask)
assert_array_equal(m1, m2, err_msg=err_msg)

View File

@ -0,0 +1,442 @@
import timeit
from functools import reduce
import numpy as np
import numpy._core.fromnumeric as fromnumeric
from numpy.testing import build_err_msg
pi = np.pi
class ModuleTester:
def __init__(self, module):
self.module = module
self.allequal = module.allequal
self.arange = module.arange
self.array = module.array
self.concatenate = module.concatenate
self.count = module.count
self.equal = module.equal
self.filled = module.filled
self.getmask = module.getmask
self.getmaskarray = module.getmaskarray
self.id = id
self.inner = module.inner
self.make_mask = module.make_mask
self.masked = module.masked
self.masked_array = module.masked_array
self.masked_values = module.masked_values
self.mask_or = module.mask_or
self.nomask = module.nomask
self.ones = module.ones
self.outer = module.outer
self.repeat = module.repeat
self.resize = module.resize
self.sort = module.sort
self.take = module.take
self.transpose = module.transpose
self.zeros = module.zeros
self.MaskType = module.MaskType
try:
self.umath = module.umath
except AttributeError:
self.umath = module.core.umath
self.testnames = []
def assert_array_compare(self, comparison, x, y, err_msg='', header='',
fill_value=True):
"""
Assert that a comparison of two masked arrays is satisfied elementwise.
"""
xf = self.filled(x)
yf = self.filled(y)
m = self.mask_or(self.getmask(x), self.getmask(y))
x = self.filled(self.masked_array(xf, mask=m), fill_value)
y = self.filled(self.masked_array(yf, mask=m), fill_value)
if (x.dtype.char != "O"):
x = x.astype(np.float64)
if isinstance(x, np.ndarray) and x.size > 1:
x[np.isnan(x)] = 0
elif np.isnan(x):
x = 0
if (y.dtype.char != "O"):
y = y.astype(np.float64)
if isinstance(y, np.ndarray) and y.size > 1:
y[np.isnan(y)] = 0
elif np.isnan(y):
y = 0
try:
cond = (x.shape == () or y.shape == ()) or x.shape == y.shape
if not cond:
msg = build_err_msg([x, y],
err_msg
+ f'\n(shapes {x.shape}, {y.shape} mismatch)',
header=header,
names=('x', 'y'))
assert cond, msg
val = comparison(x, y)
if m is not self.nomask and fill_value:
val = self.masked_array(val, mask=m)
if isinstance(val, bool):
cond = val
reduced = [0]
else:
reduced = val.ravel()
cond = reduced.all()
reduced = reduced.tolist()
if not cond:
match = 100-100.0*reduced.count(1)/len(reduced)
msg = build_err_msg([x, y],
err_msg
+ '\n(mismatch %s%%)' % (match,),
header=header,
names=('x', 'y'))
assert cond, msg
except ValueError as e:
msg = build_err_msg([x, y], err_msg, header=header, names=('x', 'y'))
raise ValueError(msg) from e
def assert_array_equal(self, x, y, err_msg=''):
"""
Checks the elementwise equality of two masked arrays.
"""
self.assert_array_compare(self.equal, x, y, err_msg=err_msg,
header='Arrays are not equal')
@np.errstate(all='ignore')
def test_0(self):
"""
Tests creation
"""
x = np.array([1., 1., 1., -2., pi/2.0, 4., 5., -10., 10., 1., 2., 3.])
m = [1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]
xm = self.masked_array(x, mask=m)
xm[0]
@np.errstate(all='ignore')
def test_1(self):
"""
Tests creation
"""
x = np.array([1., 1., 1., -2., pi/2.0, 4., 5., -10., 10., 1., 2., 3.])
y = np.array([5., 0., 3., 2., -1., -4., 0., -10., 10., 1., 0., 3.])
m1 = [1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]
m2 = [0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1]
xm = self.masked_array(x, mask=m1)
ym = self.masked_array(y, mask=m2)
xf = np.where(m1, 1.e+20, x)
xm.set_fill_value(1.e+20)
assert((xm-ym).filled(0).any())
s = x.shape
assert(xm.size == reduce(lambda x, y:x*y, s))
assert(self.count(xm) == len(m1) - reduce(lambda x, y:x+y, m1))
for s in [(4, 3), (6, 2)]:
x.shape = s
y.shape = s
xm.shape = s
ym.shape = s
xf.shape = s
assert(self.count(xm) == len(m1) - reduce(lambda x, y:x+y, m1))
@np.errstate(all='ignore')
def test_2(self):
"""
Tests conversions and indexing.
"""
x1 = np.array([1, 2, 4, 3])
x2 = self.array(x1, mask=[1, 0, 0, 0])
x3 = self.array(x1, mask=[0, 1, 0, 1])
x4 = self.array(x1)
# test conversion to strings, no errors
str(x2)
repr(x2)
# tests of indexing
assert type(x2[1]) is type(x1[1])
assert x1[1] == x2[1]
x1[2] = 9
x2[2] = 9
self.assert_array_equal(x1, x2)
x1[1:3] = 99
x2[1:3] = 99
x2[1] = self.masked
x2[1:3] = self.masked
x2[:] = x1
x2[1] = self.masked
x3[:] = self.masked_array([1, 2, 3, 4], [0, 1, 1, 0])
x4[:] = self.masked_array([1, 2, 3, 4], [0, 1, 1, 0])
x1 = np.arange(5)*1.0
x2 = self.masked_values(x1, 3.0)
x1 = self.array([1, 'hello', 2, 3], object)
x2 = np.array([1, 'hello', 2, 3], object)
# check that no error occurs.
x1[1]
x2[1]
assert x1[1:1].shape == (0,)
# Tests copy-size
n = [0, 0, 1, 0, 0]
m = self.make_mask(n)
m2 = self.make_mask(m)
assert(m is m2)
m3 = self.make_mask(m, copy=1)
assert(m is not m3)
@np.errstate(all='ignore')
def test_3(self):
"""
Tests resize/repeat
"""
x4 = self.arange(4)
x4[2] = self.masked
y4 = self.resize(x4, (8,))
assert self.allequal(self.concatenate([x4, x4]), y4)
assert self.allequal(self.getmask(y4), [0, 0, 1, 0, 0, 0, 1, 0])
y5 = self.repeat(x4, (2, 2, 2, 2), axis=0)
self.assert_array_equal(y5, [0, 0, 1, 1, 2, 2, 3, 3])
y6 = self.repeat(x4, 2, axis=0)
assert self.allequal(y5, y6)
y7 = x4.repeat((2, 2, 2, 2), axis=0)
assert self.allequal(y5, y7)
y8 = x4.repeat(2, 0)
assert self.allequal(y5, y8)
@np.errstate(all='ignore')
def test_4(self):
"""
Test of take, transpose, inner, outer products.
"""
x = self.arange(24)
y = np.arange(24)
x[5:6] = self.masked
x = x.reshape(2, 3, 4)
y = y.reshape(2, 3, 4)
assert self.allequal(np.transpose(y, (2, 0, 1)), self.transpose(x, (2, 0, 1)))
assert self.allequal(np.take(y, (2, 0, 1), 1), self.take(x, (2, 0, 1), 1))
assert self.allequal(np.inner(self.filled(x, 0), self.filled(y, 0)),
self.inner(x, y))
assert self.allequal(np.outer(self.filled(x, 0), self.filled(y, 0)),
self.outer(x, y))
y = self.array(['abc', 1, 'def', 2, 3], object)
y[2] = self.masked
t = self.take(y, [0, 3, 4])
assert t[0] == 'abc'
assert t[1] == 2
assert t[2] == 3
@np.errstate(all='ignore')
def test_5(self):
"""
Tests inplace w/ scalar
"""
x = self.arange(10)
y = self.arange(10)
xm = self.arange(10)
xm[2] = self.masked
x += 1
assert self.allequal(x, y+1)
xm += 1
assert self.allequal(xm, y+1)
x = self.arange(10)
xm = self.arange(10)
xm[2] = self.masked
x -= 1
assert self.allequal(x, y-1)
xm -= 1
assert self.allequal(xm, y-1)
x = self.arange(10)*1.0
xm = self.arange(10)*1.0
xm[2] = self.masked
x *= 2.0
assert self.allequal(x, y*2)
xm *= 2.0
assert self.allequal(xm, y*2)
x = self.arange(10)*2
xm = self.arange(10)*2
xm[2] = self.masked
x /= 2
assert self.allequal(x, y)
xm /= 2
assert self.allequal(xm, y)
x = self.arange(10)*1.0
xm = self.arange(10)*1.0
xm[2] = self.masked
x /= 2.0
assert self.allequal(x, y/2.0)
xm /= self.arange(10)
self.assert_array_equal(xm, self.ones((10,)))
x = self.arange(10).astype(np.float64)
xm = self.arange(10)
xm[2] = self.masked
x += 1.
assert self.allequal(x, y + 1.)
@np.errstate(all='ignore')
def test_6(self):
"""
Tests inplace w/ array
"""
x = self.arange(10, dtype=np.float64)
y = self.arange(10)
xm = self.arange(10, dtype=np.float64)
xm[2] = self.masked
m = xm.mask
a = self.arange(10, dtype=np.float64)
a[-1] = self.masked
x += a
xm += a
assert self.allequal(x, y+a)
assert self.allequal(xm, y+a)
assert self.allequal(xm.mask, self.mask_or(m, a.mask))
x = self.arange(10, dtype=np.float64)
xm = self.arange(10, dtype=np.float64)
xm[2] = self.masked
m = xm.mask
a = self.arange(10, dtype=np.float64)
a[-1] = self.masked
x -= a
xm -= a
assert self.allequal(x, y-a)
assert self.allequal(xm, y-a)
assert self.allequal(xm.mask, self.mask_or(m, a.mask))
x = self.arange(10, dtype=np.float64)
xm = self.arange(10, dtype=np.float64)
xm[2] = self.masked
m = xm.mask
a = self.arange(10, dtype=np.float64)
a[-1] = self.masked
x *= a
xm *= a
assert self.allequal(x, y*a)
assert self.allequal(xm, y*a)
assert self.allequal(xm.mask, self.mask_or(m, a.mask))
x = self.arange(10, dtype=np.float64)
xm = self.arange(10, dtype=np.float64)
xm[2] = self.masked
m = xm.mask
a = self.arange(10, dtype=np.float64)
a[-1] = self.masked
x /= a
xm /= a
@np.errstate(all='ignore')
def test_7(self):
"Tests ufunc"
d = (self.array([1.0, 0, -1, pi/2]*2, mask=[0, 1]+[0]*6),
self.array([1.0, 0, -1, pi/2]*2, mask=[1, 0]+[0]*6),)
for f in ['sqrt', 'log', 'log10', 'exp', 'conjugate',
# 'sin', 'cos', 'tan',
# 'arcsin', 'arccos', 'arctan',
# 'sinh', 'cosh', 'tanh',
# 'arcsinh',
# 'arccosh',
# 'arctanh',
# 'absolute', 'fabs', 'negative',
# # 'nonzero', 'around',
# 'floor', 'ceil',
# # 'sometrue', 'alltrue',
# 'logical_not',
# 'add', 'subtract', 'multiply',
# 'divide', 'true_divide', 'floor_divide',
# 'remainder', 'fmod', 'hypot', 'arctan2',
# 'equal', 'not_equal', 'less_equal', 'greater_equal',
# 'less', 'greater',
# 'logical_and', 'logical_or', 'logical_xor',
]:
try:
uf = getattr(self.umath, f)
except AttributeError:
uf = getattr(fromnumeric, f)
mf = getattr(self.module, f)
args = d[:uf.nin]
ur = uf(*args)
mr = mf(*args)
self.assert_array_equal(ur.filled(0), mr.filled(0), f)
self.assert_array_equal(ur._mask, mr._mask)
@np.errstate(all='ignore')
def test_99(self):
# test average
ott = self.array([0., 1., 2., 3.], mask=[1, 0, 0, 0])
self.assert_array_equal(2.0, self.average(ott, axis=0))
self.assert_array_equal(2.0, self.average(ott, weights=[1., 1., 2., 1.]))
result, wts = self.average(ott, weights=[1., 1., 2., 1.], returned=1)
self.assert_array_equal(2.0, result)
assert(wts == 4.0)
ott[:] = self.masked
assert(self.average(ott, axis=0) is self.masked)
ott = self.array([0., 1., 2., 3.], mask=[1, 0, 0, 0])
ott = ott.reshape(2, 2)
ott[:, 1] = self.masked
self.assert_array_equal(self.average(ott, axis=0), [2.0, 0.0])
assert(self.average(ott, axis=1)[0] is self.masked)
self.assert_array_equal([2., 0.], self.average(ott, axis=0))
result, wts = self.average(ott, axis=0, returned=1)
self.assert_array_equal(wts, [1., 0.])
w1 = [0, 1, 1, 1, 1, 0]
w2 = [[0, 1, 1, 1, 1, 0], [1, 0, 0, 0, 0, 1]]
x = self.arange(6)
self.assert_array_equal(self.average(x, axis=0), 2.5)
self.assert_array_equal(self.average(x, axis=0, weights=w1), 2.5)
y = self.array([self.arange(6), 2.0*self.arange(6)])
self.assert_array_equal(self.average(y, None), np.add.reduce(np.arange(6))*3./12.)
self.assert_array_equal(self.average(y, axis=0), np.arange(6) * 3./2.)
self.assert_array_equal(self.average(y, axis=1), [self.average(x, axis=0), self.average(x, axis=0) * 2.0])
self.assert_array_equal(self.average(y, None, weights=w2), 20./6.)
self.assert_array_equal(self.average(y, axis=0, weights=w2), [0., 1., 2., 3., 4., 10.])
self.assert_array_equal(self.average(y, axis=1), [self.average(x, axis=0), self.average(x, axis=0) * 2.0])
m1 = self.zeros(6)
m2 = [0, 0, 1, 1, 0, 0]
m3 = [[0, 0, 1, 1, 0, 0], [0, 1, 1, 1, 1, 0]]
m4 = self.ones(6)
m5 = [0, 1, 1, 1, 1, 1]
self.assert_array_equal(self.average(self.masked_array(x, m1), axis=0), 2.5)
self.assert_array_equal(self.average(self.masked_array(x, m2), axis=0), 2.5)
self.assert_array_equal(self.average(self.masked_array(x, m5), axis=0), 0.0)
self.assert_array_equal(self.count(self.average(self.masked_array(x, m4), axis=0)), 0)
z = self.masked_array(y, m3)
self.assert_array_equal(self.average(z, None), 20./6.)
self.assert_array_equal(self.average(z, axis=0), [0., 1., 99., 99., 4.0, 7.5])
self.assert_array_equal(self.average(z, axis=1), [2.5, 5.0])
self.assert_array_equal(self.average(z, axis=0, weights=w2), [0., 1., 99., 99., 4.0, 10.0])
@np.errstate(all='ignore')
def test_A(self):
x = self.arange(24)
x[5:6] = self.masked
x = x.reshape(2, 3, 4)
if __name__ == '__main__':
setup_base = ("from __main__ import ModuleTester \n"
"import numpy\n"
"tester = ModuleTester(module)\n")
setup_cur = "import numpy.ma.core as module\n" + setup_base
(nrepeat, nloop) = (10, 10)
for i in range(1, 8):
func = 'tester.test_%i()' % i
cur = timeit.Timer(func, setup_cur).repeat(nrepeat, nloop*10)
cur = np.sort(cur)
print("#%i" % i + 50*'.')
print(eval("ModuleTester.test_%i.__doc__" % i))
print(f'core_current : {cur[0]:.3f} - {cur[1]:.3f}')