Code Documentation¶
virga_detection.py¶
virga_detection.py¶
Core module for virga detection and masking.
- virga_sniffer.virga_detection.check_input_config(input_data: Dataset, config: dict, default_config: dict) None ¶
Check input requirements of virga_mask
- Parameters:
input_data (xarray.Dataset) –
config (dict) –
default_config (dict) –
- Return type:
None
- virga_sniffer.virga_detection.virga_mask(input_data: Dataset, config: Optional[dict] = None) Dataset ¶
This function identifies virga from input data of radar reflectivity, ceilometer cloud-base height and optionally doppler velocity, lifting condensation level and surface rain-sensor–rain-flag.
- Parameters:
input_data (xarray.Dataset) –
The input dataset with dimensions (‘time’,’range’,’layer’ (actual dimension-names doesn’t matter)).
Variables:
Ze: (‘time’,’range’) - radar reflectivity [dBz]
cloud_base_height: (‘time’,’layer’) - cloud base height [m]
vel: (‘time’, ‘range’) [optional] - radar doppler velocity [ms-1]
lcl: (‘time’) [optional] - lifting condensation level [m]
flag_surface_rain: (‘time’) [optional] - flags if rain at the surface [bool]
Coords:
time (‘time’) - datetime [UTC]
range (‘range’) - radar range gate altitude (mid of rangegate) [m]
layer (‘layer’) - counting cbh layer (np.arange(cbh.shape[1])) [-]
config (dict, optional) – The configuration flags and thresholds. Will be merged with the default configuration, see Configuration.
- Returns:
xarray.Dataset – The output dataset
——–
layer_utils.py¶
layer_utils.py¶
Functions to smooth, split, merge and sort layered data. Layered data are time periods of one or more layers of height data (e.g. cloud-base-height). Layer DataArrays are 2-D arrays of usually of dims (time,layer). The time dimension coordinate data is often required in numpy.datetime64 format, whereas layer dimension data is not required. The naming of the dimensions is irrelevant.
- virga_sniffer.layer_utils.clean(input_data: DataArray, clean_threshold: float) DataArray ¶
Clean input layer data by dropping layer with low number of data points according to clean_threshold.
- Parameters:
input_data (xarray.DataArray) – Input 2-D DataArray, assuming 2nd dimension refers to layer number.
clean_threshold (float) – Layer with number of datapoints < clean_threshold`*`input_data.shape[0] will be deleted
- Returns:
input_data but layer with no or low number of datapoints dropped. The 2nd dimension will be re-indexed.
- Return type:
- virga_sniffer.layer_utils.clean_N(input_data: DataArray, N: int) DataArray ¶
Clean input layer data by dropping layer with low number N of data points.
- Parameters:
input_data (xarray.DataArray) – Input 2-D DataArray, assuming 2nd dimension refers to layer number.
N (int) – Layer with number of datapoints < N
- Returns:
input_data but layer with no or low number of datapoints dropped. The 2nd dimension will be re-indexed.
- Return type:
- virga_sniffer.layer_utils.fill_nan(input_data: ~xarray.core.dataarray.DataArray, limit: ~typing.Optional[float] = None, method: str = 'ffill', return_mask: bool = True) -> (<class 'xarray.core.dataarray.DataArray'>, typing.Optional[numpy.ndarray[typing.Any, numpy.dtype[bool]]])¶
Fill input_data nan values using chosen xarray method. The gap limit is specified in seconds.
- Parameters:
input_data (xarray.DataArray) – The input layer data (N,M), where N coordinate data is assumed to be datetime64 type.
limit (float or None, optional) – The maximum gap of successive nan-values in seconds. If None fill all gaps. The default is None.
method ({'ffill', 'bfill', 'nearest', 'zero', 'slinear', 'quadratic', 'cubic', 'polynomial'}, optional) –
The method to apply for filling.
ffill: propagate last valid value forward (
xarray.DataArray.ffill()
)bfill: propagate the next valid value backward (
xarray.DataArray.bfill()
)nearest, zero, slinear, quadratic, cubic, polynomial: interpolate gaps using given method (
xarray.DataArray.interpolate_na()
)The default is ‘ffill’.
return_mask (bool, optional) – If True return array of bool indicating which data got filled (True if filled). The default is True.
- Returns:
- If return_mask is True, returns merged input_data and mask indicating filled data,
else only filled input_data is returned.
- Return type:
xarray.DataArray, (numpy.ndarray, optional)
- virga_sniffer.layer_utils.merge(input_data: DataArray, layer_threshold: float) DataArray ¶
Merging layer height data by comparing gap-filled lower layers to all layers above them. If distance smaller given threshold, upper layer data will be added to lower layer, or merged by mean value.
- Parameters:
input_data (xarray.DataArray) – The 2D input data of height values, assuming the 2nd dimension to be the layer number dimension
layer_threshold (float) – Distance threshold in [m]. Merge will be performed if distance of two layers is smaller than this threshold.
- Returns:
Copy of Input data, but layer data is merged. Note: intermediate layers might be all nan values.
- Return type:
- virga_sniffer.layer_utils.process_cbh(input_data: Dataset, config: dict = {}) Tuple[Union[DataArray, Any], ndarray, Union[ndarray, Any]] ¶
Cloud base height layer data preprocessing to split, merge, fill, sort and smooth the data for use in the virga sniffer.
- Parameters:
input_data (xarray.Dataset) –
The input dataset with dimensions (‘time’,’range’,’layer’ (actual dimension-names doesn’t matter)).
Variables:
cloud_base_height: (‘time’, ‘layer’) - cloud base height [m]
lcl: (‘time’) [optional] - lifting condensation level [m] (required if config[‘cbh_ident_function’] includes a 3 for merging lcl)
Coords:
time (‘time’) - datetime [UTC]
layer (‘layer’) - counting cbh layer (np.arange(cbh.shape[1])) [-]
config (dict, optional) – The configuration flags and thresholds. Will be merged with the default configuration, see Configuration.
- Returns:
xarray.DataArray – The processed cloud base height data
numpy.ndarray – Flag where LCL was merged into CBH at layer 0
numpy.ndarray – Flag where CBH layer data was filled by interpolation method config[‘cbh_fill_method’]
- virga_sniffer.layer_utils.replace_nan(input_data: ~xarray.core.dataarray.DataArray, input_layer: ~xarray.core.dataarray.DataArray, layer: int = 0, return_mask: bool = True) -> (<class 'xarray.core.dataarray.DataArray'>, typing.Optional[numpy.ndarray[typing.Any, numpy.dtype[bool]]])¶
Replace nan values of input_data nan values of certain layer (index of 2nd axis) with input_layer data.
- Parameters:
input_data (xarray.DataArray) – The 2D input data (M,N), assuming the 2nd dimension to be the layer number dimension.
input_layer (xarray.DataArray) – 1D layer data (M), will be used to replace nan values of input_data`[:,`layer].
layer (int, optional) – Index of N of input_data to be merged with input_layer. The default is 0.
return_mask (bool, optional) – If True return array of bool indicating which data got merged (True if merged). The default is True.
- Returns:
- If return_mask is True, returns merged input_data and mask indicating merged data,
else only merged input_data is returned.
- Return type:
xarray.DataArray, (numpy.ndarray, optional)
- virga_sniffer.layer_utils.smooth(input_data: DataArray, window: int) DataArray ¶
Smooth layered data using a rolling median filter of window size.
- Parameters:
input_data (xarray.DataArray) – Input DataArray either 1D or 2D, assuming first dimension of type np.datetime64.
window (int) – Smoothing window size in seconds
- Returns:
Copy of input_data but smoothed.
- Return type:
- virga_sniffer.layer_utils.sort(input_data: DataArray) DataArray ¶
Sort input layer data by layer mean value.
- Parameters:
input_data (xarray.DataArray) – Input 2-D DataArray, assuming 2nd dimension refers to layer number.
- Returns:
Copy of input_data but sorted .
- Return type:
- virga_sniffer.layer_utils.split(input_data: DataArray, layer_threshold: float) DataArray ¶
Split input layer data by creating new layer for values up or below layer_threshold from layer mean.
- Parameters:
input_data (xarray.DataArray) – Input 2-D DataArray (M,N), assuming 2nd dimension refers to layer number.
layer_threshold (float) –
- layer mean +- layer_threshold will be used to identify layer values to push to a new layer.
Same unit as input_data.values
- Returns:
Output 2-D DataArray (M,N+x), where x is number of newly created layer by splitting data using layer_threshold value. This DataArray is sorted using
layer_sort()
.- Return type:
utils.py¶
utils.py¶
Basic utility functions used for virga_detection.py
- virga_sniffer.utils.calc_lcl(p: Union[float, Iterable[float]], T: Union[float, Iterable[float]], rh: Union[None, float, Iterable[float]] = None, rhl: Union[None, float, Iterable[float]] = None, rhs: Union[None, float, Iterable[float]] = None, return_ldl: bool = False, return_min_lcl_ldl: bool = False) Union[float, ndarray] ¶
Calculate lifting condensation level (LCL) from surface pressure, temperature and humidity observations. Adapted from Romps (2017) adding numpy array functionality.
References
[1] Romps, D. M. (2017). Exact Expression for the Lifting Condensation Level, Journal of the Atmospheric Sciences, 74(12), 3891-3900. doi:10.1175/JAS-D-17-0102.1
- Parameters:
p (float or array_like) – Surface air pressure p in units Pascals [PA]
T (float or array_like) – Surface Air temperature T in units Kelvin [K].
rh (float or array_like or None, optional) –
- Exactly one of rh, rhl, and rhs must be specified.
rh: Relative humidity (dimensionless, from 0 to 1) with respect to liquid water if T >= 273.15 K and with respect to ice if T < 273.15 K.
rhl: Same as rh but solely with respect to liquid water.
rhs: Same as rh but solely with respect to ice.
If array_like, requires same shape as p and T. The default is None.
rhl (float or array_like or None, optional) –
- Exactly one of rh, rhl, and rhs must be specified.
rh: Relative humidity (dimensionless, from 0 to 1) with respect to liquid water if T >= 273.15 K and with respect to ice if T < 273.15 K.
rhl: Same as rh but solely with respect to liquid water.
rhs: Same as rh but solely with respect to ice.
If array_like, requires same shape as p and T. The default is None.
rhs (float or array_like or None, optional) –
- Exactly one of rh, rhl, and rhs must be specified.
rh: Relative humidity (dimensionless, from 0 to 1) with respect to liquid water if T >= 273.15 K and with respect to ice if T < 273.15 K.
rhl: Same as rh but solely with respect to liquid water.
rhs: Same as rh but solely with respect to ice.
If array_like, requires same shape as p and T. The default is None.
return_ldl (bool, optional) – If True, the lifting deposition level (LDL) is returned instead of LCL. The default is False.
return_min_lcl_ldl (bool, optional) – If true, the minimum of the LCL and LDL is returned. The default is False.
- Returns:
- Result equals:
LCL if return_ldl == False and return_min_lcl_ldl == False,
LDL if return_ld == True and return_min_lcl_ldl == False,
Min(LCL,LDL) if return_min_lcl_ldl == True and return_ldl == False.
Same shape as p.
- Return type:
- virga_sniffer.utils.fill_mask_gaps(mask: ndarray[Any, dtype[bool]], altitude: ndarray[Any, dtype[float]], max_gap: float, idxs_true: ndarray[Any, dtype[int]]) ndarray[Any, dtype[bool]] ¶
Fill vertical gaps in a boolean mask :param mask: :param altitude: :param max_gap: :param idxs_true:
- Return type:
- virga_sniffer.utils.get_firstgap_dn(gapidx, idxs)¶
Get the closest gap idx below idx
- virga_sniffer.utils.get_firstgap_up(gapidx, idxs, layer_constraint=True)¶
Get the closest gap idx above idx
- virga_sniffer.utils.get_gapidx(mask, fill_value=None)¶
Retrieve the index of gaps (False values) in a boolean mask.
- Parameters:
mask (array_like) – Boolean mask of shape (M,N)
fill_value (int or None, optional) – This value is used to pad the output array (axis 1). The default is M-1.
- Returns:
gapsidx – The index of gaps along dimension N
- Return type:
array_like, (M,L)
vsplot.py¶
vsplot.py¶
Methods for plotting Virga-Sniffer output, in the form of xarray dataset accessor “vsplot”.
- class virga_sniffer.vsplot.VirgaSnifferPlotAccessor(xarray_obj)¶
Add plotting functionality to output xr.Dataset of virga-sniffer as xr.Dataset.vsplot.
- plot_cbh(ax=None, colors=None, cbh=True, cth=True, lcl=True, fill=True, colorbar=True, fontsize=None, label_lcl='LCL from sfc-obs', label_fill='cloud base interpolated')¶
- Plot all cloud layer related output: cloud-base height, cloud-top height, lifitng-condensation-level,
and filled layer.
- Parameters:
ax (matplotlib.axes.Axes, optional) – The axes to assign the lines. If None uses matplotlib.pyplot.gca().
colors (list(str), optional) – List of colors as matplotlib string for cloud-layers. If list is shorter than layer-number, it gets repreated, if list is longer, it gets truncated.
cbh (bool, optional) – Plot cloud-base heights if True. The default is True.
cth (bool, optional) – Plot cloud-top heights if True. The default is True.
lcl (bool, optional) – Plot lifitng-condensation level as filled in cloud-base heights if True. The default is True.
fill (bool, optional) – Plot filled/interpolated data of cloud-base heights if True. The default is True.
colorbar (bool, optional) – Add a colorbar to ax if True. The default is True.
fontsize (dict, optional) – Configuration of fontsizes. Keywords and Defaults: “cbar_label”=16, “cbar_ticklabel”=14
label_lcl (str, optional) – Label of lifing-condensation level. The default is “LCL from sfc-obs”.
label_fill (str, optional) – Label of filled cloud-base heights. The default is “cloud base interpolated”.
- Return type:
matplotlib.axes.Axes, matplotlib.colorbar.Colorbar
- plot_cloud_mask(ax=None, color='#7570b3')¶
Plot cloud flag.
- Parameters:
ax (matplotlib.axes.Axes, optional) – The axes to assign the lines. If None uses matplotlib.pyplot.gca().
color (str, optional) – Matplotlib color, the default is “#7570b3”.
- Return type:
matplotlib.axes.Axes
- plot_flag_rain(ax=None, scale=None, fontsize=None, color='k')¶
Plot rain at surface flag.
- Parameters:
ax (matplotlib.axes.Axes, optional) – The axes to assign the lines. If None uses matplotlib.pyplot.gca().
scale (float, optional) – Scaling the flag value, e.g. set maximum. The default is 0.9*min(output.range).
fontsize (dict, optional) – Configuration of fontsizes. Keywords and Defaults: “legend”=16
color (str, optional) – Matplotlib color, the default is “k.
- Return type:
matplotlib.axes.Axes
- plot_virga_mask(ax=None, color='#d95f02')¶
Plot Virga flag.
- Parameters:
ax (matplotlib.axes.Axes, optional) – The axes to assign the lines. If None uses matplotlib.pyplot.gca().
color (str, optional) – Matplotlib color, the default is “#d95f02”.
- Return type:
matplotlib.axes.Axes
- plot_ze_mask(ax=None, color='#bdbdbd')¶
Plot radar signal flag.
- Parameters:
ax (matplotlib.axes.Axes, optional) – The axes to assign the lines. If None uses matplotlib.pyplot.gca().
color (str, optional) – Matplotlib color, the default is “#969696”.
- Return type:
matplotlib.axes.Axes
- quicklook_full(axs=None, ylim=None, radar='LIMRAD94', fontsize=None, plot_flags=None)¶
3-Panel combined formatted quicklook of radar reflectivity, doppler velocity and virga-sniffer output.
- Parameters:
axs (list(matplotlib.axes.Axes) of length 3, optional) – The axes to assign the lines. If None uses plt.subplots(3, 1, figsize=(15, 15), constrained_layout=True)
ylim (float, optional) – Limit y-axis to altitude [m]. The default is np.ceil(np.nanmax(self._obj.cloud_top_height) * 1e-3) * 1e3.
radar (str, optional) – Add radar name to colobar label, the default is “LIMRAD94”.
fontsize (dict, optional) – Configuration of fontsizes. Keywords and Defaults: “title”=18, “legend”=16, “ax_label”=16, “ax_ticklabel”=14, “cbar_label”=16, “cbar_ticklabel”=14
plot_flags (dict, optional) – Components to plot. Keywords and Defaults: “ze”=True, “virga”=True, “cloud”=True, “rainflag”=True
- Return type:
matplotlib.figure.Figure, matplotlib.axes.Axes, matplotlib.colorbar.Colorbar
- quicklook_vel(ax=None, vmin=-4, vmax=3, ylim=None, fontsize=None, rasterized=True)¶
Plot formatted quicklook of radar mean Doppler velocity. The mean Doppler velocity is plotted using the holly colormap from CMasher (https://doi.org/10.21105/joss.02004).
- Parameters:
ax (matplotlib.axes.Axes, optional) – The axes to assign the lines. If None uses matplotlib.pyplot.gca().
vmin (float, optional) – Lower limit of the colorbar. If None, vmin is set to np.floor(np.nanmin(vel)). The default is -4.
vmax (float, optional) – Upper limit of the colorbar. If None, vmax is set to np.ceil(np.nanmax(vel)). The default is 3.
ylim (float, optional) – Limit y-axis to altitude [m]. The default is np.ceil(np.nanmax(self._obj.cloud_top_height) * 1e-3) * 1e3.
fontsize (dict, optional) – Configuration of fontsizes. Keywords and Defaults: “legend”=16, “ax_label”=16, “ax_ticklabel”=14, “cbar_label”=16, “cbar_ticklabel”=14
rasterized (bool, optional) – Turn on rasterization of matplotlib.pyplot.pcolormesh. The default is True.
- Return type:
matplotlib.axes.Axes, matplotlib.colorbar.Colorbar
- quicklook_virga_mask(ax=None, ylim=None, legend=True, fontsize=None, plot_flags=None)¶
Plot formatted quicklook of virga-sniffer output.
- Parameters:
ax (matplotlib.axes.Axes, optional) – The axes to assign the lines. If None uses matplotlib.pyplot.gca().
ylim (float, optional) – Limit y-axis to altitude [m]. The default is np.ceil(np.nanmax(self._obj.cloud_top_height) * 1e-3) * 1e3.
legend (bool, optional) – If True add legend to plot. The default is True.
fontsize (dict, optional) – Configuration of fontsizes. Keywords and Defaults: “legend”=16, “ax_label”=16, “ax_ticklabel”=14, “cbar_label”=16, “cbar_ticklabel”=14
plot_flags (dict, optional) – Components to plot. Keywords and Defaults: “ze”=True, “virga”=True, “cloud”=True, “rainflag”=True
- Return type:
matplotlib.axes.Axes, matplotlib.colorbar.Colorbar
- quicklook_ze(ax=None, vmin=-40, vmax=20, ylim=None, fontsize=None, rasterized=True)¶
Plot formatted quicklook of radar reflectivity. The reflectivity is plotted using the pride colormap from CMasher (https://doi.org/10.21105/joss.02004).
- Parameters:
ax (matplotlib.axes.Axes, optional) – The axes to assign the lines. If None uses matplotlib.pyplot.gca().
vmin (float, optional) – Lower limit of the colorbar. If None, vmin is set to np.floor(np.nanmin(vel)). The default is -40.
vmax (float, optional) – Upper limit of the colorbar. If None, vmax is set to np.ceil(np.nanmax(vel)). The default is 20.
ylim (float, optional) – Limit y-axis to altitude [m]. The default is np.ceil(np.nanmax(self._obj.cloud_top_height) * 1e-3) * 1e3.
fontsize (dict, optional) – Configuration of fontsizes. Keywords and Defaults: “ax_label”=16, “ax_ticklabel”=14, “cbar_label”=16, “cbar_ticklabel”=14
rasterized (bool, optional) – Turn on rasterization of matplotlib.pyplot.pcolormesh. The default is True.
- Return type:
matplotlib.axes.Axes, matplotlib.colorbar.Colorbar