wiki:cookbook/RADVOL-QC_SPIKE
Last modified 3 years ago Last modified on 09/04/14 18:45:03

1. Algorithm name

Removal of geometrically-shaped non-meteorological echoes and quality characterization (as a part of RADVOL-QC package) – RADVOL-QC: SPIKE

2. Basic description

a) Physical basis of the algorithm

Investigation of echo patterns to detect non-meteorological patterns such as generated by Sun, external emitters, etc.

b) Amount of validation performed so far

Works operationally in IMGW since 2011 to correct data before using for Meteo Flight system (for air traffic control).

c) References (names and contact information of all developers during the evolutionary history, scientific papers)

  • Operational work: IMGW, Department of Ground Based Remote Sensing.
  • Scientific paper: Ośródka, K., Szturc, J., and Jurczyk, A., 2014. Chain of data quality algorithms for 3-D single-polarization radar reflectivity (RADVOL-QC system). Meteorol. Appl., 21, 256-270.

3. ODIM metadata requirements for I/O

  • Dataset-specific “what”: gain, offset, nodata, undetect.
  • Dataset-specific “where”: elangle, nbins, nrays, rscale.

4. Input data

a) What kind of radar data (including the list of previous algorithms and quality flags applied)

object=PVOL or SCAN; quantity=DBZH, otherwise TH.

b) Other data (optional and mandatory, applying “universally” agreed formats, geometry)

None

5. Logical steps, using any of: text, flow charts, graphics, equations (or references to equations), conditional branches in “all possible cases”.

The spurious echoes from sun and external interference, so called spikes, are characterized by their spatial structure that clearly differs from precipitation field pattern. The shape of such echo is very specific: it is similar to a spike along the whole or large part of a single or a few neighbouring radar beams. Reflectivity field structure is investigated to detect such echo on radar image in this algorithm, which identifies spikes, cuts them out from the precipitation field, and replaces by proper reflectivity values.

In the algorithm two stages of spike removal are introduced: for “wide” (subalgorithm A) and “narrow” (subalgorithm B) types of spikes.

Set of the algorithm parameters

Description Denotation Default value
QISPIKE value for external interference signals SPIKE_QI 0.5
QISPIKE value for uncorrected external interference signals SPIKE_QIUn 0.3
Maximum fraction of echo cover to apply the correction (algorithm A) SPIKE_ACovFrac 0.9
Number of azimuths to find variance across beam (algorithm A) SPIKE_AAzim 3
Threshold for variance across beam (algorithm A) (dBZ2) SPIKE_AVarAzim 1 000
Number of pixels along beam to find variance (algorithm A) SPIKE_ABeam 15
Threshold for variance along beam (algorithm A) ((mm6 m-3)2) SPIKE_AVarBeam 5
Minimum fraction of potential spike gates in beam (algorithm A) SPIKE_AFrac 0.45
Minimum difference between the potential spike and vicinity (algorithm B) (dBZ) SPIKE_BDiff 10
Number of azimuths to find reflectivity gradient (algorithm B) SPIKE_BAzim 3
Minimum fraction of potential spike gates in beam (algorithm B) SPIKE_BFrac 0.25

At first the XML file is checked whether there exists group for a considered radar (based on the radar name read from "what"/source(NOD)), which contains the algorithm parameters. If "yes", then parameters are read from that XML group, but if it is impossible for a particular parameter, then default value from source code is taken. If the group does not exist, parameters are read from <default> group in XML file in analogous way.

If the algorithm is run by means of BALTRAD toolbox then all the algorirthm parameters for each specific radar should be placed in relevant XML file by the BALTRAD system admin. Default parameters are placed in the file by admin as well. Moreover, the algorithm default parameters are also included in software.

“Wide” spike removal

For “wide” spikes a subalgorithm based on analysis of spatial structure of radar echo is employed. It is run if fraction of echo cover is smaller than SPIKE_ACovFrac. The variability of the echo across and along radar beam is examined using locally determined reflectivity variances, so a given echo is classified as potential spike if the first variance is high whereas the latter is low:

$ \text{if}  $

$ (var_{across}(Z(\alpha,l)[\text{dBZ}])> \text{SPIKE}\_\text{AVarAzim}) $

$ \text{and  }  $

$ (var_{along}(Z(\alpha,l)[\text{mm6/m3}]) < \text{SPIKE}\_\text{AVarBeam}) $

$ \text{then}  $

$ \text{there is potential spike in the gate }(\alpha,l)  $

where varacross is the variance for a given gate calculated across radar beam at distance up to ±SPIKE_AAzim (°) in azimuth; varalong is the variance in a given gate calculated along radar beam at distance up to ±SPIKE_ABeam (km); Z is the radar reflectivity expressed in dBZ or mm6/m3. If fraction of gates along radar beam classified as burdened with potential spike is higher than SPIKE_AFrac then presence of A-type spike in the gates is confirmed.

“Narrow” spike removal

The second subalgorithm is employed to recognize and remove “narrow” spikes, i.e. not wider than ±SPIKE_BAzim (°) in azimuth. In the first step gates with potential “narrow” spikes are detected. This subalgorithm is applied to each gate (α, l) with echo detected, that means Z(α, l) > –32 dBZ (which is the lowest reflectivity measured by radar). For these gates reflectivity at neighbouring beams of azimuth α±d (where d = SPIKE_BAzim (°)) in the same distance to radar l are checked:

$ \text{if} $

$ (((Z(\alpha -d,l)=-32 \text{dBZ}) \text{ and } (Z(\alpha,l)- Z(\alpha -d,l)>\text{SPIKE}\_\text{BDiff} )) $

$ \text{or } (\text{spike A in gate }(\alpha -d,l)) \text{ or } (\text{potential spike B in gate } (\alpha -d,l))) $

$ \text{and } $

$ (((Z(\alpha+d,l)=-32 \text{dBZ}) \text{ and } (Z(\alpha,l)- Z(\alpha +d,l)>\text{SPIKE}\_\text{BDiff} )) $

$ \text{or } (\text{spike A in gate }(\alpha +d,l)) \text{ or } (\text{potential spike B in gate } (\alpha +d,l))) $

$ \text{then}  $

$ \text{there is potential spike B in the gate }(\alpha,l)  $

The procedure is repeated for d = SPIKE_BAzim - 1 downto 1. Next number of gates with potential spike B along given radar beam is computed and if their fraction is higher than SPIKE_BFrac then then presence of B-type spike in the gates is confirmed.

Finally, all the gates with spikes of A- or B-type are replaced by reflectivities interpolated from neighbouring beams.

Quality index

Quality index QISPIKE for the whole radar beam in which spike is detected equals:

$ QI_{SPIKE} = \begin{cases}
\text{SPIKE}\_\text{QI}  & \textrm{for gate with spike echo\ } \\
1                        & \textrm{for gate without spike echo \ } 
\end{cases} $

Diagram

Generally:

GraphViz image

6. Output

a) Data type using ODIM notation where possible, e.g. DBZH

Corrected DBZH, with "pl.imgw.radvolqc.spike" added to data-specific "how"/task, and the algorithm parameters added to "how"/task_args.

b) Quality index (QI) field

Quality index (QI = 0 for bad data, QI = 1 for excellent data) with "pl.imgw.radvolqc.spike" in quality-specific "how"/task, and the algorithm parameters in "how"/task_args.

7. Outline of a test concept exemplifying the algorithm, as a suggestion for checking that an implementation has been successful.