[IDD #EBR-704030]: Decoding NLDN

Hi Jennifer,

re:
> I am starting a new ticket, even though I’ve been conversing with Tom Y. 
> about NLDN in
> other threads, because the subject has changed — our LDM is getting the data 
> just fine,
> now I am working on decoding it.

You may have been hoping to "talk" to someone else, but you got me (Tom Y) 
instead :-)

re:
> I am trying to translate NLDN records into GrADS station
> data reports. I am scanning a file named 20172420000.nldn (for julian day 
> 242) and am not
> getting past the first record without errors. My colleague (cc’d on this 
> email) is using
> Matlab to extract the data and succeeding. My data concurs with Matlab for 
> the the date,
> lat, lon, and signal strength, but I am failing to get anything but NULL for 
> the final six
> 'char[1]’ elements; Matlab is returning valid numbers for multiplicity, 
> semi-major axis,
> eccentricity, ellipse angle, and chi-square.

I would be suspicious that the Matlab code is returning incorrect values IF
the indicated multiplicity is anything other than 0.  The multiplicity has
been 0 since Vaisala started reporting strokes instead of flashes some time ago.

re:
> I say this is 
> Also, after reading 112 bytes, I am failing to
> find another “NLDN” marker indicating the start of a new record. With three 
> 28-byte header
> blocks plus four 4-byte ints, three 2-byte shorts, and six 1-byte chars, the 
> record should
> be 112 bytes long.

Each NLDN product sent via the LDM will have one 'NLDN' marker.  If your LDM 
pattern-action
file action is combining more than one product into an output file, then you 
should find
more than one 'NLDN' marker.

re:
> Here is the output from my C program:
> 
> Scanning 20172420000.nldn
> Record 1 starts at byte 0
> 3 header blocks
> tsec = 1504051176
> nsec = 902188
> lat = 34.014
> lon = -110.183
> signal = 6.000000
> mult =
> fill =
> semimaj =
> angle =
> chisqr =
> NLDN indicator not found at byte 112

Comments:

- NLDN flash record format:
 
  int[4]   seconds since 1970
  int[4]   nanoseconds since tsec (appears be milliseconds)
  int[4]   latitude [deg] * 1000
  int[4]   longitude [deg] * 1000 (west negative convention)
  short[2] null padding
  short[2] signal strength * 10 (150 NLDN measures ~= 30 kAmps)
  short[2] null padding
  char[1]  multiplicity (#strokes per flash)
  char[1]  null padding
  char[1]  semi-major axis
  char[1]  eccentricity
  char[1]  ellipse angle
  char[1]  chi-square

- mult should be 0

- fill values are currently all zero

- semimaj may well be 0

  The value reported is the result of int(semimajor) where semimajor is a float
  which may be less than 1.

- angle

  Your code does not report a value for eccentricity which it extracts.

- chisqr

  chisqr may be zero as it is the result of int(chisquare) where chisquare is
  a float that may be less than 1.

> 
> And here is what Matlab found (just the first three records are shown):
> 
> 2017-08-30 00:14:38.188, 34.014000, -110.183000, 6.00, 0, 0, 0, 81, 0
> 2017-08-30 00:15:23.370, 50.090000, -88.811000, -5.00, 0, 5, 9, 80, 0
> 2017-08-30 00:15:55.652, 50.214000, -88.857000, -10.00, 0, 4, 9, 8220, 1

Assuming that the order of this listing is the same as the order in your
listing (meaning that the values after signal strength are multiplicity,
semi-major axis, eccentricity, ellipse angle and then chi-square, I have
to say that this output looks to be incorrect.

re:
> My reading of the tsec and nsec variables from the first record do translate 
> into an exact
> date match with Matlab, and my lat/lon and signal values also match. But 
> after that I don't
> understand how Matlab can read a 1-byte char and turn it into a number like 
> ‘8220’.

I agree completely.

re:
> I copied the relevant C code below. I am working on a little-endian machine, 
> and so I am
> using a couple of byte-swapping routines I stole from stackoverflow.com for 
> the ints and
> shorts (but not the chars).

OK.

re:
> Is it obvious what am I missing?

No.

If you want me to check a particular file of values you are ingesting with
code that I wrote for McIDAS, please send the file as an attachment, and I
will run it through my decoder.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: EBR-704030
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.