[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[IDD #EBR-704030]: Decoding NLDN

Subject: [IDD #EBR-704030]: Decoding NLDN
Date: Tue, 03 Oct 2017 13:58:27 -0600
Hi Jennifer,

First, many apologies for taking _so_ long to get to examining the
NLDN data you made available for me to use in testing!  Too many
things going on around here :-(

re:
> If you are only archiving a few days worth of raw data, I put a fresh set of 
> our LDM
> data files in ftp://cola.gmu.edu/pub/jma/nldn/.

I grabbed both of the .nldn files that you made available a few minutes ago,
thanks.

I ran my ldm-mcidas decoder (decoder that works with both NLDN and
USPLN lightning data and creates a McIDAS output POINT file) on both
of the files you made available by FTP, and I did not get any indication
of errors in my decoder output.

I put the log file generated by my decoder (which is 19 MB in size since
I ran the decoder in DEBUG mode), on our FTP server to grab if you feel
so inclined:

user:      anonymous
pass:      address@hidden
directory: pub/ldm-mcidas/log
file:      lgt2md.log

This file has a variety of things in it, but the important ones are
the results of reading each record, and the formatting of the values
read in each record.  Perhaps the most easily digested line is the
second from the end:

Oct 03 13:31:12 lgt2md[6735] INFO: 25218 NLDN flash records in MD file

This says that 25218 records were read, decoded, and written into
the McIDAS output POINT file.

Question:

- does the number of output records that my decoder match what yours
  did?

re:
> I also posted scan.txt, which has the decoded data in ascii format (grep for 
> “Err”
> to find the bad data) for the two files for 18z and 19z on 25sep2017.

The following is a comparison of the records that my decoder processed but your 
decoder
had problems with.  NB: the formats are a bit different, but one can see where 
things
match, and then don't.

ldm-mcidas decoder:

Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn...
Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362408  717386  19726  
-98701  -450   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362408 717386   19.7260  
-98.7010   -45.00   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime...
Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 
1506362408, cyd = 2017268, hms = 180008, stime = 25 Sep 2017
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180008 180008   
19.7260   98.7010   -45.00  0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN
Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180008 data into ./MDXX0208
Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 156
Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn...
Oct 03 13:30:15 lgt2md[6728] INFO: nldninput(): Product header record found
Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362409  948743  32866 
-103245  -330   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362409 948743   32.8660 
-103.2450   -33.00   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime...
Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 
1506362409, cyd = 2017268, hms = 180009, stime = 25 Sep 2017
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180009 180009   
32.8660  103.2450   -33.00  0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN
Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180009 data into ./MDXX0208
Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 157
Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn...
Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362409  949797  32870 
-103247  -160   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362409 949797   32.8700 
-103.2470   -16.00   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime...
Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 
1506362409, cyd = 2017268, hms = 180009, stime = 25 Sep 2017
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180009 180009   
32.8700  103.2470   -16.00  0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN
Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180009 data into ./MDXX0208
Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 158
Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn...
Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362411  596018  40349  
-96261  -150   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362411 596018   40.3490  
-96.2610   -15.00   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime...
Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 
1506362411, cyd = 2017268, hms = 180011, stime = 25 Sep 2017
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180011 180011   
40.3490   96.2610   -15.00  0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN
Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180011 data into ./MDXX0208
Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 159
Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn...
Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362411  601527  40342  
-96262  -640   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362411 601527   40.3420  
-96.2620   -64.00   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime...
Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 
1506362411, cyd = 2017268, hms = 180011, stime = 25 Sep 2017
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180011 180011   
40.3420   96.2620   -64.00  0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN
Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180011 data into ./MDXX0208
Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 160
Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn...
Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362411  646703  40349  
-96253  -190   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362411 646703   40.3490  
-96.2530   -19.00   0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime...
Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 
1506362411, cyd = 2017268, hms = 180011, stime = 25 Sep 2017
Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180011 180011   
40.3490   96.2530   -19.00  0
Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN
Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180011 data into ./MDXX0208
Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 161

your decoder:

OK  Record  156:    4424   1506362408       717386       19.73      -98.70   
-45.0    25sep2017-18:12
Err Record  157:    4452   1313621070            3        0.28  1195660.00  
2005.7    DDmonYEAR-HH:MM
Err Record  158:    4480   1299148320   1399156768   842342.44   943337.56  
1232.0    DDmonYEAR-HH:MM
Err Record  159:    4508    538976288   1937011305  1801810.38  1635020.12  
2494.0    DDmonYEAR-HH:MM
OK  Record  160:    4536   1506362409       948743       32.87     -103.25   
-33.0    25sep2017-18:16
OK  Record  161:    4564   1506362409       949797       32.87     -103.25   
-16.0    25sep2017-18:16
OK  Record  162:    4592   1506362411       596018       40.35      -96.26   
-15.0    25sep2017-18:10
OK  Record  163:    4620   1506362411       601527       40.34      -96.26   
-64.0    25sep2017-18:10

It looks to me like your decoder may not be correctly recognizing the
product header record that the file contains right after the 156th data
record.

re:
> Since I ran my
> decoder in that directory, I also left the GrADS station data file (.stn) and 
> a
> descriptor (.ctl) for completeness.

OK.  I didn't look at either of these.

Please let me know if what I wrote above make sense...

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: EBR-704030
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.
Prev by Date: [LDM #ANW-550486]: LDM access
Next by Date: [Support #FFU-213127]: RE: IDD feeds question
Previous by thread: [IDD #EBR-704030]: Decoding NLDN
Next by thread: [LDM #LIT-351851]: ipadress in allow
Index(es):
- Date
- Thread