This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Hi Jennifer, First, many apologies for taking _so_ long to get to examining the NLDN data you made available for me to use in testing! Too many things going on around here :-( re: > If you are only archiving a few days worth of raw data, I put a fresh set of > our LDM > data files in ftp://cola.gmu.edu/pub/jma/nldn/. I grabbed both of the .nldn files that you made available a few minutes ago, thanks. I ran my ldm-mcidas decoder (decoder that works with both NLDN and USPLN lightning data and creates a McIDAS output POINT file) on both of the files you made available by FTP, and I did not get any indication of errors in my decoder output. I put the log file generated by my decoder (which is 19 MB in size since I ran the decoder in DEBUG mode), on our FTP server to grab if you feel so inclined: user: anonymous pass: address@hidden directory: pub/ldm-mcidas/log file: lgt2md.log This file has a variety of things in it, but the important ones are the results of reading each record, and the formatting of the values read in each record. Perhaps the most easily digested line is the second from the end: Oct 03 13:31:12 lgt2md[6735] INFO: 25218 NLDN flash records in MD file This says that 25218 records were read, decoded, and written into the McIDAS output POINT file. Question: - does the number of output records that my decoder match what yours did? re: > I also posted scan.txt, which has the decoded data in ascii format (grep for > “Err” > to find the bad data) for the two files for 18z and 19z on 25sep2017. The following is a comparison of the records that my decoder processed but your decoder had problems with. NB: the formats are a bit different, but one can see where things match, and then don't. ldm-mcidas decoder: Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn... Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362408 717386 19726 -98701 -450 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362408 717386 19.7260 -98.7010 -45.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime... Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 1506362408, cyd = 2017268, hms = 180008, stime = 25 Sep 2017 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180008 180008 19.7260 98.7010 -45.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180008 data into ./MDXX0208 Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 156 Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn... Oct 03 13:30:15 lgt2md[6728] INFO: nldninput(): Product header record found Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362409 948743 32866 -103245 -330 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362409 948743 32.8660 -103.2450 -33.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime... Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 1506362409, cyd = 2017268, hms = 180009, stime = 25 Sep 2017 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180009 180009 32.8660 103.2450 -33.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180009 data into ./MDXX0208 Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 157 Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn... Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362409 949797 32870 -103247 -160 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362409 949797 32.8700 -103.2470 -16.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime... Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 1506362409, cyd = 2017268, hms = 180009, stime = 25 Sep 2017 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180009 180009 32.8700 103.2470 -16.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180009 data into ./MDXX0208 Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 158 Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn... Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362411 596018 40349 -96261 -150 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362411 596018 40.3490 -96.2610 -15.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime... Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 1506362411, cyd = 2017268, hms = 180011, stime = 25 Sep 2017 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180011 180011 40.3490 96.2610 -15.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180011 data into ./MDXX0208 Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 159 Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn... Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362411 601527 40342 -96262 -640 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362411 601527 40.3420 -96.2620 -64.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime... Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 1506362411, cyd = 2017268, hms = 180011, stime = 25 Sep 2017 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180011 180011 40.3420 96.2620 -64.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180011 data into ./MDXX0208 Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 160 Oct 03 13:30:15 lgt2md[6728] DEBUG: In decode_nldn... Oct 03 13:30:15 lgt2md[6728] DEBUG: nldn_read:: 1506362411 646703 40349 -96253 -190 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_nldn:: 1506362411 646703 40.3490 -96.2530 -19.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In sectodaytime... Oct 03 13:30:15 lgt2md[6728] DEBUG: sectodaytime:: secs since 1970 = 1506362411, cyd = 2017268, hms = 180011, stime = 25 Sep 2017 Oct 03 13:30:15 lgt2md[6728] DEBUG: decode_lgt:: 2017268 180011 180011 40.3490 96.2530 -19.00 0 Oct 03 13:30:15 lgt2md[6728] DEBUG: In write_mcidas, lightning type = NLDN Oct 03 13:30:15 lgt2md[6728] DEBUG: Decoding 2017268.180011 data into ./MDXX0208 Oct 03 13:30:15 lgt2md[6728] DEBUG: In file_observation, row = 1, col = 161 your decoder: OK Record 156: 4424 1506362408 717386 19.73 -98.70 -45.0 25sep2017-18:12 Err Record 157: 4452 1313621070 3 0.28 1195660.00 2005.7 DDmonYEAR-HH:MM Err Record 158: 4480 1299148320 1399156768 842342.44 943337.56 1232.0 DDmonYEAR-HH:MM Err Record 159: 4508 538976288 1937011305 1801810.38 1635020.12 2494.0 DDmonYEAR-HH:MM OK Record 160: 4536 1506362409 948743 32.87 -103.25 -33.0 25sep2017-18:16 OK Record 161: 4564 1506362409 949797 32.87 -103.25 -16.0 25sep2017-18:16 OK Record 162: 4592 1506362411 596018 40.35 -96.26 -15.0 25sep2017-18:10 OK Record 163: 4620 1506362411 601527 40.34 -96.26 -64.0 25sep2017-18:10 It looks to me like your decoder may not be correctly recognizing the product header record that the file contains right after the 156th data record. re: > Since I ran my > decoder in that directory, I also left the GrADS station data file (.stn) and > a > descriptor (.ctl) for completeness. OK. I didn't look at either of these. Please let me know if what I wrote above make sense... Cheers, Tom -- **************************************************************************** Unidata User Support UCAR Unidata Program (303) 497-8642 P.O. Box 3000 address@hidden Boulder, CO 80307 ---------------------------------------------------------------------------- Unidata HomePage http://www.unidata.ucar.edu **************************************************************************** Ticket Details =================== Ticket ID: EBR-704030 Department: Support IDD Priority: Normal Status: Closed =================== NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.