[IDD #BLE-413085]: [IDD #ZQK-275228] RE: NLDN data set

Hi Jennifer,


re: Ah Ha!
> Not quite…

:-(

re:
> cola.gmu.edu[ldm]:~/data/lightning # ls -lt
> total 260
> -rw-r--r-- 1 ldm cola 3108 Aug 29 17:29 2017(241:mm)(241:dd)2129.nldn
> -rw-r--r-- 1 ldm cola 2240 Aug 29 17:28 2017(241:mm)(241:dd)2128.nldn
> -rw-r--r-- 1 ldm cola 3080 Aug 29 17:27 2017(241:ddd)2127.nldn

OK, when in doubt, read the manual... I just did read the LDM reference
guide on pqact:

http://www.unidata.ucar.edu/software/ldm/ldm-current/basics/pqact.conf.html

It is quite clear that the reference has to be to the day of the month,
so nothing that I said had any validity/value.

But, you have other options:

1) Replacment-Strings Derived from the Data-Product Creation-Time

   Best described in the LDM reference materials, URL above

2) pipe the product and Product ID into a script which then
   breaks down the CCYYJJJHHMMSS into CCYY MM DD HH MM SS
   and writes the product to the file you specify

Option 1) will work whenever the product is created (meaning first
inserted into an LDM queue for relay) on the same day as the data
represents.  This might be iffy at 0 UTC

Option 2) is straightforward and allows one to use the scripting
language of choice.  But, you must be aware/remember that it is the
duty of every "decoder" (the script in this case) to read the entire
product PIPEd to it by 'pqact'.  If it does not, then 'pqact' will
assume that the PIPE action failed for some reason, and it will try
it one more time while writing a dire-looking message to the LDM
log file.

re: how to store the data depends on how you intend to use it

> Yes, and I am not sure exactly what that is. For dazzling
> animations, I could see that minute-level resolution might
> be worth the bother, but for integrating with other data sources
> that don’t have that kind of resolution, it may be more of a nuisance.

The data records themselves have times down to the millisecond.
The products being distributed by the LDM are collections of those
records sent once-per-minute.  Dazzling animations will be possible
however you decide to write the received products to disk.

re:
> I am not sure what the final list of software tools will be. Others
> in our group have been doing all the data wrangling so far, I
> have just been asked to make sure everything can be read by GrADS.

OK, gotcha.

re:
> GrADS can read gridded arrays in NetCDF format, but I have not encountered
> any GLM data in NetCDF to say ‘YES’ for sure.

It is netCDF4 which is HDF5 underneath.  GrADS will need to be able to read
HDF5 to get at any of the GOES-16 imagery or products.

re:
> The lightning data seems better suited to station data format, so even if
> I had NetCDF I would probably convert it anyway.

That is exactly how the NLDN and USPLN lightning data are treated in
both GEMPAK and McIDAS, as point data.  The displays that can be made
are very nice, and are easily overlaid on top of other displays.  The
GLM lightning data, on the other hand, is a bit more complicated.

re:
> GOES-16 seems to be setting some new standards to keep us all on our toes.

Well said :-)

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: BLE-413085
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.