[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Datastream #QJW-779379]: NLDN lightning data
- Subject: [Datastream #QJW-779379]: NLDN lightning data
- Date: Wed, 20 Mar 2013 08:15:38 -0600
Hi Rick,
re:
> I hope you can answer this question. We're using NLDN data
> and there are quite a few holes in our archive. I'm not
> sure if this is because the data files aren't created when
> there's zero lightning in the data domain, or if we just have
> an unreliable data feed. I realize this is somewhat ambiguous
> but would like to know if the files are produced even when
> there's no data. Thanks.
The NLDN IDD products are no longer produced when there is no data;
they used to be, but this was changed a couple of years ago.
David Knight said:
> As you know, the data is binned in 6 minute raw files
This also is no longer true; the 6-minute binning of the data
was discontinued at the same time as the removal of blank, "heartbeat"
products were removed.
I hope that this helps!
Cheers,
Tom
> I'm not positive about this - I did not write the code, and don't have time
> right now to dig through it. In fact we are really no longer maintaining
> this code, the person who wrote it left, Unidata has been doing it for some
> years
> now. I'll cc Tom Yoksas, the last person to touch the code.
> As you know, the data is binned in 6 minute raw files. I'm pretty sure that
> if the file is empty it is not added to the LDM queue. We archive what we
> put in the ldm queue (even though we don't use it) and there are several
> holes in Jan 2013. Several days with apparently no lightning in the
> continental
> US. I don't think this is uncommon in the winter. We store by hour and Julian
> day. Here are the days we have data for in Jan 2013.
>
> striker2:/nldnarchive/ingest/nldn2013> ls -l
> total 152
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 4 05:05 001
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 6 05:05 003
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 7 05:05 004
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 8 05:05 005
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 9 05:05 006
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 10 05:05 007
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 11 05:05 008
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 12 05:05 009
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 13 05:05 010
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 14 05:05 011
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 15 05:05 012
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 16 05:05 013
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 17 05:05 014
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 18 05:05 015
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 19 05:05 016
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 20 05:05 017
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 20 05:05 018
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 22 05:05 019
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 23 05:05 020
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 24 05:05 021
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 25 05:05 022
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 26 05:05 023
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 27 05:05 024
> drwxr-xr-x 2 ldmnldn unidata 512 Jan 28 05:05 025
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 29 05:05 026
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 30 05:05 027
> drwxr-xr-x 2 ldmnldn unidata 1024 Jan 31 05:05 028
> drwxr-xr-x 2 ldmnldn unidata 1024 Feb 1 05:05 029
> drwxr-xr-x 2 ldmnldn unidata 1024 Feb 2 05:05 030
> drwxr-xr-x 2 ldmnldn unidata 512 Feb 3 05:05 031
>
> And here are the hours we have data for on Jan 1 2013:
> striker2:/nldnarchive/ingest/nldn2013> ls -l 001
> total 132
> -rw-r--r-- 1 ldmnldn unidata 224 Jan 1 00:51 201300100.ingest
> -rw-r--r-- 1 ldmnldn unidata 784 Jan 1 02:31 201300102.ingest
> -rw-r--r-- 1 ldmnldn unidata 980 Jan 1 03:55 201300103.ingest
> -rw-r--r-- 1 ldmnldn unidata 2212 Jan 1 05:00 201300104.ingest
> -rw-r--r-- 1 ldmnldn unidata 2408 Jan 1 05:48 201300105.ingest
> -rw-r--r-- 1 ldmnldn unidata 2632 Jan 1 07:00 201300106.ingest
> -rw-r--r-- 1 ldmnldn unidata 3024 Jan 1 08:00 201300107.ingest
> -rw-r--r-- 1 ldmnldn unidata 5040 Jan 1 09:00 201300108.ingest
> -rw-r--r-- 1 ldmnldn unidata 4788 Jan 1 10:01 201300109.ingest
> -rw-r--r-- 1 ldmnldn unidata 8904 Jan 1 11:00 201300110.ingest
> -rw-r--r-- 1 ldmnldn unidata 7672 Jan 1 11:59 201300111.ingest
> -rw-r--r-- 1 ldmnldn unidata 5488 Jan 1 12:59 201300112.ingest
> -rw-r--r-- 1 ldmnldn unidata 4368 Jan 1 13:55 201300113.ingest
> -rw-r--r-- 1 ldmnldn unidata 2912 Jan 1 15:01 201300114.ingest
> -rw-r--r-- 1 ldmnldn unidata 4480 Jan 1 15:59 201300115.ingest
> -rw-r--r-- 1 ldmnldn unidata 2576 Jan 1 17:00 201300116.ingest
> -rw-r--r-- 1 ldmnldn unidata 868 Jan 1 17:46 201300117.ingest
> -rw-r--r-- 1 ldmnldn unidata 196 Jan 1 23:04 201300123.ingest
>
>
> Hope this helps,
> David
>
>
> On Mar 19, 2013, at 6:58 PM, Rick Steed wrote:
>
> > Hi David-
> >
> > I hope you can answer this question. We're using NLDN data and there are
> > quite a few holes in our archive. I'm not sure if this is because the data
> > files aren't created when there's zero lightning in the data domain, or if
> > we just have an unreliable data feed. I realize this is somewhat ambiguous
> > but would like to know if the files are produced even when there's no data.
> > Thanks.
> >
> > -Rick Steed
> > University of Washington
> >
> > On 03/19/13 15:42, Harry Edmon wrote:
> >> Don't know the answer. Our contact for that data is Dr. David Knight at
> >> SUNY Albany - address@hidden. He is a UW grad.
> >>
> >> On 03/19/13 15:30, Rick Steed wrote:
> >>> Harry-
> >>>
> >>> That file (just an example) DOES exist. But there are numerous dates
> >>> which do not throughout the winter. For example, Dec 2012 only has 640
> >>> and Jan 2013 only 513 out of a possible 744 files each.
> >>>
> >>> -Rick
> >>>
> >>> On 03/19/13 15:27, Harry Edmon wrote:
> >>>> I'm confused (easy to do) - that file exists:
> >>>>
> >>>> 112 moist% ls -l /home/disk/archive/nldn/raw/201207/2012073018.nldn
> >>>> -rw-rw-r-- 1 ldm ldm 384440 Jul 30 2012
> >>>> /home/disk/archive/nldn/raw/201207/2012073018.nldn
> >>>>
> >>>>
> >>>> On 03/19/13 15:23, Rick Steed wrote:
> >>>>> Hey Mark-
> >>>>>
> >>>>> There is a fair number of times (esp winter months) when the nldn data
> >>>>> is missing from the archive. Is it possible that no file is created
> >>>>> when there is ZERO lightning anywhere in the US? I can see how that
> >>>>> might happen but I need to confirm. I'm looking at the files such as:
> >>>>>
> >>>>> /home/disk/archive/nldn/raw/201207/2012073018.nldn
> >>>>>
> >>>>> Thanks.
> >>>>>
> >>>>> -Rick
> >>>>>
> >>>>
> >>>>
> >>
> >>
> >
>
> David Knight
> Dept. of Atmospheric and Environmental Sciences Tel: (518)-442-4204
> University at Albany, ES-228 Fax: (518)-442-4494
> Albany, NY 12222 Email: address@hidden
>
>
>
Cheers,
Tom
--
****************************************************************************
Unidata User Support UCAR Unidata Program
(303) 497-8642 P.O. Box 3000
address@hidden Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage http://www.unidata.ucar.edu
****************************************************************************
Ticket Details
===================
Ticket ID: QJW-779379
Department: Support Datastream
Priority: Normal
Status: Closed