[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Datastream #MQU-657040]: L2 products sent in NIMAGE



> I do have those files, thanks

OK.

re:
> We are only using the first exec line.
> I thought that the TI was only for L1B data.

The second EXEC line was the one that runs the Python script to stitch
together the ABI L2 tiles that are sent in NOAAPort into full scenes.
It doesn't have anything to do with the other L2 products.

re: 
> We are not getting CMIP files, does this mean that CMIP products are
> coming in with a TI WMO ID?

To be clear, the ABI L2 products that are sent in NOAAPort are CMIP
tiles that need to be stitched together to make full scenes (images).

The action in the pqact.conf_npgoesr pattern-action file processes
products that begin with 'TI', and these are the ABI L2 tiles.  Here
is the action in pqact.conf_goesr:

#
# -------------------------------------
# - NOAAPORT GOES-16 netCDF4 Data -
# -------------------------------------
#
NOTHER  (^TI..).. KNES ([0-9][0-9][0-9][0-9][0-9][0-9]) ...
        PIPE    -metadata
        util/goes-restitch.py -v -d /data/ldm/pub/native/satellite/GOES -t 360 
-l logs/goes-restitch/\1.log \1 \2

So, to answer your question directly, the ABI L2 products have Product IDs
that look like 'TI.. KNES DDHHMM'.  The rest of the Product IDs, the piece
that uniquely defines the various tiles is the next field after DDHHMM.
Here is a snippit of a 'notifyme' listing that shows the full Product IDs
for the ABI L2 products:

~: notifyme -vl- -f NOTHER 
20190813T171905.454278Z notifyme[10442]             notifyme.c:main:363         
        NOTE  Starting Up: localhost: 20190813171905.454076 TS_ENDT {{NOTHER, 
".*"}}
20190813T171905.454325Z notifyme[10442]             ldm5_clnt.c:forn5:460       
        NOTE  LDM-5 desired product-class: 20190813171905.454076 TS_ENDT 
{{NOTHER, ".*"}}
20190813T171905.454727Z notifyme[10442]             error.c:err_log:236         
        INFO  Resolving localhost to 127.0.0.1 took 0.000344 seconds
20190813T171905.456730Z notifyme[10442]             ldm5_clnt.c:forn_signon:294 
        NOTE  NOTIFYME(localhost): OK
20190813T171906.457330Z notifyme[10442]             
notifyme.c:notifymeprog_5:212       INFO      322034 20190813171905.663916 
NOTHER 605307  TIRW15 KNES 131716 PAK
20190813T171906.457576Z notifyme[10442]             
notifyme.c:notifymeprog_5:212       INFO      315690 20190813171905.715041 
NOTHER 605308  TIRW13 KNES 131716 PAK
20190813T171906.497578Z notifyme[10442]             
notifyme.c:notifymeprog_5:212       INFO      281106 20190813171905.940081 
NOTHER 605309  TIRW11 KNES 131716 PAL
20190813T171906.497777Z notifyme[10442]             
notifyme.c:notifymeprog_5:212       INFO      292287 20190813171906.164924 
NOTHER 605310  TIRW16 KNES 131716 PAK

So, the DDHHMM referenced above means:

DD - day of the month
HH - hour of the product
MM - minute of the product

Important comment:

The Product IDs for the non-ABI L2 product tiles do not contain enough
information to differentiate between the same product for the same time
but for different coverages.  Here is a 'notifyme' snippit that demonstrates
what I am highlighting:

notifyme -vl- -p ^IXT -f NOTHER -o 300
20190813T172149.796276Z notifyme[18461]             notifyme.c:main:363         
        NOTE  Starting Up: localhost: 20190813171649.796075 TS_ENDT {{NOTHER, 
"^IXT"}}
20190813T172149.796323Z notifyme[18461]             ldm5_clnt.c:forn5:460       
        NOTE  LDM-5 desired product-class: 20190813171649.796075 TS_ENDT 
{{NOTHER, "^IXT"}}
20190813T172149.796690Z notifyme[18461]             error.c:err_log:236         
        INFO  Resolving localhost to 127.0.0.1 took 0.000311 seconds
20190813T172149.798593Z notifyme[18461]             ldm5_clnt.c:forn_signon:294 
        NOTE  NOTIFYME(localhost): OK
20190813T172150.812267Z notifyme[18461]             
notifyme.c:notifymeprog_5:212       INFO      502543 20190813171718.910464 
NOTHER 3759  IXTW01 KNES 131717
20190813T172150.825838Z notifyme[18461]             
notifyme.c:notifymeprog_5:212       INFO      306811 20190813171737.626551 
NOTHER 3764  IXTW01 KNES 131717
20190813T172150.881874Z notifyme[18461]             
notifyme.c:notifymeprog_5:212       INFO      502475 20190813171818.805783 
NOTHER 3776  IXTW01 KNES 131718
20190813T172150.886814Z notifyme[18461]             
notifyme.c:notifymeprog_5:212       INFO      307094 20190813171836.593412 
NOTHER 3780  IXTW01 KNES 131718

Notice that the Product IDs for the first two products listed are identical,
but the products are different sizes.  The different sizes represent two
different coverages, but there is nothing in the actual Product IDs that
differentiate them.  The BASH script that I provided, L2File.sh, gets
around this limitation (which originates at the NOAAPort uplink; it is
NOT a deficiency with the LDM) by writing the product into a disk file
that is named using the Product ID pieces AND the process number of the
script, so the file that is written _is_ named uniquely.

The reason that I included the lengthy explanation in the previous
paragraph is to warn you that if you are not doing something similar,
you _will_ miss/incorrectly process non-ABI L2 products.  I have no idea
if this is what is happening in your setup, but I was prompted to
explain based on your comment: "We are missing some of the L2 products
I see on your website.".

re:
> I'll add that to see if it helps.

Very good.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: MQU-657040
Department: Support Datastream
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.