20040308: mcidas: wxtlist 0 GMT raob's (cont.)

>From: "Fingerhut, William A" <address@hidden>
>Organization: Lyndon State
>Keywords: 200403041858.i24Iw5rV013299 McIDAS WXTLIST

Hi Bill,

>I just tried the wxtlist.k command, using omega (our adde server), 
>and I see 12 and 0 Z from today, some 12 Z yesterday, and some 12 Z
>the day before.

Hmm... very interesting indeed!

>wxtlist.k WMO=USUS UKUS WSTN=KWBC MATCH='72518' FORM=HEAD NUM=66
>
>SOU bytes    day    time  WMO     WSTN APRO ASTN Actual Header
>--- -----  ------- ------ ------- ---- ---- ---- ------------------
>DDS  2000  2004068 124044 UKUS 1  KWBC           UKUS01 KWBC 081200
>DDS  4960  2004068 124011 UKUS90  KWBC           UKUS90 KWBC 081200
>DDS  4640  2004068 124007 USUS90  KWBC           USUS90 KWBC 081200
>DDS  4800  2004068 124002 UKUS50  KWBC           UKUS50 KWBC 081200
>DDS  4880  2004068 124002 USUS50  KWBC           USUS50 KWBC 081200
>DDS  2480  2004068 123955 USUS 1  KWBC           USUS01 KWBC 081200
>DDS   800  2004068 121407 UKUS 1  KWBC           UKUS01 KWBC 081200
>DDS  9280  2004068 121353 UKUS90  KWBC           UKUS90 KWBC 081200
>DDS  8480  2004068 121353 USUS90  KWBC           USUS90 KWBC 081200
>DDS 10640  2004068 121351 UKUS50  KWBC           UKUS50 KWBC 081200
>DDS  9920  2004068 121351 USUS50  KWBC           USUS50 KWBC 081200
>DDS  1120  2004068 121343 USUS 1  KWBC           USUS01 KWBC 081200
>DDS   560  2004068 010807 UKUS 1  KWBC           UKUS01 KWBC 080000
>DDS   480  2004068 010800 USUS 1  KWBC           USUS01 KWBC 080000
>DDS  2400  2004068 005308 UKUS90  KWBC           UKUS90 KWBC 080000
>DDS   880  2004068 005308 USUS90  KWBC           USUS90 KWBC 080000
>DDS  2400  2004068 005307 UKUS50  KWBC           UKUS50 KWBC 080000
>DDS   880  2004068 005307 USUS50  KWBC           USUS50 KWBC 080000
>DDS  1360  2004068 001428 UKUS 1  KWBC           UKUS01 KWBC 080000
>DDS  8560  2004068 001359 UKUS90  KWBC           UKUS90 KWBC 080000
>DDS  6720  2004068 001357 USUS90  KWBC           USUS90 KWBC 080000
>DDS  9600  2004068 001351 UKUS50  KWBC           UKUS50 KWBC 080000
>DDS  7840  2004068 001351 USUS50  KWBC           USUS50 KWBC 080000
>DDS  1360  2004068 001347 USUS 1  KWBC           USUS01 KWBC 080000
>DDS  1280  2004067 121419 UKUS 1  KWBC           UKUS01 KWBC 071200
>DDS  9280  2004067 121356 UKUS90  KWBC           UKUS90 KWBC 071200
>DDS  8080  2004067 121354 USUS90  KWBC           USUS90 KWBC 071200
>DDS 10880  2004067 121351 UKUS50  KWBC           UKUS50 KWBC 071200
>DDS  9440  2004067 121351 USUS50  KWBC           USUS50 KWBC 071200
>DDS  1120  2004067 121333 USUS 1  KWBC           USUS01 KWBC 071200
>DDS  1680  2004066 123927 UKUS 1  KWBC           UKUS01 KWBC 061200
>DDS  2000  2004066 123914 USUS 1  KWBC           USUS01 KWBC 061200
>DDS  4880  2004066 122401 UKUS90  KWBC           UKUS90 KWBC 061200
>DDS  3680  2004066 122401 USUS90  KWBC           USUS90 KWBC 061200
>DDS  4800  2004066 122359 UKUS50  KWBC           UKUS50 KWBC 061200
>DDS  3600  2004066 122357 USUS50  KWBC           USUS50 KWBC 061200
>DDS   800  2004066 121500 UKUS 1  KWBC           UKUS01 KWBC 061200
>DDS  9120  2004066 121429 UKUS90  KWBC           UKUS90 KWBC 061200
>DDS  5440  2004066 121429 USUS90  KWBC           USUS90 KWBC 061200
>DDS 10720  2004066 121425 UKUS50  KWBC           UKUS50 KWBC 061200
>DDS  5280  2004066 121425 USUS50  KWBC           USUS50 KWBC 061200
>DDS   800  2004066 121421 USUS 1  KWBC           USUS01 KWBC 061200
>wxtlist.k: done
>
>Today, some 0 Z data is showing up !

It is interesting that 0Z obs started showing up all of a sudden...

>I pointed to adde.ucar.edu, and repeated. 
>
>Today is similar; yesterday I see 7 12 Z headers and several 0 Z
>headers.
>So, adde.ucar.edu is showing 0 Z data for 72518 from KWBC under USUS 
>and/or UKUS for yesterday.
>
>I repeated this test on another pc - same results.
>Mark repeated this test on the server - same results.

So, the problem is not with the application, WXTLIST, but, rather,
with the server.

>So, adde.ucar.edu has text bulletins that are not on omega. ????
>We thought our NOAAPORT system passed all text bulletins to the LDM.
>Any thoughts about where this system might fail ?

Since I am not intimately aware of how your NOAAPORT system is setup to
pass things to the LDM, I don't feel comfortable venturing guesses on
what be messed wrong.

>RAOBRPT finds and displays 00 Z data on omega. It must use different WMO
>headers than USUS and UKUS, like UMUS (?), from KWBC. Do you happen to 
>know which headers RAOBRPT uses ?

RAOBRPT runs the more general purpose routine OBSRPT which actually
does the server calls.  You can see the run sequence for OBSRPT by
tacking the DEV=CCC keyword onto the RAOBRPT command invocation:

RAOBRPT 72518 40 DEV=CCC

Looking at the output of this invocation I can see that OBSRPT is
not specifying the WMO header for its search:

% raobrpt.k 72518 40 DEV=CCC
raobrpt.k* Data Set Name = 'RTWXTEXT/UPPERAIR'
raobrpt.k* Number of stations=1 Regions=0 Countries=0 1st time=40
raobrpt.k* ID[ 0]  = '72518'
raobrpt.k* OBSRPT RTWXTEXT/UPPERAIR TRACE=0 OLDEST=2004067 22 FORM=5 1 60 
NPERIOD=9999 SUPPRESS=DONE  ID= 72518
OBSRPT* Data Set Name = 'RTWXTEXT/UPPERAIR'
OBSRPT* Actual Data Set Name = 'RTWXTEXT UPPERAIR'
OBSRPT* select_string[ 0] = 'IDREQ=LIST'
OBSRPT* select_string[ 1] = 'NPERIOD= 9999'
OBSRPT* select_string[ 2] = 'TRACE= 0'
OBSRPT* select_string[ 3] = 'OLDEST=2004067 22:00:00'
OBSRPT* select_string[ 4] = 'ID= 72518'
OBSRPT* RTWXTEXT UPPERAIR IDREQ=LIST NPERIOD= 9999 TRACE= 0 OLDEST=2004067 
22:00:00 ID= 72518
OBSRPT* M0cxreq returns 0 msg_len=96 prtmsg=1
OBSRPT* header length =96
OBSRPT* trying to allocate 480 bytes
OBSRPT* M0obtxread returns 0 with 480 bytes
OBSRPT* Ver =   1  ObType =   3  ActFlg =   1 IDType =   0
OBSRPT* Starting at 2004069 000000Z 120000Z
OBSRPT* Ending   at 2004069 000000Z 120000Z
 ...

So, I would refine my WXTLIST invocatoin to either generalize the
WMO headers specified, or not use them at all.

>We are out of ideas. Do you have any?

I apologize for not being able to dive into this more thoroughly, but
I am doing email support as I get a few minutes here and there while
I am down here in Costa Rica helping folks get a Unidata NOAAPORT
ingest system up and running with the LDM.

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publically available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.

>From address@hidden  Tue Mar  9 07:08:24 2004

Tom,

> -----Original Message-----
> From: Unidata Support [mailto:address@hidden] 
> Sent: Tuesday, March 09, 2004 1:17 PM
> To: Fingerhut, William A
> Cc: address@hidden; Tucker, Mark H
> Subject: 20040308: mcidas: wxtlist 0 GMT raob's (cont.)
>
> >Today, some 0 Z data is showing up !
> 
> It is interesting that 0Z obs started showing up all of a sudden...
>

I'll keep an eye on this for clues.

I just ran the original WXTLIST and see 12 Z and 0 Z reports for today
day 069, only 12 Z reports for yesterday 068. Yesterday at this time, 
when I did this the 0 Z reports for 068 were listed !? Might they be
scoured ? We'll have to check this out.
 
> 
> So, the problem is not with the application, WXTLIST, but, 
> rather, with the server.
> 

Apparently, but I am still bothered that RAOBRPT works okay and
WXTLIST does not. The 0 Z data must be present on omega for RAOBRPT to
work; it might help to know where the data are located (wmo headers).
I tried DEV=CCC on a RAOBRPT; noted the call to OBSRPT; looked at the
source code, but got lost. 

> >So, adde.ucar.edu has text bulletins that are not on omega. ???? We 
> >thought our NOAAPORT system passed all text bulletins to the 
> LDM. Any 
> >thoughts about where this system might fail ?
> 
> Since I am not intimately aware of how your NOAAPORT system 
> is setup to pass things to the LDM, I don't feel comfortable 
> venturing guesses on what be messed wrong.
> 

This problem is outside the usual domain of Unidata support, but I
thought you might have an idea or two. We will double check what we can.

> >RAOBRPT finds and displays 00 Z data on omega. It must use different 
> >WMO headers than USUS and UKUS, like UMUS (?), from KWBC. Do 
> you happen 
> >to know which headers RAOBRPT uses ?
> 
> RAOBRPT runs the more general purpose routine OBSRPT which 
> actually does the server calls.  You can see the run sequence 
> for OBSRPT by tacking the DEV=CCC keyword onto the RAOBRPT 
> command invocation:
> 
> RAOBRPT 72518 40 DEV=CCC
> 
> Looking at the output of this invocation I can see that 
> OBSRPT is not specifying the WMO header for its search:
> 

Already tried this.

> So, I would refine my WXTLIST invocatoin to either generalize 
> the WMO headers specified, or not use them at all.
> 

Already tried this too. Using the keyword WMO=U?US with WXTLIST
produced an error message.
Invalid WMO= format specified

When I left out the WMO= and WSTN= keywords, and tried APRO=MAN SGL 
I found data in UMUS from KALY. Searching on each of these did not
list any 0 Z reports. So, I still don't know where the 0 Z reports
are located (headers).

> 
> I apologize for not being able to dive into this more 
> thoroughly, but I am doing email support as I get a few 
> minutes here and there while I am down here in Costa Rica 
> helping folks get a Unidata NOAAPORT ingest system up and 
> running with the LDM.
> 

Yes, the Internet does some cool things. Thanks for fittings us in.

The good news is I can find the data I want, I just have to use
plan B. 

Stay well,

Bill

>From address@hidden  Thu Mar 11 06:41:41 2004

Tom,

I found the 'missing' 0 Z upper air data on our server. :-)

Part of it becomes garbled after roughly one day ? :-(
I would guess that wxtlist was searching on the garbled field in the
original search.
Searching on APRO & ASTN works since these info are still available.

So, now we have to find out why the data is garbled (written over?)  ?

wxtlist.k APRO=MAN ASTN=ALY FORM=HEAD NUM=44

SOU bytes    day    time  WMO     WSTN APRO ASTN Actual Header
--- -----  ------- ------ ------- ---- ---- ---- ------------------
DDS   640  2004071 122349 USUS41  KALY MAN  ALY  USUS41 KALY 111223
DDS   560  2004071 001427 USUS41  KALY MAN  ALY  USUS41 KALY 110014
DDS   640  2004070 122202 USUS41  KALY MAN  ALY  USUS41 KALY 101221
DDS   560  2004070 004230 USUS41  KALY MAN  ALY  ka.=F7=E0 =DCp]
DDS   640  2004069 122646 USUS41  KALY MAN  ALY  USUS41 KALY 091226
PuTTYDDS   560  2004069 001116 USUS41  KALY MAN  ALY  =C0"=B083@
                                                            "
wxtlist.k: done
fingerhutb@kangaroo:~/mcidas/data> wxtlist.k APRO=MAN ASTN=ALY
NUM=44
USUS41 KALY 111223                                              2004071
1223
MANALY
72518 TTAA  61121 72518 99011 05129 34003 00181 01748 00505
92801 03548 03011 85469 04362 04508 70997 07985 04508 50555
19779 18010 40716 31372 22510 30914 47768 21508 25032 55367
34023 20174 54972 33530 15359 52977 31042 10614 60375 27024
88999 77999=
NNNN
USUS41 KALY 110014                                              2004071
0014
MANALY
72518 TTAA  61001 72518 99015 05862 36003 00214 04662 01006
92842 01357 03511 85509 07323 05510 70021 08566 18003 50554
27163 23522 40712 35163 20023 30907 48561 04508 25026 53761
03514 20168 54561 34009 15352 55762 30020 10608 56166 29527
88237 55361 03017 77999=
USUS41 KALY 101221                                              2004070
1222
MANALY
72518 TTAA  60121 72518 99017 00607 00000 00227 00310 19001
92846 05104 17003 85506 09505 21507 70981 18156 27502 50544
31181 21505 40699 41379 21018 30888 54975 25018 25006 52977
27007 20150 52577 25513 15335 54177 27525 10595 56376 27028
88999 77999=
NNNN
ka.=F7=E0 =DCp]=C0=B9
H
H
w=DCp.aA@?i:=EF}=EA
w=DCp.aA@?i:=EF}=EA
*?=B3=DD.@p]=C1w=E8
*?=B3=DD.@p]=C1w=E8
USUS41 KALY 091226                                              2004069
1226
MANALY
72518 TTAA  59121 72518 99004 00709 18003 00124 01143 20003
92742 05513 30507 85401 09708 25507 70879 17750 20504 50532
34363 22513 40685 46358 19517 30872 51962 24022 25990 50777
26017 20136 49178 27525 15325 51378 27034 10589 52577 26523
88318 53760 22514 77999=
NNNN
PuTTY=C0"=B083@
           "=D8
x%=CA=D7.>xA@=A8=EB
           O=E4-=F2^uq=F5=E7P
x%=CA=D7.>xA@=A8=EB
           O=E4-=F2^uq=F5=E7P
=B8|=C0=B2Y5y=E2=E3=E7tn=A8`=C0=FDh=DF@=F2W[^x=B8=F9=E1N=A96?P=D8
=B8|=C0=B2Y5y=E2=E3=E7tn=A8`=C0=FDh=DF@=F2W[^x=B8=F9=E1N=A96?P=D8
=AAsk=F5
=AAsk=F5
wxtlist.k: done

Bill