Mitch,
Here's my status, I can decode the MOS Guidance files up to the point of
the actual data in section 4, the data section. I can get the length of
section 4, skip one octet, then my decoding process gives me wrong values.
I attached a sample file:
Header : JSML33 KWNO 150000
Reference Time : 2008-08-15T00:00:00Z
Nominal Time : 2008-08-15T00:00:00Z
Center Id : 7
Sub Center Id : 0
Center Name : US National Weather Service (NCEP)
Category : 0 Surface data - land
Sub Category : 0 Unknown
Master table : NCEPtable-ABD.diff
isCompressed : true
here's the list of descriptors I decode:
First descriptors :[0-1-63, 0-5-2, 0-6-2, 0-4-1, 0-4-2, 0-4-3, 0-4-4,
0-8-21, 0-1-32, 0-2-200, 0-4-24, 0-8-13, 0-12-21, 0-8-13, 0-12-22, 0-8-13,
0-12-104, 0-12-106, 0-11-1, 0-11-200, 0-11-3, 0-11-4, 0-60-1, 0-60-2,
0-60-3, 0-60-58, 0-60-59, 0-60-60, 0-60-61, 0-60-62, 0-60-63, 0-60-150,
0-60-151, 0-60-152, 0-60-20, 0-60-21, 0-60-22, 0-60-91, 0-60-4, 0-60-7,
0-60-10, 0-60-13, 0-4-31, 0-60-90, 0-60-5, 0-60-8, 0-60-11, 0-60-14,
0-60-16, 0-4-31, 0-60-90, 0-60-6, 0-60-9, 0-60-12, 0-60-15, 0-60-17,
0-4-31, 0-60-90, 0-60-144, 0-60-31, 0-60-33, 0-60-35, 0-60-145, 0-4-31,
0-60-93, 0-60-100, 0-60-108, 0-60-109, 0-60-103, 0-60-104, 0-20-11,
0-60-42, 0-60-43, 0-60-44, 0-60-80, 0-60-81, 0-60-46, 0-60-47, 0-60-48,
0-60-94, 0-60-49, 0-60-112, 0-60-118, 0-60-113, 0-60-114, 0-60-115,
0-60-190, 0-60-57, 0-60-55, 0-60-116, 0-60-117, 0-60-54, 0-60-96, 0-60-23,
0-60-24, 0-60-25, 0-60-92, 0-60-148, 0-60-149]
--------------------------------------------------------------------
The first descriptor is 0-1-63, ie
0-01-063 ICAO location identifier CCITT IA5
0
0 64 from tableB
That states this is char data, 8 chars long.
The length of section 4 is 818664 octets seems correct since the file is
822602 bytes.
so i read the first 8 chars and I get wrong values. I get non char data
and for the 2nd descriptor 0-1-5 lat, i get:
0-5-2 Latitude size =12285 varCount =1
-1.7044362E7 3976000.0 type values
so there is something wrong with the way I'm interpreting the compressed
char data or ... So my question is how is the char data entered in
compressed format. My decoder can interpret almost all other bufr data so
i know that there must be something small that I"m not accounting for?
Thanks for your help,
RObb...
On Fri, 29 Aug 2008, Mitchell Weiss wrote:
Robb,
Just to repeat what Becky has already stated, our lab is responsible for
encoding BUFR messages,
but the decoding process is something we do not do. However, this does not
mean that we don't
check the decoded output of our BUFR messages, but we need help from AWIPS
decoder specialist
in order to this. My suggestion for dumping a BUFR file for a specific
file is to do the following.
1. Have an AWIPS decoder person decode a BUFR file into a NETCDF formated
file.
2. You can then use a utility called NCDUMP to dump data values from the
NETCDF
files. NCDUMP has a number of very interesting and powerful options for
dumping
a NETCDF file. They can be found at
http://www.unidata.ucar.edu/software/netcdf/docs/ncdump-man-1.html
I'm not sure if you have access to the NCDUMP utility but if you do, I
would
suggest going this route.
I hope this helps
Mitch
Rebecca Cosgrove wrote:
Robb,
Before I send you to an AWIPS decoder person, let me pass you on to Mitch
Weiss. He actually worked on the code we use to code up our MOS BUFR
products and he's now heavily involved in the GFS LAMP BUFR products. I
think he will be much more knowledgable about the bits and bytes of our
BUFR products than I am. I told him you were using the JSML33 KWNO
product. I hope he can help you work things out.
Becky
Robb Kambic wrote:
On Fri, 29 Aug 2008, Rebecca Cosgrove wrote:
Robb,
I'm afraid I don't even know what CCITT means. But let me see if this
helps
Hi Becky,
CCITT just means 8 bit char data. From the Bufr table B at NCEP 0-1-63
is:
0-01-063 ICAO location identifier CCITT
IA5
0 0 64 ICLI
http://www.emc.ncep.noaa.gov/mmb/data_processing/bufrtab_tableb.htm
you at all. The first thing we're putting in that file is the call
letter record. For the JSML33 product, the first few call letters are
K1H2 K2WX K8D3 K9V9 KAAA KAAO KABR KACB
We treat each call letter as an 8 digit "thing". So it's "K1H2 ".
i get that data correctly in my decoder.
GFS MOS GUIDANCE FOR K1H2 K2WX K8D3 K9V9 KAAA KAAO
KABR KACB KACQ KADC KADG KADU KAEL KAFK KAIA
KAID KAIG KAIO KAIT KAIZ KALN KALO KAMN KAMW KANE
KANJ KANW KAPN KAQP KARB KARR KARV KASX KATW KATY
KAUH KAUM KAUW KAWG KAXA KAXN KAZO KBAX KBBB KBBW
KBDE KBEH KBFF KBFW KBIE KBIS KBIV KBJI KBKX KBLV
KBMG KBMI KBNW KBRD KBRL KBTL KBUU KBWG KC09 KC75
KCAD KCAV KCBF KCBG KCCY KCDD KCDJ KCDR KCFV KCGI
KCGX KCID KCIN KCIU KCKC KCKN KCLI KCMI KCMX KCMY
KCNC KCNK KCNU KCOQ KCOU KCPS KCQM KCSQ KCUT KCVG
KCVX KCWA KCWI KD07 KDBQ KDDC KDEC KDEH KDET
KDIK KDKB KDLH KDLL KDMO KDNS KDNV KDPA KDSM KDTL
KDTW KDUH KDVL KDVN KDXX KDYT KEAR KEAU KEBS KEFT
KEGV KEHA KEHR KELO KEMP KENL KENW KEOK KERY KESC
KEST KETB KETH KEVM KEVV KEWK KEYE KFAM KFAR KFBL
KFCM KFEP KFET KFFL KFFM KFFT KFGN KFKA KFLD KFNB
KFNT KFOA KFOD KFOE KFOZ KFPK KFRM KFSD KFSE KFSW
KFTK KFWA KFWC KGBD KGBG KGCK KGEZ KGFK KGHW KGLD
KGLR KGNA KGOV KGPZ KGRB KGRI KGRR KGSH KGUS KGYL
KGYY KHCD KHCO KHDE KHEI KHIB KHLC KHNB KHNR
KHON KHOP KHSB KHSI KHTL KHUF KHUT KHYR KHYS KHYX
KIAB KIBM KICL KICT KIEN KIGQ KIIB KIJX KIKK KIKV
KILL KIML KIMT KIND KINL KIOW KIRK KIRS KISN KISQ
KISW KIWD KIXD KJEF KJKJ KJKL KJLN KJMR KJMS KJOT
KJVL KJXN KJYG KJYM KJYR KLAF KLAN KLBF KLBL KLDM
KLEX KLJF KLNK KLNR KLOT KLOU KLOZ KLRJ KLSE KLVN
KLWC KLWD KLWV KLXL KLXN KLXT KMBG KMBL KMBS KMCD
KMCI KMCK KMCW KMDH KMDW KMDZ KMFI KMGG KMGN KMHE
KMHK KMIB KMIC KMIE KMIW KMJQ KMKC KMKE KMKG
KMKT KMLE KMLI KMML KMNM KMOP KMOT KMOX KMPZ KMQB
KMQT KMRJ KMSN KMSP KMTC KMTO KMTW KMUT KMVE KMVN
KMWA KMWM KMXO KMZH KN60 KODX KOEB KOEO KOFF KOFK
KOGA KOJC KOKK KOLU KOLY KOLZ KOMA KONA KONL KONZ
KORB KORC KORD KOSC KOSH KOTG KOTM KOVL KOVS KOWA
KOXV KOZW KP28 KP58 KP59 KP75 KPAH KPBH KPDC KPEA
KPHN KPHP KPIA KPIR KPKD KPLN KPNM KPNT KPOF KPPF
KPPQ KPQN KPRG KPTK KPWC KPWK KRAC KRAP KRCA KRDK
KRDR KRFD KRGK KRHI KRMY KRNH KROS KROX KRPD
KRPJ KRQB KRRL KRRT KRSL KRST KRWF KRYV KRZN KSAR
KSAW KSAZ KSBM KSBN KSDA KSDF KSET KSFD KSGF KSGS
KSHL KSJX KSLB KSLH KSLN KSLO KSME KSNY KSPI KSPW
KSQI KSTC KSTE KSTJ KSTL KSTP KSUE KSUS KSUW KSUX
KSZL KTAZ KTBN KTEW KTIP KTNU KTOB KTOP KTQE KTTF
KTVC KTVF KTWM KUES KUGN KUIN KULM KUNO KUNU KVIH
KVOK KVPZ KVTI KVTN KVVV KVWU KVYS KWLD KXVG KYIP
KYKN 45001 45002 45003 45004 45006 45007 45008 DISW3
LSCM4 PILM4 ROAM4 SGNW3 STDM4
We don't actually have code to decode our BUFR. We send the products
to the AWIPS developers here in the NWS and they have a decoder.
Do you have an awips contact person so i can send a detail msg with the
metadata from the file.
Thanks for your help,
RObb...
If me giving you
the call letters doesn't clear anything up, can you write me an email
that describes the whole situation as it stands right now as if you
hadn't asked me anything. I might be able to send this on to someone
who can help you out, but they'll kind of need the questions from the
beginning.
Becky
Robb Kambic wrote:
any progress...
Thanks,
Robb...
On Wed, 27 Aug 2008, Rebecca Cosgrove wrote:
Robb,
We're having problems with our supercomputer, so I can't get on and
take a look at any of this for you today. I hope to be able to take
a look tomorrow afternoon.
Sorry for the delay, but I will get back to you.
Becky
Robb Kambic wrote:
Becky,
I also did an octal dump on the msg, it appears my decoding is
correct upto actually decoding the data. since the first descriptor
is a text one, i wondering if my reading compress String routine is
correct? I'm includeding it, basically it just reads a char at a
time, up to the width. the width of 0-1-63 is 64 bits so i'm reading
8 chars. Could you send me the code of how you read compressed text
data? Any ideas here?
Thanks,
RObb...
private void getCompressString(DescriptorTableB des) throws
IOException {
BufrData bd = bufrdatas.get(des.getKey());
for (int j = 0; j < dds.getNumberDatasets(); j++) {
StringBuffer sb = new StringBuffer();
for (int i = 0; i < ((des.getWidth() + xWidth) / 8); i++) {
sb.append((char) bits2UInt(8));
}
bd.setValue( sb.toString() );
}
}
On Tue, 26 Aug 2008, Robb Kambic wrote:
Hi,
I was wondering if you could go into more detail with this
particular Bufr msg. I've been working on this one Bufr msg for
about 2 wks and have read much documentation but the answer eludes
me. I already had the table, i was trying to verify that it was
correct. It seems the table is correct, so are the descriptors
correct? I'm have never seen a CCITT type descriptor in a
compressed file, ie 0-1-63 is there anything special about CCITT
in a compressed file? If the descriptors are correct, then i'm
assuming that i've decoded the file correctly up to the data
section. i get 818,664 for the data section length, is that
correct? What data values do you get for the first field. I know
i'm asking a lot here but ...
thanks for your time,
RObb...
Here is the metadata i get from the file, is this correct?
--------------------------------------------------------------------
BUFR Product
Header : JSML33 KWNO 250000
Reference Time : 2008-08-25T00:00:00Z
Nominal Time : 2008-08-25T00:00:00Z
Center Id : 7
Sub Center Id : 0
Center Name : US National Weather Service
(NCEP)
Category : 0 Surface data - land
Sub Category : 0 Unknown
Master table : NCEPtable-ABD.diff
isCompressed : true
--------------------------------------------------------------------
First descriptors :[0-1-63, 0-5-2, 0-6-2, 0-4-1, 0-4-2, 0-4-3,
0-4-4, 0-8-21, 0-
1-32, 0-2-200, 0-4-24, 0-8-13, 0-12-21, 0-8-13, 0-12-22, 0-8-13,
0-12-104, 0-12-
106, 0-11-1, 0-11-200, 0-11-3, 0-11-4, 0-60-1, 0-60-2, 0-60-3,
0-60-58, 0-60-59,
0-60-60, 0-60-61, 0-60-62, 0-60-63, 0-60-150, 0-60-151,
0-60-152, 0-60-20, 0-60
-21, 0-60-22, 0-60-91, 0-60-4, 0-60-7, 0-60-10, 0-60-13, 0-4-31,
0-60-90, 0-60-5
, 0-60-8, 0-60-11, 0-60-14, 0-60-16, 0-4-31, 0-60-90, 0-60-6,
0-60-9, 0-60-12, 0
-60-15, 0-60-17, 0-4-31, 0-60-90, 0-60-144, 0-60-31, 0-60-33,
0-60-35, 0-60-145,
0-4-31, 0-60-93, 0-60-100, 0-60-108, 0-60-109, 0-60-103,
0-60-104, 0-20-11, 0-6
0-42, 0-60-43, 0-60-44, 0-60-80, 0-60-81, 0-60-46, 0-60-47,
0-60-48, 0-60-94, 0-
60-49, 0-60-112, 0-60-118, 0-60-113, 0-60-114, 0-60-115,
0-60-190, 0-60-57, 0-60
-55, 0-60-116, 0-60-117, 0-60-54, 0-60-96, 0-60-23, 0-60-24,
0-60-25, 0-60-92, 0
-60-148, 0-60-149]
--------------------------------------------------------------------
for instanous for year descriptor 0-4-1 i get values:
0-4-1 Year size =12285 varCount =1
3.0784312E7 2.5368012E7 4.727478E7
I notice these files use many class 60 descriptors, is this
correct?
thanks for your help,
RObb...
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric
Research
address@hidden WWW:
http://www.unidata.ucar.edu/
===============================================================================
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric
Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric
Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================