Mitch,
Here's my status, I can decode the MOS Guidance files up to the point
of the actual data in section 4, the data section. I can get the
length of section 4, skip one octet, then my decoding process gives me
wrong values.
I attached a sample file:
Header : JSML33 KWNO 150000
Reference Time : 2008-08-15T00:00:00Z
Nominal Time : 2008-08-15T00:00:00Z
Center Id : 7
Sub Center Id : 0
Center Name : US National Weather Service (NCEP)
Category : 0 Surface data - land
Sub Category : 0 Unknown
Master table : NCEPtable-ABD.diff
isCompressed : true
here's the list of descriptors I decode:
First descriptors :[0-1-63, 0-5-2, 0-6-2, 0-4-1, 0-4-2, 0-4-3, 0-4-4,
0-8-21, 0-1-32, 0-2-200, 0-4-24, 0-8-13, 0-12-21, 0-8-13, 0-12-22,
0-8-13, 0-12-104, 0-12-106, 0-11-1, 0-11-200, 0-11-3, 0-11-4, 0-60-1,
0-60-2, 0-60-3, 0-60-58, 0-60-59, 0-60-60, 0-60-61, 0-60-62, 0-60-63,
0-60-150, 0-60-151, 0-60-152, 0-60-20, 0-60-21, 0-60-22, 0-60-91,
0-60-4, 0-60-7, 0-60-10, 0-60-13, 0-4-31, 0-60-90, 0-60-5, 0-60-8,
0-60-11, 0-60-14, 0-60-16, 0-4-31, 0-60-90, 0-60-6, 0-60-9, 0-60-12,
0-60-15, 0-60-17, 0-4-31, 0-60-90, 0-60-144, 0-60-31, 0-60-33,
0-60-35, 0-60-145, 0-4-31, 0-60-93, 0-60-100, 0-60-108, 0-60-109,
0-60-103, 0-60-104, 0-20-11, 0-60-42, 0-60-43, 0-60-44, 0-60-80,
0-60-81, 0-60-46, 0-60-47, 0-60-48, 0-60-94, 0-60-49, 0-60-112,
0-60-118, 0-60-113, 0-60-114, 0-60-115, 0-60-190, 0-60-57, 0-60-55,
0-60-116, 0-60-117, 0-60-54, 0-60-96, 0-60-23, 0-60-24, 0-60-25,
0-60-92, 0-60-148, 0-60-149]
--------------------------------------------------------------------
The first descriptor is 0-1-63, ie
0-01-063 ICAO location identifier
CCITT IA5 0
0 64 from tableB
That states this is char data, 8 chars long.
The length of section 4 is 818664 octets seems correct since the file
is 822602 bytes.
so i read the first 8 chars and I get wrong values. I get non char data
and for the 2nd descriptor 0-1-5 lat, i get:
0-5-2 Latitude size =12285 varCount =1
-1.7044362E7 3976000.0 type values
so there is something wrong with the way I'm interpreting the
compressed char data or ... So my question is how is the char data
entered in compressed format. My decoder can interpret almost all
other bufr data so i know that there must be something small that I"m
not accounting for?
Thanks for your help,
RObb...
On Fri, 29 Aug 2008, Mitchell Weiss wrote:
Robb,
Just to repeat what Becky has already stated, our lab is responsible
for encoding BUFR messages,
but the decoding process is something we do not do. However, this
does not mean that we don't
check the decoded output of our BUFR messages, but we need help from
AWIPS decoder specialist
in order to this. My suggestion for dumping a BUFR file for a
specific file is to do the following.
1. Have an AWIPS decoder person decode a BUFR file into a NETCDF
formated file.
2. You can then use a utility called NCDUMP to dump data values from
the NETCDF
files. NCDUMP has a number of very interesting and powerful options
for dumping
a NETCDF file. They can be found at
http://www.unidata.ucar.edu/software/netcdf/docs/ncdump-man-1.html
I'm not sure if you have access to the NCDUMP utility but if you do,
I would
suggest going this route.
I hope this helps
Mitch
Rebecca Cosgrove wrote:
Robb,
Before I send you to an AWIPS decoder person, let me pass you on to
Mitch Weiss. He actually worked on the code we use to code up our
MOS BUFR products and he's now heavily involved in the GFS LAMP BUFR
products. I think he will be much more knowledgable about the bits
and bytes of our BUFR products than I am. I told him you were using
the JSML33 KWNO product. I hope he can help you work things out.
Becky
Robb Kambic wrote:
On Fri, 29 Aug 2008, Rebecca Cosgrove wrote:
Robb,
I'm afraid I don't even know what CCITT means. But let me see if
this helps
Hi Becky,
CCITT just means 8 bit char data. From the Bufr table B at NCEP
0-1-63 is:
0-01-063 ICAO location identifier
CCITT IA5
0 0 64 ICLI
http://www.emc.ncep.noaa.gov/mmb/data_processing/bufrtab_tableb.htm
you at all. The first thing we're putting in that file is the
call letter record. For the JSML33 product, the first few call
letters are
K1H2 K2WX K8D3 K9V9 KAAA KAAO KABR KACB
We treat each call letter as an 8 digit "thing". So it's "K1H2 ".
i get that data correctly in my decoder.
GFS MOS GUIDANCE FOR K1H2 K2WX K8D3 K9V9 KAAA
KAAO KABR KACB KACQ KADC KADG KADU KAEL
KAFK KAIA KAID KAIG KAIO KAIT KAIZ KALN KALO
KAMN KAMW KANE KANJ KANW KAPN KAQP KARB KARR
KARV KASX KATW KATY KAUH KAUM KAUW KAWG KAXA
KAXN KAZO KBAX KBBB KBBW KBDE KBEH KBFF KBFW
KBIE KBIS KBIV KBJI KBKX KBLV KBMG KBMI KBNW
KBRD KBRL KBTL KBUU KBWG KC09 KC75 KCAD KCAV
KCBF KCBG KCCY KCDD KCDJ KCDR KCFV KCGI
KCGX KCID KCIN KCIU KCKC KCKN KCLI KCMI KCMX
KCMY KCNC KCNK KCNU KCOQ KCOU KCPS KCQM KCSQ
KCUT KCVG KCVX KCWA KCWI KD07 KDBQ KDDC
KDEC KDEH KDET KDIK KDKB KDLH KDLL KDMO KDNS
KDNV KDPA KDSM KDTL KDTW KDUH KDVL KDVN KDXX
KDYT KEAR KEAU KEBS KEFT KEGV KEHA KEHR KELO
KEMP KENL KENW KEOK KERY KESC KEST KETB KETH
KEVM KEVV KEWK KEYE KFAM KFAR KFBL KFCM
KFEP KFET KFFL KFFM KFFT KFGN KFKA KFLD KFNB
KFNT KFOA KFOD KFOE KFOZ KFPK KFRM KFSD KFSE
KFSW KFTK KFWA KFWC KGBD KGBG KGCK KGEZ KGFK
KGHW KGLD KGLR KGNA KGOV KGPZ KGRB KGRI KGRR
KGSH KGUS KGYL KGYY KHCD KHCO KHDE KHEI
KHIB KHLC KHNB KHNR KHON KHOP KHSB KHSI KHTL
KHUF KHUT KHYR KHYS KHYX KIAB KIBM KICL KICT
KIEN KIGQ KIIB KIJX KIKK KIKV KILL KIML
KIMT KIND KINL KIOW KIRK KIRS KISN KISQ KISW
KIWD KIXD KJEF KJKJ KJKL KJLN KJMR KJMS KJOT
KJVL KJXN KJYG KJYM KJYR KLAF KLAN KLBF KLBL
KLDM KLEX KLJF KLNK KLNR KLOT KLOU KLOZ KLRJ
KLSE KLVN KLWC KLWD KLWV KLXL KLXN KLXT KMBG
KMBL KMBS KMCD KMCI KMCK KMCW KMDH KMDW KMDZ
KMFI KMGG KMGN KMHE KMHK KMIB KMIC KMIE
KMIW KMJQ KMKC KMKE KMKG KMKT KMLE KMLI
KMML KMNM KMOP KMOT KMOX KMPZ KMQB KMQT KMRJ
KMSN KMSP KMTC KMTO KMTW KMUT KMVE KMVN KMWA
KMWM KMXO KMZH KN60 KODX KOEB KOEO KOFF KOFK
KOGA KOJC KOKK KOLU KOLY KOLZ KOMA KONA KONL
KONZ KORB KORC KORD KOSC KOSH KOTG KOTM KOVL
KOVS KOWA KOXV KOZW KP28 KP58 KP59 KP75 KPAH
KPBH KPDC KPEA KPHN KPHP KPIA KPIR KPKD KPLN
KPNM KPNT KPOF KPPF KPPQ KPQN KPRG KPTK KPWC
KPWK KRAC KRAP KRCA KRDK KRDR KRFD KRGK
KRHI KRMY KRNH KROS KROX KRPD KRPJ KRQB
KRRL KRRT KRSL KRST KRWF KRYV KRZN KSAR KSAW
KSAZ KSBM KSBN KSDA KSDF KSET KSFD KSGF KSGS
KSHL KSJX KSLB KSLH KSLN KSLO KSME KSNY KSPI
KSPW KSQI KSTC KSTE KSTJ KSTL KSTP KSUE KSUS
KSUW KSUX KSZL KTAZ KTBN KTEW KTIP KTNU KTOB
KTOP KTQE KTTF KTVC KTVF KTWM KUES KUGN KUIN
KULM KUNO KUNU KVIH KVOK KVPZ KVTI KVTN
KVVV KVWU KVYS KWLD KXVG KYIP KYKN 45001 45002
45003 45004 45006 45007 45008 DISW3 LSCM4 PILM4 ROAM4
SGNW3 STDM4
We don't actually have code to decode our BUFR. We send the
products to the AWIPS developers here in the NWS and they have a
decoder.
Do you have an awips contact person so i can send a detail msg with
the metadata from the file.
Thanks for your help,
RObb...
If me giving you
the call letters doesn't clear anything up, can you write me an
email that describes the whole situation as it stands right now as
if you hadn't asked me anything. I might be able to send this on
to someone who can help you out, but they'll kind of need the
questions from the beginning.
Becky
Robb Kambic wrote:
any progress...
Thanks,
Robb...
On Wed, 27 Aug 2008, Rebecca Cosgrove wrote:
Robb,
We're having problems with our supercomputer, so I can't get on
and take a look at any of this for you today. I hope to be able
to take a look tomorrow afternoon.
Sorry for the delay, but I will get back to you.
Becky
Robb Kambic wrote:
Becky,
I also did an octal dump on the msg, it appears my decoding is
correct upto actually decoding the data. since the first
descriptor is a text one, i wondering if my reading compress
String routine is correct? I'm includeding it, basically it
just reads a char at a time, up to the width. the width of
0-1-63 is 64 bits so i'm reading 8 chars. Could you send me the
code of how you read compressed text data? Any ideas here?
Thanks,
RObb...
private void getCompressString(DescriptorTableB des) throws
IOException {
BufrData bd = bufrdatas.get(des.getKey());
for (int j = 0; j < dds.getNumberDatasets(); j++) {
StringBuffer sb = new StringBuffer();
for (int i = 0; i < ((des.getWidth() + xWidth) / 8);
i++) {
sb.append((char) bits2UInt(8));
}
bd.setValue( sb.toString() );
}
}
On Tue, 26 Aug 2008, Robb Kambic wrote:
Hi,
I was wondering if you could go into more detail with this
particular Bufr msg. I've been working on this one Bufr msg
for about 2 wks and have read much documentation but the
answer eludes me. I already had the table, i was trying to
verify that it was correct. It seems the table is correct, so
are the descriptors correct? I'm have never seen a CCITT type
descriptor in a compressed file, ie 0-1-63 is there anything
special about CCITT in a compressed file? If the descriptors
are correct, then i'm assuming that i've decoded the file
correctly up to the data section. i get 818,664 for the data
section length, is that correct? What data values do you get
for the first field. I know i'm asking a lot here but ...
thanks for your time,
RObb...
Here is the metadata i get from the file, is this correct?
--------------------------------------------------------------------
BUFR Product
Header : JSML33 KWNO 250000
Reference Time : 2008-08-25T00:00:00Z
Nominal Time : 2008-08-25T00:00:00Z
Center Id : 7
Sub Center Id : 0
Center Name : US National Weather Service
(NCEP)
Category : 0 Surface data - land
Sub Category : 0 Unknown
Master table : NCEPtable-ABD.diff
isCompressed : true
--------------------------------------------------------------------
First descriptors :[0-1-63, 0-5-2, 0-6-2, 0-4-1, 0-4-2,
0-4-3, 0-4-4, 0-8-21, 0-
1-32, 0-2-200, 0-4-24, 0-8-13, 0-12-21, 0-8-13, 0-12-22,
0-8-13, 0-12-104, 0-12-
106, 0-11-1, 0-11-200, 0-11-3, 0-11-4, 0-60-1, 0-60-2,
0-60-3, 0-60-58, 0-60-59,
0-60-60, 0-60-61, 0-60-62, 0-60-63, 0-60-150, 0-60-151,
0-60-152, 0-60-20, 0-60
-21, 0-60-22, 0-60-91, 0-60-4, 0-60-7, 0-60-10, 0-60-13,
0-4-31, 0-60-90, 0-60-5
, 0-60-8, 0-60-11, 0-60-14, 0-60-16, 0-4-31, 0-60-90,
0-60-6, 0-60-9, 0-60-12, 0
-60-15, 0-60-17, 0-4-31, 0-60-90, 0-60-144, 0-60-31,
0-60-33, 0-60-35, 0-60-145,
0-4-31, 0-60-93, 0-60-100, 0-60-108, 0-60-109, 0-60-103,
0-60-104, 0-20-11, 0-6
0-42, 0-60-43, 0-60-44, 0-60-80, 0-60-81, 0-60-46, 0-60-47,
0-60-48, 0-60-94, 0-
60-49, 0-60-112, 0-60-118, 0-60-113, 0-60-114, 0-60-115,
0-60-190, 0-60-57, 0-60
-55, 0-60-116, 0-60-117, 0-60-54, 0-60-96, 0-60-23, 0-60-24,
0-60-25, 0-60-92, 0
-60-148, 0-60-149]
--------------------------------------------------------------------
for instanous for year descriptor 0-4-1 i get values:
0-4-1 Year size =12285 varCount =1
3.0784312E7 2.5368012E7 4.727478E7
I notice these files use many class 60 descriptors, is this
correct?
thanks for your help,
RObb...
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric
Research
address@hidden WWW:
http://www.unidata.ucar.edu/
===============================================================================
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric
Research
address@hidden WWW:
http://www.unidata.ucar.edu/
===============================================================================
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric
Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric
Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================