20001129: test NIDS floater products

>From:  Robb Kambic <address@hidden>
>Organization:  UCAR/Unidata
>Keywords:  200011291843.eATIhJp06962 IDD NIDZ floater

Robb,

Last week, you sent the following:

>I saved some of the floaters products on zero, ~ldm/data/radar.  These
>products have 2 extras nulls at the end so they have different md5
>checksums.
>ie.
>...
>...
>0102220 327 020   W 347 020   W 027  \0 310   l 233 372  \r  \r  \n 003
>0102240  \0  \0   
>
>Hopefully your decoder can handle the extra chars.  Also are the file
>names in a usable format?

First, the file names are fine.

Second, the products you saved do not appear to be the zlib-compressed
NIDS products.

In support of the files not looking like the zlib-compressed images I
note that the file sizes are exceedingly large:

% ls -alt *.N0RARX
-rw-rw-r--   1 ldm      ustaff     24366 Nov 29 11:31 200011291829.N0RARX
-rw-rw-r--   1 ldm      ustaff     28621 Nov 29 11:31 200011291730.N0RARX
-rw-rw-r--   1 ldm      ustaff     28393 Nov 29 11:31 200011291740.N0RARX
-rw-rw-r--   1 ldm      ustaff     27250 Nov 29 11:31 200011291750.N0RARX
-rw-rw-r--   1 ldm      ustaff     26702 Nov 29 11:31 200011291800.N0RARX
-rw-rw-r--   1 ldm      ustaff     26188 Nov 29 11:31 200011291810.N0RARX
-rw-rw-r--   1 ldm      ustaff     25232 Nov 29 11:31 200011291819.N0RARX

Note that these files are all larger than about 25 KB.  I suspect that
these files are probably the encrypted ones from the datastream, not
the zlib-compressed ones.

Compare these sizes with a slew of zlib-compressed NIDS products:

% ls -alt *_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14359 Dec  7 05:03 200012071201_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14419 Dec  7 04:53 200012071151_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14557 Dec  7 04:43 200012071141_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14903 Dec  7 04:33 200012071132_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     15145 Dec  7 04:25 200012071122_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14645 Dec  7 04:14 200012071112_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14598 Dec  7 04:04 200012071102_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14938 Dec  7 03:54 200012071052_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     15190 Dec  7 03:44 200012071042_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     15245 Dec  7 03:34 200012071033_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     15046 Dec  7 03:24 200012071023_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14623 Dec  7 03:15 200012071013_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14605 Dec  7 03:05 200012071003_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14349 Dec  7 02:55 200012070953_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14354 Dec  7 02:45 200012070943_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14502 Dec  7 02:35 200012070934_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     13793 Dec  7 02:25 200012070924_N0RARX.nidz
-rw-rw-r--   1 ldm      ustaff     14060 Dec  7 02:16 200012070914_N0RARX.nidz

The current file sizes are typically less than 15 KB AND the station in
question (ARX, La Crosse, WI) is just now seeing some weather.  Back on
Nov 29, the radar was continuously operating in clear air mode, so one
could expect the file sizes to be smaller since there was less data to
compress.

To test my server against altered zlib-compressed NIDS files, I did the
following:

o copy a set of the *.nidz files to a separate directory
o appended two-byte sequence to the end of each file (0x010a instead of 0x0000)
o ran my ADDE image list and get servers against those files

In this test, everything ran fine, so I feel confident that my servers
will work on altered zlib-compressed NIDS products.

>The pqact entry:
>
># save FLOATER
>NMC3    ^FLOATER SDUS5. .... ([0-3][0-9])([0-2][0-9])([0-5][0-9])
>/p(......)
>        FILE    data/radar/(\1:yyyy)(\1:mm)\1\2\3.\4
>#

Can you check to make sure that the files being redirected here are not
the encrypted ones instead of the zlib-compressed ones?  It would be
_very_ useful to me to clear up the discrepency with the test files you
saved soon.  I am just about to announce the availability of my
distribution to the McIDAS community, and I would rather not have to
release patches to fix a potential problem after the official release.

Tom
--
+-----------------------------------------------------------------------------+
* Tom Yoksas                                             UCAR Unidata Program *
* (303) 497-8642 (last resort)                                  P.O. Box 3000 *
* address@hidden                                   Boulder, CO 80307 *
* Unidata WWW Service                             http://www.unidata.ucar.edu/*
+-----------------------------------------------------------------------------+