[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20010404: 20010403: 20010321: rebuilding gempak for large gribs
- Subject: 20010404: 20010403: 20010321: rebuilding gempak for large gribs
- Date: Tue, 10 Apr 2001 13:23:24 -0600
Art,
The problem is still likely the array sizes when the program is run,
calling the subroutines which use the large arrays.
I have changed decode_grib.c for:
static unsigned char *gribbul=NULL;
static int *xgrid=NULL;
if(gribbul == NULL)
{
gribbul = (unsigned char *)malloc(MAXGRIBSIZE);
xgrid = (int *)malloc(3*LLMXGD*sizeof(int));
}
The largest use of space remaining is the dcfillgrid.c routine, which should
not be called since this isn't an arakawa grid, but the float arrays can be
rearranged to allocate and free upon use of this routine as is done in
dcsubgrid.c.
Since you aren't enetring that routine, the core dump shouldn't be there.
If dbx shows "where" the program is when the core dump occurs, that would
be useful. I believe it will be at the point of a subroutine where space
is being allocated.
I looked at the table reading and it appears to be reading the last entry
but will double check this.
Steve Chiswell
Unidata User Support
>From: "Arthur A. Person" <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200104041804.f34I4iL00470
>On Tue, 3 Apr 2001, Unidata Support wrote:
>
>> Art,
>>
>> If the program core dumps without any input etc, then it is likely
>> running into stack size problems in allocating the space required.
>>
>> You might try checking any stack size limits you have.
>> You can compile with -g to see where the dcgrib2 program is dumping.
>> It may be that an array size is larger than allowed by the system.
>>
>> I have defined MAXGRIBSIZE as 1000000 here for an ECMWF data set
>> (.5 degree global grid) and that works for the product size, however
>> the data size is only 720x361, so not using very large arrays in the
>> gemlib defined LLMXGD.
>
>Okay, I increased the stack limit and that solved that problem. However,
>I now observe the following:
>
>/opt1/gempak/NAWIPS-5.6.a-big/unidata/ldmbridge/dcgrib2/dcgrib2 -v 3 -d
>dcgrib2.log -e GEMTBL=/opt1/gempak/NAWIPS-5.6.a-big/gempak/tables
> PDS bytes 1- 3 (pds.length) = 28
> PDS byte 4 (pds.version) = 2
> PDS byte 5 (pds.center) = 7
> PDS byte 6 (pds.process) = 152
> PDS byte 7 (pds.grid_id) = 240
> PDS byte 8 (pds.flag) = 192
> PDS byte 9 (pds.parameter) = 61
> PDS byte 10 (pds.vcoord) = 1
> PDS bytes 11 (pds.level_1) = 0
> PDS bytes 12 (pds.level_2) = 0
> PDS bytes 11-12 (pds.level) = 0
> PDS byte 13 (pds.year) = 100
> PDS byte 14 (pds.month) = 5
> PDS byte 15 (pds.day) = 5
> PDS byte 16 (pds.hour) = 23
> PDS byte 17 (pds.minute) = 0
> PDS byte 18 (pds.time_unit) = 1
> PDS byte 19 (pds.time_p1) = 0
> PDS byte 20 (pds.time_p2) = 1
> PDS byte 21 (pds.time_range) = 4
> PDS bytes 22-23 (pds.avg_num) = 0
> PDS byte 24 (pds.avg_miss) = 0
> PDS byte 25 (pds.century) = 20
> PDS byte 26 (pds.izero) = 0
> PDS bytes 27-28 (pds.dec_scale) = 1
> PDS EXT FLAG (1-app,0-nc,-1-rep) = 0
> PDS EXT STRING =
> GDS bytes 1 - 3 (gds.length) = 32
> GDS byte 4 (gds.NV) = 0
> GDS byte 5 (gds.PV) = 255
> GDS byte 6 (gds.grid_proj) = 5
> GDS bytes 7 - 8 (Nx) = 1160
> GDS bytes 9 - 10 (Ny) = 880
> GDS bytes 11 - 13 (La1) = 22774
> GDS bytes 14 - 16 (Lo1) = 239624
> GDS byte 17 (flag1) = 8
> GDS bytes 18 - 20 (LoV) = 255000
> GDS bytes 21 - 23 (Dx) = 4763
> GDS bytes 24 - 26 (Dy) = 4763
> GDS byte 27 (flag2) = 0
> GDS byte 28 (scan_mode) = 64
> GDS bytes 29 - 32 (skipped)
> BDS bytes 1 - 3 (bds.length) = 274394
> BDS byte 4 (bds.flag) = 4
> BDS bytes 5 - 6 (bds.binary_scale) = 0
> BDS bytes 7 - 10 (bds.ref_value) = 0.000000
> BDS byte 11 (bds.num_bits) = 10
> Changing center table to cntrgrib1.tbl
> Changing vertical coord table to vcrdgrib1.tbl
> Changing WMO parameter table to wmogrib2.tbl
> Changing center parameter table to ncepgrib2.tbl
>Segmentation Fault (core dumped)
>
>
>I added a line as follows to gribkey.tbl before running the above:
>
>007 x 152 240 data/YYYYMMDDHH_pcn@@@.gem 2000
>
>By-the-way... I added the above line to the bottom of the gribkey.tbl file
>and it had no effect until I moved it to the top of the file, so I think
>there may be a bug in dcgrib2 that prevents it from reading the last
>line(s) or doesn't notify of an array shortage for entries or some such
>thing, FYI.
>
>The output log file contains:
>
>[28000] 010404/1347 [DC 3] Starting up.
>[28000] 010404/1347 [DC -11] No command line arguments found.
>[28000] 010404/1347 [DC 2] read 10240/16383 bytes strt 0 newstrt 10240
>[28000] 010404/1347 [DC 2] read 6144/6144 bytes strt 10240 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9216/10238 bytes strt 0 newstrt 9216
>[28000] 010404/1347 [DC 2] read 7168/7168 bytes strt 9216 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9214/9214 bytes strt 0 newstrt 9214
>[28000] 010404/1347 [DC 2] read 7170/7170 bytes strt 9214 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9212/9212 bytes strt 0 newstrt 9212
>[28000] 010404/1347 [DC 2] read 7172/7172 bytes strt 9212 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9210/9210 bytes strt 0 newstrt 9210
>[28000] 010404/1347 [DC 2] read 7174/7174 bytes strt 9210 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9208/9208 bytes strt 0 newstrt 9208
>[28000] 010404/1347 [DC 2] read 7176/7176 bytes strt 9208 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9206/9206 bytes strt 0 newstrt 9206
>[28000] 010404/1347 [DC 2] read 7178/7178 bytes strt 9206 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9204/9204 bytes strt 0 newstrt 9204
>[28000] 010404/1347 [DC 2] read 7180/7180 bytes strt 9204 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9202/9202 bytes strt 0 newstrt 9202
>[28000] 010404/1347 [DC 2] read 7182/7182 bytes strt 9202 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9200/9200 bytes strt 0 newstrt 9200
>[28000] 010404/1347 [DC 2] read 7184/7184 bytes strt 9200 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9198/9198 bytes strt 0 newstrt 9198
>[28000] 010404/1347 [DC 2] read 7186/7186 bytes strt 9198 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9196/9196 bytes strt 0 newstrt 9196
>[28000] 010404/1347 [DC 2] read 7188/7188 bytes strt 9196 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9194/9194 bytes strt 0 newstrt 9194
>[28000] 010404/1347 [DC 2] read 7190/7190 bytes strt 9194 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9192/9192 bytes strt 0 newstrt 9192
>[28000] 010404/1347 [DC 2] read 7192/7192 bytes strt 9192 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9190/9190 bytes strt 0 newstrt 9190
>[28000] 010404/1347 [DC 2] read 7194/7194 bytes strt 9190 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9188/9188 bytes strt 0 newstrt 9188
>[28000] 010404/1347 [DC 2] read 7196/7196 bytes strt 9188 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9186/9186 bytes strt 0 newstrt 9186
>[28000] 010404/1347 [DC 2] read 7198/7198 bytes strt 9186 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9184/9184 bytes strt 0 newstrt 9184
>[28000] 010404/1347 [DC 2] read 7200/7200 bytes strt 9184 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9182/9182 bytes strt 0 newstrt 9182
>[28000] 010404/1347 [DC 2] read 7202/7202 bytes strt 9182 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9180/9180 bytes strt 0 newstrt 9180
>[28000] 010404/1347 [DC 2] read 7204/7204 bytes strt 9180 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9178/9178 bytes strt 0 newstrt 9178
>[28000] 010404/1347 [DC 2] read 7206/7206 bytes strt 9178 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9176/9176 bytes strt 0 newstrt 9176
>[28000] 010404/1347 [DC 2] read 7208/7208 bytes strt 9176 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9174/9174 bytes strt 0 newstrt 9174
>[28000] 010404/1347 [DC 2] read 7210/7210 bytes strt 9174 newstrt 16384
>[28000] 010404/1347 [DC 2] read 9172/9172 bytes strt 0 newstrt 9172
>[28000] 010404/1347 [DC 2] read 7212/7212 bytes strt 9172 newstrt 16384
>[28000] 010404/1347 [DC 2] read 8856/9170 bytes strt 0 newstrt 8856
>[28000] 010404/1347 [DCGRIB 0] grib tables [cntr 7 edtn 1 vers 2]
>[28000] 010404/1347 [DCGRIB 0] Opened data/2000050523_pcn240.gem model:152
>[28000] 010404/1347 [DCGRIB 0] grid:240
>
>
>A gempak file is produced, but I'm not sure if it's usable. gdinfo shows:
>
> GEMPAK-GDINFO>list
> GDFILE = 2000050523_pcn240.gem
> LSTALL = YES
> OUTPUT = T
> GDATTIM = ALL
> GLEVEL = ALL
> GVCORD = ALL
> GFUNC = ALL
> GEMPAK-GDINFO>r
>
> GRID FILE: 2000050523_pcn240.gem
>
> GRID NAVIGATION:
> UNKNOWN GRID NAVIGATION
> GRID ANALYSIS BLOCK:
> UNKNOWN ANALYSIS TYPE
>
> Number of grids in file: 0
>
> Maximum number of grids in file: 2000
>
> [GDU 2] Did not find any matching grids.
> Parameters requested: GDFILE,LSTALL,OUTPUT,GDATTIM,GLEVEL,GVCORD,GFUNC.
>
>
>Can you give me some ideas? Is there a problem trying to use the 240
>grid, or something else? I have a description for the 240 grid if it's
>needed.
>
> Thanks again.
>
> Art.
>>
>> Steve Chiswell
>> Unidata User Support
>>
>>
>>
>> >From: "Arthur A. Person" <address@hidden>
>> >Organization: UCAR/Unidata
>> >Keywords: 200104032111.f33LBAL14985
>>
>> >On Wed, 21 Mar 2001, Unidata Support wrote:
>> >
>> >> >From: "Arthur A. Person" <address@hidden>
>> >> >Organization: UCAR/Unidata
>> >> >Keywords: 200103211815.f2LIFnL03774
>> >>
>> >> >Hi,
>> >> >
>> >> >I want to decode 4km precip grib data from NCEP using dcgrib2 but need t
> o
>> >> >make mods to allow for larger gribs and grids (1160 X 880). The NCEP
>> >> >instructions said to use nagrib but I should be able to use dcgrib2,
>> >> >correct? I've identified that I need to make MAXGRIBSIZE larger in
>> >> >decode_grib.c and I think I need to make LLMXGD larger for the overall
>> >> >grid size. Can you tell me if LLMXGD is the only other thing I need to
>> >> >modify to make this work correctly? Do I then need to rebuild the whole
>> >> >gempak package, or can I just build part of it to get dcgrib2 to work?
>> >> >
>> >> > Thanks.
>> >> >
>> >> > Art.
>> >> >
>> >> >Arthur A. Person
>> >> >Research Assistant, System Administrator
>> >> >Penn State Department of Meteorology
>> >> >email: address@hidden, phone: 814-863-1563
>> >> >
>> >>
>> >>
>> >> Art,
>> >>
>> >> You need to rebuild the entire package (including all the $GEMLIB library
>> >> files) whenever changing any array sizes defined in the $GEMPAK/include
>> >> files (since this will change common block sizes and sizes passed in
>> >> subroutine calls). You can run "make distclean" from $NAWIPS to
>> >> remove the $GEMLIB files as well as any other build files from the tree.
>> >>
>> >> LLMXGD should be changed in both the fortran GEMPRM.xxx file as well as
>> >> the C gemprm.h file. The computation heap for grids defined as LLMDGG
>> >> is related since this is the amount of space used by computations
>> >> (eg each grid in the computation uses this space) that
>> >> use more than 1 grid- so it should be large enough to allow you to
>> >> hold at least 4x the grid size probably.
>> >>
>> >> The MAXGRIBSIZE parameter in decode_grib.c is the size of the buffer
>> >> for the largest "grib message" you will encounter in the data stream
>> >> (that is the grib packed message).
>> >
>> >Okay, I made a new gempak installation tree called NAWIPS-5.6.a-big and
>> >put a fresh copy of gempak there. I modified the following:
>> >
>> > gempak/include/MCHPRM.SunOS -> LLMXGD=1200000
>> > gempak/include/gemprm.h -> LLMXGD=1200000
>> > gempak/include/GEMPRM.PRM -> LLMDGG=7200000
>> > unidata/ldmbridge/dcgrib2/decode_grib.c -> MAXGRIBSIZE=500000
>> >
>> >I re-make'd and re-install'd (on Solaris 2.7) and tried running the
>> >dcgrib2 decoder and it core dumps. I tried just an empty command to see
>> >what it would do and I get:
>> >
>> > [16863] 010403/1658 [DC 3] Starting up.
>> > [16863] 010403/1658 [DC -11] No command line arguments found.
>> > Segmentation Fault (core dumped)
>> >
>> >I've apparently missed something else that needs modified... any clues?
>> >
>> > Thanks.
>> >
>> > Art.
>> >
>> >
>> >Arthur A. Person
>> >Research Assistant, System Administrator
>> >Penn State Department of Meteorology
>> >email: address@hidden, phone: 814-863-1563
>> >
>>
>> ****************************************************************************
>> Unidata User Support UCAR Unidata Program
>> (303)497-8644 P.O. Box 3000
>> address@hidden Boulder, CO 80307
>> ----------------------------------------------------------------------------
>> Unidata WWW Service http://www.unidata.ucar.edu/
>> ****************************************************************************
>>
>
>Arthur A. Person
>Research Assistant, System Administrator
>Penn State Department of Meteorology
>email: address@hidden, phone: 814-863-1563
>