This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Hi Samuel, Sorry for the late reply. It was a busy day... re: > I'm running into a similar type problem in /data/ldm/mcidas/grib, where > there are files named *2008081*, *2008080*, *2008074*, and *2008073* files. > Should I delete them too? Hes, all of these are very old. > There are also some files in /data/ldm/mcidas/bufr that have access times > greater than 20 days ago (*2008072*,*2000001*, *1999001*, etc...) should I > delete these types of files also? Actually, McIDAS does not do anything with the BUFR files yet, so you are free to delete all of them. They are filed so that sites that have written their own BUFR access routines/decoders have them accessible. Their existence is _not_ recorded in the MySQL database. > One possible solution I can think of for removing files that have access > times greater than expected, is setup a crontab entry that would have > something > like the following entry: > find /data/ldm/mcidas/grib -name "*" -atime +10 -print -exec rm -f {} \; Looks like the scouring approach that is used in the LDM utility 'scour'. > I'm not certain if this would cleanup the sql database, however. There is no MySQL impact for the BUFR files. I don't believe that there is anything you need to do for GRIB files that have names that indicate old data in the MySQL database. I say this after reviewing the method used in the XCD scouring utility, xcdscour: GRIB|grib) i=$max_days_to_delete # MySQL # Do this before deleting files so client doesnt get false positives LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH export LD_LIBRARY_PATH daynum=`days_ago_year $days` MBIN=`which mysql |grep -v no` if [ -n "$MBIN" ]; then strSQL="DELETE FROM file_list WHERE j_day <= \"$daynum\";" echo "$strSQL" |$MBIN --user=gribwrite --password=gribwrite mcrtgrib >/dev/null 2>&1 strSQL="OPTIMIZE table file_list;" echo "$strSQL" |$MBIN --user=gribwrite --password=gribwrite mcrtgrib >/dev/null 2>&1 fi while [[ $i -ge $days ]]; do daynum=`days_ago_year $i` hournum=0 while [[ $hournum -le 23 ]]; do rm -f *.*.$daynum.$hournum??.*.*.grib rm -f *.*.$daynum.$hournum??.*.grb2 hournum=$((hournum+1)) done rm -f *.*.$daynum.*.*.*.grib rm -f *.*.$daynum.*.*.grb2 i=$((i-1)) done ;; The operative portion of this snippit is: daynum=`days_ago_year $days` MBIN=`which mysql |grep -v no` if [ -n "$MBIN" ]; then strSQL="DELETE FROM file_list WHERE j_day <= \"$daynum\";" echo "$strSQL" |$MBIN --user=gribwrite --password=gribwrite mcrtgrib >/dev/null 2>&1 strSQL="OPTIMIZE table file_list;" echo "$strSQL" |$MBIN --user=gribwrite --password=gribwrite mcrtgrib >/dev/null 2>&1 fi which says to delete entries in the MySQL database whose dates are <= $daynum. To me this means that all files older than $daynum will be deleted. Note, however, that this does not account for files that get named with dates in the future. They should be scoured eventually, however. Cheers, Tom **************************************************************************** Unidata User Support UCAR Unidata Program (303) 497-8642 P.O. Box 3000 address@hidden Boulder, CO 80307 ---------------------------------------------------------------------------- Unidata HomePage http://www.unidata.ucar.edu **************************************************************************** Ticket Details =================== Ticket ID: XRR-599672 Department: Support McIDAS Priority: Normal Status: Closed