[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #RMC-391545]: Variable names in pqact output file path?

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [LDM #RMC-391545]: Variable names in pqact output file path?
  • Date: Thu, 29 Oct 2015 11:51:29 -0600
Neil,

> Is there a syntax available to reference a variable’s content somewhere in 
> the output path name specification in a pqact entry?
> 
> I’m wanting to investigate a way of removing a previous version of a product 
> file when the latest version has arrived.
> 
> For example, say a NEXRCOMP pqact entry looks like
> 
> FNEXRAD ^rad/NEXRCOMP/(...)/(...)_(........)_(....)
> FILE    -close  data/gempak/nexrad/NEXRCOMP/\1/\2/\2_\3_\4
> 
> and with product output file path
> 
> /data/gempak/nexrad/NEXRCOMP/1km/n0r/n0r_20151026_0615
> 
> When considering a particular use of the linux ‘date’ command that looks like
> 
> > date -d '20151029 -3 days' +%Y%m%d
> 20151026
> 
> it is very tempting to consider something like
> 
> FNEXRAD       ^rad/NEXRCOMP/(...)/(...)_(........)_(....)
> FILE    -close  data/gempak/nexrad/NEXRCOMP/\1/\2/\2_\3_\4
> 
> FNEXRAD       ^rad/NEXRCOMP/(...)/(...)_(........)_(....)
> EXEC    set plugin = `/bin/date  -d '\3 -3 day’`
> 
> FNEXRAD       ^rad/NEXRCOMP/(...)/(...)_(........)_(....)
> EXEC    /bin/rm -f data/gempak/nexrad/NEXRCOMP/\1/\2/test_\2_${plugin}_\4
> 
> to ‘scour’ in real time the on-disk population of an ingested IDD product by 
> removing
> 
> /data/gempak/nexrad/NEXRCOMP/1km/n0r/n0r_20151026_0615
> 
> (I assume that the ldm will not except more than one pattern action per 
> pattern)
> 
> Is this a bit hair-balled?
> 
> -Neil

You idea isn't "hair-balled", as you say. It might not be what you want, 
however. Modifying the parsing of the pqact(1) configuration-file is 
straightforward but non-trivial; consequently, it would take time (which is in 
short supply). Additionally, the result might not perform as well as you'd like 
because a separate process would be created, executed, and terminated for every 
arriving data-product that matched the pattern. This would result in deletion 
that would be no better than using the command "find ... -exec rm {}", which is 
already too slow in some circumstances.

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: RMC-391545
Department: Support LDM
Priority: Normal
Status: Closed