3.3.1 - Update Data Files
Action Option -UF (-UpdateFile) :
is a comprehensive action that downloads or copies server files, validates and converts them into remote files, builds local files from the available remote files, archives local files to the GDEX server, and cleans up temporary data files generated during the update procedure.
Available Mode options:
works with -BP (-BatchProcess) to allow the recorded dsupdt command to start from any directory |
|
runs in the background; suppresses screen output and errors |
|
when the server file is available locally, checks whether it has changed on the server and re-downloads if so |
|
allows the end date/hour to be processed even if it falls within the current update period |
|
sends email only when an error occurs during updates |
|
forces an update for at least one end date/time, even if the update is not yet due |
|
advances time tracking to hours after a successful file update |
|
works with Mode option -MU (-MultipleUpdate) to skip failed updates and continue processing the remaining updates |
|
copies rather than moves the remote file to the local file location, preserving the remote file on disk |
|
copies rather than moves the server file to the remote file location, preserving the server file on disk |
|
enables detailed logging when -PL is present |
|
processes only files that have not yet been archived |
|
processes all available update periods; without this option, only the most recent period is processed |
|
suppresses post-update email notification |
|
skips February 29 in leap years |
|
stops all update processing for the dataset on the first error, rather than skipping and continuing |
|
re-downloads the remote file even if it already exists locally |
|
resets the end date/hour based on the local file timestamp |
|
sends a summary email to the specialist after the update without detail logging information |
|
uses the period’s beginning time instead of the end time when substituting temporal patterns |
|
forces an update of the data end time and the next-due update times |
Provide a dataset number via -DS to restrict updates to a single dataset. Runtime Info options can identify specific records or override GDEXDB values. -CD (-CurrentDate) and -CH (-CurrentHour) are runtime-only options that substitute a different date or hour for the system default.
When -MU (-MultipleUpdate) is present, every elapsed update period between the data end date/hour and the current date/hour is processed.
As noted in the introduction, an error during an individual file’s update stops that file’s processing. However, remote file download errors are tolerated when -MR (-MissRemote) is ‘Y’ for records whose local file is built from multiple remote files, permitting a partial update. If -VI (-ValidInterval) is also set, the partial update is held open until the missing files arrive or the valid period expires.
When Info option -PL is set to a value greater than 1, dsupdt forks multiple child processes, each handling one update record. This improves performance for datasets containing many time-consuming, independent update records.
Additional notification recipients can be added via -CC (-CarbonCopy). GDEX specialists may use login names directly, e.g., ‘-CC zji schuster’.
-UF can be run from the command line, via cron job, or through the daemon dscheck when local file records are linked to an update control record. Rather than supplying Mode options at run time, most can be pre-configured in the update control record.
Alternatively, the individual actions -DR (-DownloadRemote), -BL (-BuildLocal), -AF (-ArchiveFile), and -CF (-CleanFile) can be used to step through the download/archive procedure. Their command-line syntax is similar to -UF (-UpdateFile) and is described in the following sections.