Subset a Sipnet clim file line-by-line without parsing columns
Source:R/subset_met_lines.R
subset_met_lines.RdIgnores lines before the target date, reads and writes wanted lines as character without parsing any columns, and exits without touching lines after the end date. This is faster than reading the entire table and subsetting, especially when taking small subsets of large files that live on slow disks.
Details
If the input file is fixed-width (as files produced by met2model.SIPNET are), gets a further speedup by seeking directly to the byte offset for start_day. Variable-width clim files do not get this speedup (because file position needs to be determined by scanning for end-of-line markers), but they still benefit from skipping all lines after stop_day.
Only works for files with a constant timestep, and only copies whole days, i.e. if you have hourly data the output length will be a multiple of 24.
The step size and line length are detected by reading the first n_head lines
of the original file; this number is tunable with less being likely faster
and more being more likely to detect variation, but probably doesn't need to
be more than one day.
TODO: To match split.inputs.SIPNET, the end date is exclusive: stop_day is the date to stop before, not the last day to include in the output. This is confusing and differs from most of the rest of PEcAn – how committed to this definition are we?