Apply functions to PEcAn MultiSettings (EXPERIMENTAL parallel version)
Source:R/future_papply.R
future_papply.RdWorks like lapply(), but for PEcAn Settings and MultiSettings objects. The "p" is for "PEcAn".
Arguments
- settings
A
MultiSettingsorSettingsobject to operate on- fn
The function to apply to
settings- ...
additional arguments to
fn- stop.on.error
Whether to halt execution if a single element in
settingsresults in error. See Details.
Value
A single fn return value, or a list of such values
(coerced to MultiSettings if appropriate; see Details)
Details
This is an experimental parallel-enabled version of papply, released under a
new name for testing. If it performs well in testing, the functionality in
future_papply() may take over the existing papply() name in a future release.
papply calls a function on each Settings object in a
MultiSettings object, and returns the results in a list.
It has some additional features, however:
If the result of
fnis aSettingsobject, thenpapplywill coerce the returned list into a newMultiSettings.If
settingsis aSettingsobject, thenpapplyknows to callfnon it directly.By default,
papplywill proceed even iffnthrows an error for one or more of the elements insettings. Note that if this option is used, the returned results list will have entries for only those elements that did not result in an error.
MultiSettings objects are processed using furrr::future_map.
Components will be run in parallel if your session configures a parallel
backend via future::plan and in series if it does not.
Warning & feedback request
This implementation is experimental! If it performs well in testing it may
eventually replace the existing papply function, but we want to be sure it
is a truly drop-in replacement before doing that. Please report your
experience with it, good or bad.
We are especially interested in reports/patches/suggestions/test cases around
Identifying applications of papply that require serial execution or where parallel execution may be unwanted
Testing for concurrency issues (esp. in calls that write a lot of files)
Ensuring that console output (print(), logger.info(), etc) is handled correctly and not lost
Testing against unusually-formatted MultiSettings objects (e.g. anything that varies a dimension other than site)
Examples
f = function(settings, ...) {
# Here's how I envisioned a typical use case within a standard PEcAn function
if(is.MultiSettings(settings)) {
return(papply(settings, f, ...))
}
# Don't worry about the beolow, it's just some guts to make the function do something we can see
l <- list(...)
for(i in seq_along(l)) {
ind <- length(settings) + 1
settings[[ind]] <- l[[i]]
if(!is.null(names(l))) {
names(settings)[ind] <- names(l)[i]
}
}
return(settings)
}
# Example
settings1 <- Settings(list(a="aa", b=1:3, c="NA"))
settings2 <- Settings(list(a="A", b=4:5, c=paste))
multiSettings <- MultiSettings(settings1, settings2)
# The fucntion should add element $d = D to either a Settings, or each entry in a MultiSettings
f(settings1, d="D")
#> $a
#> [1] "aa"
#>
#> $b
#> [1] 1 2 3
#>
#> $c
#> [1] "NA"
#>
#> $d
#> [1] "D"
#>
#> attr(,"class")
#> [1] "Settings" "SafeList" "list"
print(f(multiSettings, d="D"), TRUE)
#> $settings.1
#> $a
#> [1] "aa"
#>
#> $b
#> [1] 1 2 3
#>
#> $c
#> [1] "NA"
#>
#> $d
#> [1] "D"
#>
#> attr(,"class")
#> [1] "Settings" "SafeList" "list"
#>
#> $settings.2
#> $a
#> [1] "A"
#>
#> $b
#> [1] 4 5
#>
#> $c
#> function (..., sep = " ", collapse = NULL, recycle0 = FALSE)
#> .Internal(paste(list(...), sep, collapse, recycle0))
#> <bytecode: 0x5fce77d6e578>
#> <environment: namespace:base>
#>
#> $d
#> [1] "D"
#>
#> attr(,"class")
#> [1] "Settings" "SafeList" "list"
#>
#> attr(,"class")
#> [1] "MultiSettings" "list"