Create Future EPW Files

Hongyuan Jia

Source: vignettes/articles/future-epw-workflow.Rmd 
library(epwshiftr)

if (!exists("shift_request", mode = "function")) {
    stop(
        "This raw article must be rendered from the package source with ",
        "`Rscript tools/raw-vignettes.R render`.",
        call. = FALSE
    )
}

This article is the recommended main workflow for epwshiftr. It runs a real store-native shift workflow from ESGF File records to generated EPW files. It uses monthly Amon data so the live remote reads are much smaller than an equivalent daily workflow.

The first example uses the task-oriented API. The staged walkthrough that follows exposes the lower-level shift_*() facade for inspection and teaching. When a step touches a lower-level engine, this article links to the companion article for that layer: ESGF query results, ESG dictionaries, ESG stores, Downloader, EpwMorpher, CLI operations, and ESGF troubleshooting.

Use a temporary store for this article run. For a real project, replace this with a persistent project cache path.

workflow_root <- file.path(tempdir(), "epwshiftr-future-epw-workflow")
if (dir.exists(workflow_root)) {
    unlink(workflow_root, recursive = TRUE)
}
dir.create(workflow_root, recursive = TRUE, showWarnings = FALSE)

options(
    epwshiftr.dir_cache = file.path(workflow_root, "cache")
)

The recommended user call gives Belcher an explicit matching historical CMIP6 reference. When no suitable reference data exist, belcher() can instead use the baseline EPW climatology; reference = NULL never creates a historical CMIP6 request implicitly.

epw <- system.file(
    "extdata/examples/SGP_Singapore.486980_IWEC.epw",
    package = "epwshiftr",
    mustWork = TRUE
)

run <- shift_future_epw(
    epw = epw,
    climate = shift_cmip6(
        model = "BCC-CSM2-MR",
        scenarios = c("ssp126", "ssp585")
    ),
    periods = list(`2060s` = 2055:2065),
    method = belcher(
        reference = historical_reference(1995:2014)
    ),
    dir = file.path(workflow_root, "future-epw")
)

shift_status(run)
shift_outputs(run)
shift_missing(run)

The default belcher() profile is enhanced. With shift_cmip6(table = NULL), the resolver selects exact table/grid partitions per variable: atmospheric inputs normally come from Amon, while optional snd comes from LImon only when both future and historical cases provide it. Use belcher(profile = "legacy") only when reproducing the earlier algorithm and headers is required.

Profile options are part of the scientific task specification rather than UI preferences. Configure them on the method and keep table = NULL unless a dataset requires an explicit override:

method <- belcher(
    reference = historical_reference(1995:2014),
    options = belcher_options(
        transition_hours = 72L,
        humidity_source = "auto",
        snow_depth = "auto"
    )
)

# Named values override individual variables; unnamed scalars force every
# variable into one table and are rarely suitable when snd is enabled.
climate <- shift_cmip6(
    "BCC-CSM2-MR", "ssp585",
    table = c(snd = "LImon")
)

See Inspect EPW Morphing for the complete option matrix and the enhanced/legacy defaults.

Use the same call with dry_run = TRUE to obtain a ShiftPlan, inspect it with shift_explain(), and execute it later with shift_run(). The persisted run ID supports shift_status(), shift_resume(), and the corresponding CLI commands.

Inspect Shift Objects

Every public Shift object prints as a compact semantic receipt. Configuration objects summarize scientific intent without expanding stored rules, function environments, yearly vectors, or every ESGF node. Stage objects add a preview of the records that matter at that point in the workflow. ShiftRun refreshes its persisted state and prints the same static dashboard used by shift_watch().

climate <- shift_cmip6(
    model = "BCC-CSM2-MR",
    scenarios = c("ssp126", "ssp585")
)
method <- belcher(reference = historical_reference(1995:2014))
plan <- shift_future_epw(
    epw = epw,
    climate = climate,
    periods = list(`2060s` = 2055:2065),
    method = method,
    dir = file.path(workflow_root, "future-epw"),
    dry_run = TRUE
)

print(climate)
print(method)
print(method, verbose = TRUE)
print(plan, n = 3L, width = 80L)

Data previews show at most 10 rows by default. Use n = Inf for every row, set width when rendering into a constrained console or report, and use verbose = TRUE for store paths, stable IDs, filters, nodes, method overrides, resolved profile options, and additional diagnostics. Normal receipts identify the active method profile and whether CMIP tables are automatic or forced. After resolution, progress and watch views report exact partitions such as Amon=gn · LImon=gr instead of collapsing them to one representative grid. These controls affect presentation only and do not change the workflow specification or its spec_hash.

print(plan, n = Inf, width = 120L, verbose = TRUE)
print(run, verbose = TRUE)

Live Feedback and Background Runs

The foreground runner starts reporting before its first ESGF request. Its shared stage vocabulary is resolve, optional download, extract_future, optional extract_reference, coverage, morph, and write_epw. Within a stage it identifies the node, future/reference role, variable, scenario, period, reuse/fallback outcome, and output count.

shift_ui() controls only presentation. "auto" selects a fixed four-row stage/current/case/last-event view in a capable terminal and scoped log lines elsewhere; "none" suppresses non-error Console output while retaining persisted events. The detail levels are "normal", "detail", and "debug"; only debug output includes full URLs and internal paths.

run <- shift_future_epw(
    epw = epw,
    climate = shift_cmip6("BCC-CSM2-MR", c("ssp126", "ssp585")),
    periods = list(`2060s` = 2055:2065),
    method = belcher(reference = historical_reference(1995:2014)),
    dir = file.path(workflow_root, "future-epw"),
    ui = shift_ui(progress = "log", detail = "detail", heartbeat = 10)
)

Use a detached Rscript worker when the run should outlive the current R session. The call returns a queued ShiftRun immediately. Inspectors refresh that handle from the store, and shift_watch() follows the same structured events shown by the foreground reporter.

run <- shift_future_epw(
    epw = epw,
    climate = shift_cmip6("BCC-CSM2-MR", c("ssp126", "ssp585")),
    periods = list(`2060s` = 2055:2065),
    method = belcher(reference = historical_reference(1995:2014)),
    dir = file.path(workflow_root, "future-epw"),
    background = TRUE
)

shift_status(run)
shift_watch(run)
shift_logs(run, tail = 50)

# Request cancellation at the next workflow boundary:
shift_cancel(run)

# Terminate the recorded worker PID after persisting the request:
shift_cancel(run, force = TRUE)

Interrupting shift_watch() stops only the monitor. It does not cancel the worker. A failed, cancelled, or partial run can create a new attempt with shift_resume(run, background = TRUE); the resolved node, member, and exact per-table grid partitions remain pinned.

Site and Request

shift_site() describes the location that will be extracted from climate projection data. The id is the stable site key used in extraction plans, manifest rows, and output naming, so use a short value that will still make sense when you process several sites.

This article writes a deterministic Singapore baseline EPW into the temporary workflow directory so the raw render is self-contained. The climate query and remote data reads below still use live ESGF services.

You can provide the site metadata directly:

epw <- write_vignette_epw(
    file.path(workflow_root, "baseline", "SGP_Singapore.486980_IWEC.epw")
)

site <- shift_site(
    id = "SIN",
    lon = 103.98,
    lat = 1.37,
    label = "Singapore"
)

site
#> ══ EPW Site ════════════════════════════════════════════════════════════════════
#> • ID: SIN
#> • Label: Singapore
#> • Coordinates: 103.980000, 1.370000

Or you can read the same information directly from the EPW LOCATION header:

epw_site <- shift_site(epw)
epw_site
#> ══ EPW Site ════════════════════════════════════════════════════════════════════
#> • ID: SGP_Singapore.486980_IWEC
#> • Label: Singapore
#> • Coordinates: 103.980000, 1.370000
#> • EPW: SGP_Singapore.486980_IWEC.epw

This article keeps using the explicit site object so the site ID is short and predictable, while the EPW file itself is passed later as the morphing baseline.

epw_morph_variables() returns the CMIP variable IDs needed by the selected morphing recipe. The result is a plain character vector because the same variable IDs are used at several workflow stages: first as the ESGF variable_id filter in shift_request(), then as the extraction variable list in shift_extract(), and finally as the coverage check used by shift_morph().

The helper provides three named variable sets:

  • "minimal": air temperature and relative humidity, useful for relaxed demonstrations.
  • "recommended": the current strict Belcher recipe set, including precipitation.
  • "extended": the recommended set plus related max/min variables and snow depth.

This staged walkthrough explicitly uses the Belcher recipe. It is a change-factor backend, so the morphing step needs both a future climate extraction and a reference climate extraction.

shift_recipe <- epw_morph_recipe("belcher")
variables <- epw_morph_variables(shift_recipe)
optional_variables <- epw_morph_variables(
    shift_recipe, include_optional = TRUE
)
variables
#> [1] "tas"     "hurs"    "psl"     "rlds"    "rsds"    "sfcWind" "clt"
#> [8] "pr"

shift_request() describes the remote climate data you want. The future request targets one ESGF ScenarioMIP model, scenario, member, and monthly table. The reference request uses the matching historical experiment so Belcher change-factor morphing can compare future monthly fields with reference monthly fields before applying those changes to the baseline EPW.

The values inside filters are ESGF search fields. options is not a search filter. options$index_node chooses the ESGF search node; if omitted, epwshiftr uses its default ESGF index. The other request option currently recognized by the ESGF adapter is time_filter_method, which controls how File records are filtered by time after Dataset collection; the default is "drs" filename parsing. The time argument narrows File records after Dataset collection; it does not mean the whole period is downloaded. A numeric year such as 2060L is expanded to that whole calendar year.

Read more about the lower-level query object in ESGF query results, and about request-value validation in ESG dictionaries.

request <- shift_request(
    project = "CMIP6",
    time = 2060L,
    filters = list(
        activity_id = "ScenarioMIP",
        source_id = "MPI-ESM1-2-LR",
        experiment_id = "ssp585",
        variant_label = "r1i1p1f1",
        frequency = "mon",
        variable_id = variables,
        data_node = "esgf.ceda.ac.uk",
        table_id = "Amon"
    ),
    options = list(index_node = "https://esgf-data.dkrz.de")
)

request
#> ══ ESGF request ════════════════════════════════════════════════════════════════
#> • Index node: https://esgf-data.dkrz.de
#> ── Query parameters ────────────────────────────────────────────────────────────
#> • project = CMIP6
#> • activity_id = ScenarioMIP
#> • experiment_id = ssp585
#> • source_id = MPI-ESM1-2-LR
#> • variable_id = tas, hurs, psl, rlds, rsds, sfcWind, clt, pr
#> • frequency = mon
#> • variant_label = r1i1p1f1
#> • data_node = esgf.ceda.ac.uk
#> • fields = *
#> • type = Dataset
#> • offset = 0
#> • distrib = true
#> • limit = 10
#> • format = application/solr+json
#> • table_id = Amon
#> • datetime_start: [* TO 2060-01-01T00:00:00Z]
#> • datetime_stop: [2060-12-31T23:59:59Z TO *]

reference_request <- shift_request(
    project = "CMIP6",
    time = 1995L,
    filters = list(
        activity_id = "CMIP",
        source_id = "MPI-ESM1-2-LR",
        experiment_id = "historical",
        variant_label = "r1i1p1f1",
        frequency = "mon",
        variable_id = variables,
        data_node = "esgf.ceda.ac.uk",
        table_id = "Amon"
    ),
    options = list(index_node = "https://esgf-data.dkrz.de")
)

reference_request
#> ══ ESGF request ════════════════════════════════════════════════════════════════
#> • Index node: https://esgf-data.dkrz.de
#> ── Query parameters ────────────────────────────────────────────────────────────
#> • project = CMIP6
#> • activity_id = CMIP
#> • experiment_id = historical
#> • source_id = MPI-ESM1-2-LR
#> • variable_id = tas, hurs, psl, rlds, rsds, sfcWind, clt, pr
#> • frequency = mon
#> • variant_label = r1i1p1f1
#> • data_node = esgf.ceda.ac.uk
#> • fields = *
#> • type = Dataset
#> • offset = 0
#> • distrib = true
#> • limit = 10
#> • format = application/solr+json
#> • table_id = Amon
#> • datetime_start: [* TO 1995-01-01T00:00:00Z]
#> • datetime_stop: [1995-12-31T23:59:59Z TO *]

Run the Workflow

The diagram below is the map for the rest of the article. Each shift_*() call returns a stage object that can be printed, checked, and passed to the next step. The stage also carries run_id and step_id, so the next shift_*() call automatically continues the same persisted run. There is no separate session object to create or pass.

shift_download() is optional. Use it when you want to prefetch full NetCDF files for offline work, repeated extraction, or unstable OPeNDAP access.

The ordinary path goes directly from collected File records to extraction. Full NetCDF downloads are not required unless you intentionally want a local source-file cache.

Inspect Dataset Matches

Before collecting File records, inspect the Dataset matches. If this table is broader or narrower than intended, change the request filters before continuing. shift_datasets() runs the Dataset-level ESGF search described by shift_request(). The returned EsgResultDataset object is not the data to download yet; it is the list of Dataset records that will later be expanded into File records. This standalone query uses the same dashboard, persisted run, and query heartbeat reporting as the other shift_*() tasks. Use ui = shift_ui(...) to control its presentation; low-level EsgQuery$collect() continues to expose its native progress control.

For a deeper look at Dataset, File, and Aggregation results, see ESGF query results.

The remaining chunks are a live ESGF walkthrough. They are displayed but not executed while the package documentation is precompiled, which keeps installed vignettes deterministic and avoids turning documentation builds into remote network jobs. Run them interactively to reproduce the workflow.

datasets <- shift_datasets(request)
datasets

Use $to_data_table() when you want row-level details for decisions such as whether the request matched the expected variables, model, variant, and data node.

dataset_table <- datasets$to_data_table(fields = c(
    "id", "source_id", "experiment_id", "variant_label",
    "variable_id", "data_node", "number_of_files"
), formatted = TRUE)

dataset_table

Summarise the table before moving on. In this request, each variable should have one matching Dataset record on the selected data node.

dataset_table[, .(
    datasets = .N,
    files = sum(number_of_files, na.rm = TRUE),
    variables = paste(sort(unique(variable_id)), collapse = ", ")
), by = .(source_id, experiment_id, variant_label, data_node)]

The Dataset result can also be filtered locally before collecting child File records. This is useful when a broad request intentionally returns several models, variants, data nodes, or variables and you want to inspect or keep only part of the match. The example below keeps only two variables so the effect is easy to see.

selected_datasets <- datasets$filter(function(x) {
    x$variable_id %in% c("tas", "hurs")
})

selected_datasets$to_data_table(fields = c(
    "id", "variable_id", "data_node", "number_of_files"
))

For a lower-level workflow, you can collect File records from that subset directly. The staged shift_collect() call below performs the same Dataset-to-File expansion for the original request and stores the result in an EsgStore, so the main workflow continues with shift_collect().

selected_files <- selected_datasets$collect(
    type = "File",
    fields = "*",
    all = TRUE,
    limit = NULL
)

Collect File Records

store is the local directory where epwshiftr records ESGF File metadata, download tasks, extraction outputs, morphing factors, and generated EPWs. shift_collect() first collects Dataset records, then uses Dataset$collect(type = "File") to collect the concrete files needed by the rest of the workflow. The returned ShiftFiles object is the workflow stage: it remembers the store path and internal query ID so later steps do not need the user to pass file paths or manifest IDs.

For store internals such as query snapshots, file catalogs, artifacts, and tracked updates, see ESG stores.

files <- shift_collect(
    request,
    store = file.path(workflow_root, "singapore-store")
)

files

reference_files <- shift_collect(
    reference_request,
    store = file.path(workflow_root, "singapore-store")
)

reference_files

Use shift_files() when you want to inspect the underlying EsgResultFile object that was saved into the store. Printing it gives the same high-level summary as a direct ESGF File query result.

file_result <- shift_files(files)
file_result

Convert the File result to a table when you want to inspect exactly which files were found. The URL columns are long, so this view shows whether each file has OPeNDAP and HTTPServer access instead of printing the full URLs.

file_table <- file_result$to_data_table(fields = c(
    "filename", "variable_id", "data_node", "size",
    "url_opendap", "url_download"
), formatted = TRUE)

file_table[, .(
    filename,
    variable_id,
    data_node,
    size,
    opendap = !is.na(url_opendap) & nzchar(url_opendap),
    http = !is.na(url_download) & nzchar(url_download)
)]

Optional: Prefetch NetCDF Files

For the normal single-site workflow, you can skip shift_download() and go directly to shift_extract(). Extraction opens the OPeNDAP URL first and reads only the requested site, variables, and time range before storing the extracted result as Parquet.

shift_download() is useful when you deliberately want a complete local copy of the original ESGF NetCDF files before extraction. It downloads full source files through selected HTTPServer URLs into the store’s downloads/ directory. This is different from OPeNDAP, which lets shift_extract() read only the requested site, variables, and time range.

Use this optional prefetch step when you plan to reuse the same source files for many sites or periods, need offline extraction later, or expect OPeNDAP to be unavailable or unstable.

By default, shift_download() runs in the foreground (run = TRUE, background = FALSE). In an interactive session, keep progress = TRUE to see per-file progress bars. This article sets progress = FALSE only to keep the precompiled output compact.

If the network drops, the downloader keeps partial .part files and resume = TRUE lets the next run continue where possible. If the final file is already present and complete, it is reused. Use overwrite = TRUE only when you want to discard an existing completed file and download it again.

If a data node becomes unstable, rerun shift_download() with the same stage. The store keeps the File records and download session metadata, while the downloader records task status and data-node history. If you run the optional chunk below, inspect the result with shift_status(downloads), shift_check(downloads), and data.table::as.data.table(downloads).

For persistent sessions, background jobs, daemon mode, retries, and node history, see Downloader. For the same operations from a terminal, see CLI operations.

downloads <- shift_download(
    files,
    replica = "current",
    service = "HTTPServer",
    strategy = "stable",
    probe = FALSE,
    progress = FALSE
)

downloads

Extract Site Climate

This is where the remote climate data are actually read in the default workflow. shift_extract() opens OPeNDAP when possible, extracts only the requested site and period, and stores the extracted rows as Parquet artifacts in the store. In the code below the result is named extracted because it is the extracted site-level climate stage. Its class is ShiftClimate, because that stage is the climate data that shift_morph() will summarise and compare with the baseline EPW.

epw_morph_periods() maps user-facing period labels to one or more years. The name, such as 2060s, becomes the period label in summaries, morphing cases, and output paths. The numeric value is the year or years used to calculate that period. This article uses one year so the remote extraction stays small:

epw_morph_periods(`2060s` = 2060L)

A wider period is also valid, for example:

epw_morph_periods(`2060s` = 2055:2064)

The collected files must cover every year used by the period.

fallback = "auto" means extraction tries OPeNDAP first and may fall back to HTTP file downloads when remote OPeNDAP access is unavailable. Use "error" when you want remote access failures to stop the extraction instead.

Extraction is recorded in the local EsgStore; see ESG stores for the lower-level API. If OPeNDAP, data-node, or coverage problems appear, see ESGF troubleshooting.

periods <- epw_morph_periods(`2060s` = 2060L)
reference_periods <- epw_morph_periods(reference = 1995L)

extracted <- shift_extract(
    files,
    site = site,
    periods = periods,
    variables = variables,
    fallback = "auto"
)

extracted

reference <- shift_extract(
    reference_files,
    site = site,
    periods = reference_periods,
    variables = variables,
    fallback = "auto"
)

reference

shift_coverage() checks whether every requested variable has extracted rows for the selected site and period. This is the main sanity check before morphing.

coverage <- shift_coverage(extracted)
coverage[, .(variable_id, complete, status, output_rows, output_file_count)]

The extracted values are not stored inside the small stage object. They are written as partitioned Parquet files under the store and registered in the store manifest. shift_artifacts() shows those registered files:

extract_artifacts <- shift_artifacts(extracted)
extract_artifacts[kind == "extract", .(kind, role, relative_path)]

Use shift_data() when you want to inspect the actual extracted table without manually finding or reading those Parquet files. By default it returns a preview instead of loading everything into memory.

extracted_data <- shift_data(
    extracted,
    n = 20L,
    columns = c("site_id", "variable_id", "time", "lon", "lat", "value", "units")
)

extracted_data

Morph Hourly Weather

shift_morph() summarises the extracted monthly climate, compares it with the baseline EPW, creates morphing factors, and writes morphed hourly results back to the store. With strict = TRUE, missing required variables or incomplete coverage are blocking errors instead of warnings.

shift_morph() wraps the lower-level EpwMorpher planning and execution API. See EpwMorpher when you need to inspect monthly summaries, factor diagnostics, case grouping, or custom backend registration.

When available, matching historical CMIP6 data should be supplied so Belcher computes future-versus-historical change factors. Pass either an extracted historical ShiftClimate stage or an explicit reference spec such as historical_reference(1995:2014). If no suitable reference data exist, reference = NULL falls back to monthly statistics from the baseline EPW.

The same shift_recipe used to choose request variables can be passed into shift_morph(). Adjust the recipe when you want to change Belcher methods or select another registered backend:

shift_recipe <- epw_morph_recipe(
    "belcher",
    methods = c(tdb = "shift", rh = "shift"),
    options = belcher_options(snow_depth = "off")
)

shift_morph(
    extracted,
    reference = reference,
    baseline = epw,
    recipe = shift_recipe,
    strict = TRUE
)
morphed <- shift_morph(
    extracted,
    reference = reference,
    baseline = epw,
    recipe = shift_recipe,
    strict = TRUE
)

morphed

The morphed stage is still store-native. It contains hourly future weather data as Parquet artifacts, not EPW text files yet. Inspect the artifact rows when you want to see where those intermediate results live:

morph_artifacts <- shift_artifacts(morphed)
morph_artifacts[, .(kind, role, relative_path)]

Use the same shift_data() helper to preview the hourly morphed weather table. The metadata columns identify the morphing case; the weather columns are the hourly EPW-style values that will be written to the final EPW. The preview below omits long IDs to keep the table readable; include case_id in columns when you need to join rows back to a specific morphing case.

morphed_data <- shift_data(
    morphed,
    n = 24L,
    columns = c(
        "period", "year", "month", "day", "hour",
        "dry_bulb_temperature", "relative_humidity", "wind_speed"
    )
)

morphed_data

Write EPW Files

shift_epw() writes EnergyPlus Weather files from the morphed hourly results. It returns a ShiftOutputs stage. The first chunk assigns the result while hiding verbose writer output; the second prints the stage object.

For the lower-level write path and output registry, see EpwMorpher.

epws <- shift_epw(morphed)
epws

shift_outputs() lists the EPW files written by shift_epw(). These are the files you can pass directly to EnergyPlus.

outputs <- shift_outputs(epws)
outputs[, .(path, source_id, experiment_id, variant_label, period)]

shift_data(epws) reads the written EPW file back and returns its hourly weather data with output metadata attached. This is useful for confirming that the final file contains the same kind of hourly weather values you inspected in the store-native morphed Parquet step. Output metadata such as output_id, case_id, and path are available in shift_data(epws); they are omitted here so the weather values stay visible.

epw_data <- shift_data(
    epws,
    n = 24L,
    columns = c(
        "period", "year", "month", "day", "hour",
        "dry_bulb_temperature", "relative_humidity", "wind_speed"
    )
)

epw_data

Validate and Reuse the Result

At this point the workflow has produced EPW files, but there are still a few checks worth doing before using them in EnergyPlus or passing them to someone else. These checks answer three practical questions:

  • Did every stage finish?
  • If something failed or looks incomplete, where should you look first?
  • Where are the reusable files and store artifacts?

Check Stage Health

Use shift_status() when you want a compact stage-level check. It returns a single status string so it can be used in scripts, reports, or simple guards. For a successful run, the sequence should end with an EPW stage marked written.

data.table::data.table(
    stage = c("request", "collect", "extract", "reference", "morph", "epw"),
    status = c(
        shift_status(request),
        shift_status(files),
        shift_status(extracted),
        shift_status(reference),
        shift_status(morphed),
        shift_status(epws)
    )
)

Stage status and run status answer different questions. shift_status(epws) reports that the EPW artifact is written; shift_status(shift_run_get(epws)) is waiting because a store-local EPW can still be exported. Calling shift_export_epw(epws, dir) completes the run. If the store-local file is the intentional endpoint, use shift_complete(epws) instead.

Read Diagnostics When Something Looks Wrong

Diagnostics are the first place to look when a stage is blocked, failed, or returns fewer outputs than expected. An empty diagnostics table is the normal successful result. Non-empty rows are intended to explain the stage, severity, and action rather than expose internal manifest IDs first.

For common causes and the first place to look for each class of failure, see ESGF troubleshooting.

Confirm Extraction Coverage

The final EPW depends on the extracted climate table. Even after the EPW file is written, it is useful to confirm that each morphing variable has complete monthly coverage for the requested period. Missing or incomplete rows here usually mean the original ESGF query, Dataset selection, or extraction period needs to be adjusted.

Coverage problems usually originate in query selection, time filtering, or remote access. The ESGF troubleshooting article collects those checks in one place.

coverage <- shift_coverage(epws)
coverage[, .(variable_id, complete, status, output_rows, output_file_count)]

Locate Reusable Files

shift_outputs() lists the written EPW files and their case metadata. These are the files to pass to EnergyPlus, archive with a simulation run, or reopen with the package’s internal EPW reader. The store also keeps the intermediate Parquet artifacts so the workflow can be inspected or reused without repeating the remote query.

outputs <- shift_outputs(epws)
outputs[, .(
    file = basename(path),
    source_id,
    experiment_id,
    variant_label,
    period
)]

Advanced: Manifest IDs

Ordinary workflows do not need manifest IDs. They are kept available for advanced users who want to inspect the lower-level EsgStore, EpwMorpher, or manifest tables directly.