Inspect EPW Morphing

shift_morph() and shift_epw() are the recommended user-facing interface in Create Future EPW Files. They wrap EpwMorpher, the lower-level store-native morphing engine.

Use EpwMorpher directly when you need to inspect monthly summaries, preview factor calculations, diagnose missing variables, choose morphing case grouping, change the statistical downscaling backend, or rerun only the morph/write part of a workflow.

Inputs

EpwMorpher consumes completed extraction outputs in an EsgStore and a baseline EPW file. The examples below assume the store already has a future extraction plan named region and a historical reference extraction plan named reference_region.

store <- EsgStore$new("~/cmip6-singapore-store", create = FALSE)

epw <- system.file(
    "extdata", "vignettes", "future-weather",
    "SGP_Singapore.486980_IWEC.epw",
    package = "epwshiftr",
    mustWork = TRUE
)

morpher <- epw_morpher(
    store = store,
    epw = epw,
    site_id = "SIN",
    recipe = epw_morph_recipe("belcher"),
    label = "Singapore baseline"
)

The helper functions used by the high-level workflow are available directly:

epw_morph_variables("minimal")
epw_morph_variables("recommended")
epw_morph_variables("extended")

periods <- epw_morph_periods(`2060s` = 2060L)
reference_periods <- epw_morph_periods(reference = 1995L)
reference_spec <- shift_reference_historical(reference_periods)

"recommended" is the strict Belcher recipe set used by the main workflow. "minimal" is useful for relaxed demonstrations, and "extended" adds related variables for future recipes.

When you already have a recipe or backend object, ask that object for its variables instead of choosing a named set manually:

belcher_recipe <- epw_morph_recipe("belcher")
epw_morph_variables(belcher_recipe)
epw_morph_variables(epw_morph_backend("belcher"))

Recipes and Backends

A recipe selects a backend and carries backend-specific method overrides. The default recipe is "belcher", which applies the Belcher-style statistical downscaling steps used by the high-level workflow.

epw_morph_backends()

belcher <- epw_morph_backend("belcher")
belcher$label
belcher$required_variables()
belcher$methods()
belcher$rules()

The backend rules are the contract between planning and execution. Each rule declares a backend step, the EPW weather field it produces or checks, required and optional CMIP variables, method choices, and whether the step is derived from other backend outputs. preview_plan(), summarise_baseline(), and epw_morph_variables() read these rules; they do not keep a separate hard-coded list of Belcher fields.

Use method overrides when the algorithm is still Belcher but you want a different transformation for selected primary variables:

belcher_shift_recipe <- epw_morph_recipe(
    "belcher",
    methods = c(
        tdb = "shift",
        rh = "shift"
    )
)

morpher <- epw_morpher(
    store = store,
    epw = epw,
    site_id = "SIN",
    recipe = belcher_shift_recipe,
    label = "Singapore baseline"
)

The override names must be backend step names. For the built-in Belcher backend, the primary overrideable steps are tdb, rh, p, hor_ir, glob_rad, and wind; allowed values are "shift", "stretch", and "combined". Precipitation is handled by a dedicated conservative step that preserves the baseline wet-hour timing. Derived steps such as dew point, direct normal radiation, and precipitation rate are declared in the rules but are not independent method overrides.

Adding a Backend

Register a new backend when the variable requirements, output fields, or execution algorithm change. A backend is an EpwMorphBackend R6 object with a rule table and a runner function. The built-in "belcher" backend is just the first registration; it is not a special execution path.

The runner receives one canonical context:

context$epw: the baseline eplusr::Epw object;
context$climate: store-native climate rows with variable_id, time, period, year, lon, lat, units, value, and case metadata;
context$recipe: the selected recipe, including method overrides and rules;
context$by, context$case, context$strict, and context$warning.

The runner returns epw_morph_result() with complete hourly EPW weather data. EpwMorpher$run() writes that data as Parquet and adds the case metadata columns needed by write_epw().

Rule tables must include step, epw_field, method, and required. They can use either the compact scalar columns variable_id and optional_variable_id or list columns required_variables and optional_variables. Use a method_choices list column when one step supports a narrower set of method values than the backend as a whole.

This minimal backend copies the baseline EPW and offsets dry-bulb temperature. It is deliberately simple: a real backend would read context$climate, validate its required variables, and calculate weather fields from those rows.

offset_rules <- data.table::data.table(
    step = "dry",
    epw_field = "dry_bulb_temperature",
    variable_id = "tas",
    optional_variable_id = NA_character_,
    method = "plus_one",
    required = TRUE,
    derived = FALSE,
    method_choices = list(c("plus_one", "plus_two"))
)

offset_runner <- function(context, backend) {
    epw <- context$epw$clone()
    suppressMessages(epw$drop_unit())

    weather <- data.table::as.data.table(epw$data())
    amount <- switch(
        context$recipe$methods[["dry"]],
        plus_two = 2,
        plus_one = 1
    )
    weather[, dry_bulb_temperature := dry_bulb_temperature + amount]

    epw_morph_result(context, epw = epw, data = weather)
}

offset_backend <- EpwMorphBackend$new(
    name = "constant_offset",
    label = "Constant dry-bulb offset",
    methods = c(dry = "plus_one"),
    method_choices = c("plus_one", "plus_two"),
    rules = offset_rules,
    runner = offset_runner
)

epw_morph_register_backend("constant_offset", offset_backend)

offset_recipe <- epw_morph_recipe(
    "constant_offset",
    backend = "constant_offset",
    methods = c(dry = "plus_two")
)

epw_morph_variables(offset_recipe)

Use the recipe exactly like the built-in one:

morpher <- epw_morpher(
    store = store,
    epw = epw,
    site_id = "SIN",
    recipe = offset_recipe,
    label = "Singapore baseline"
)

Prefer registering a new backend name over overwriting "belcher". Use overwrite = TRUE only for interactive experiments where replacing an existing registration is intentional.

Preflight Before Writing

Preflight checks extraction coverage and baseline readiness without modifying store state.

diagnostics <- morpher$preflight(
    plan_id = region$plan_id,
    periods = periods,
    reference_plan_id = reference_region$plan_id,
    reference_periods = reference_periods,
    strict = TRUE
)

diagnostics

Blocking diagnostics should be fixed before planning. Common causes are missing required variables, incomplete monthly coverage, missing baseline EPW fields, or using periods that are not covered by the extracted climate data.

Summarise Climate and Baseline

Morphing uses monthly summary statistics. The climate summary comes from store extraction outputs; the baseline summary comes from the EPW file.

climate <- morpher$summarise_climate(
    plan_id = region$plan_id,
    periods = periods,
    strict = TRUE,
    overwrite = FALSE
)

reference_climate <- morpher$summarise_climate(
    plan_id = reference_region$plan_id,
    periods = reference_periods,
    strict = TRUE,
    overwrite = FALSE
)

baseline <- morpher$summarise_baseline(overwrite = FALSE)

unique(climate$summary_id)
unique(reference_climate$summary_id)
unique(baseline$baseline_id)

The summary IDs are stable for the selected extraction plans, periods, baseline, and recipe. Reusing them lets you preview or rerun later stages without re-extracting climate data.

Preview and Create a Morphing Plan

preview_plan() calculates plan rows, factor rows, and diagnostics without writing them. plan() persists the selected plan and factor rows.

preview <- morpher$preview_plan(
    summary_id = unique(climate$summary_id),
    reference_summary_id = unique(reference_climate$summary_id),
    baseline_id = unique(baseline$baseline_id),
    by = c("source_id", "experiment_id", "variant_label", "period"),
    strict = TRUE
)

preview$plan
preview$factors
preview$diagnostics

plan <- morpher$plan(
    summary_id = unique(climate$summary_id),
    reference_summary_id = unique(reference_climate$summary_id),
    baseline_id = unique(baseline$baseline_id),
    by = c("source_id", "experiment_id", "variant_label", "period"),
    strict = TRUE
)

The by columns define morphing cases. The default creates one case per model, experiment, member, and period. Add or remove grouping columns only when those columns exist in the climate summary and reflect the cases you intend to write.

Diagnose and Run

Run diagnostics before executing strict plans.

morph_id <- plan$morph_id[[1L]]

morpher$diagnose(morph_id)
morpher$check(morph_id)

results <- morpher$run(
    morph_id,
    overwrite = FALSE,
    resume = TRUE
)

run() writes hourly morphed weather as Parquet artifacts under the store. It does not write EPW text files yet.

Write EPW Files

Write EnergyPlus Weather files after morphing succeeds.

outputs <- morpher$write_epw(
    morph_id,
    dir = "outputs/future-epw",
    separate = TRUE,
    overwrite = FALSE,
    resume = TRUE
)

morpher$status(morph_id)
morpher$outputs(morph_id)

The high-level shift_epw() function wraps this step and returns a ShiftOutputs stage. Use shift_outputs() and shift_data() when you do not need lower-level manifest IDs.

One-Call Workflow

For scripts that need the lower-level engine but not every intermediate object, use workflow().

result <- morpher$workflow(
    plan_id = region$plan_id,
    periods = periods,
    reference_plan_id = reference_region$plan_id,
    reference_periods = reference_periods,
    by = c("source_id", "experiment_id", "variant_label", "period"),
    strict = TRUE,
    dir = "outputs/future-epw",
    overwrite = FALSE,
    resume = TRUE
)

names(result)

For most users, the equivalent high-level path is still clearer:

epws <- files |>
    shift_extract(site = site, periods = periods) |>
    shift_morph(
        baseline = epw,
        reference = shift_reference_historical(reference_periods),
        recipe = epw_morph_recipe("belcher"),
        strict = TRUE
    ) |>
    shift_epw()

Relationship to Other Layers

ESG stores owns extraction results and artifacts consumed by EpwMorpher.
ESGF troubleshooting explains incomplete coverage and missing variable diagnostics.
Create Future EPW Files shows the recommended high-level path.

Hongyuan Jia