ESGF Query results for Aggregation type

EsgResultAggregation is a class that represents query results for Aggregation type from ESGF search RESTful API.

In general, there is no need to create an EsgResultAggregation manually. Usually, it is created by calling EsgResultDataset$collect().

Author

Hongyuan Jia

Super class

epwshiftr::EsgResult -> EsgResultAggregation

Active bindings

url_opendap

A character vector of the OPeNDAP URLs of the files.

url_download

A character vector of the download URLs of the files.

fields

A character vector indicating all response fields, followed by derived URL fields when their source fields are available.

Methods

Public methods

• EsgResultAggregation$to_data_table()

• EsgResultAggregation$filter_time()

• EsgResultAggregation$download_plan()

• EsgResultAggregation$repair_urls()

• EsgResultAggregation$download()

• EsgResultAggregation$print()

• EsgResultAggregation$open_dataset()

• EsgResultAggregation$clone()

Inherited methods

• epwshiftr::EsgResult$count()
• epwshiftr::EsgResult$filter()
• epwshiftr::EsgResult$initialize()
• epwshiftr::EsgResult$load()
• epwshiftr::EsgResult$query_url()
• epwshiftr::EsgResult$reachable()
• epwshiftr::EsgResult$save()
• epwshiftr::EsgResult$selection()
• epwshiftr::EsgResult$slice()
• epwshiftr::EsgResult$to_dt()

---

Method to_data_table()

Convert the results into a data.table

Usage

EsgResultAggregation$to_data_table(fields = NULL, formatted = FALSE)

Arguments

fields

A non-empty character vector indicating the fields to put into the data.table. If NULL, all fields in the query result will be used. Default: NULL.

formatted

Whether to use formatted values for special fields, including url and size. Default: FALSE.

Returns

A data.table.

---

Method filter_time()

Filter aggregation records by the time range covered by each file

See EsgResultFile$filter_time() for the method semantics.

Usage

EsgResultAggregation$filter_time(start, stop, method = c("drs", "opendap"))

Arguments

start, stop

Time range boundaries. Character, Date, and POSIXt inputs are accepted and parsed in UTC.

method

How to determine file time ranges. One of "drs" or "opendap". Default: "drs".

Returns

A new EsgResultAggregation object.

---

Method download_plan()

Build a persistent downloader plan for aggregation records.

Usage

EsgResultAggregation$download_plan(
  replica = c("current", "auto"),
  service = "HTTPServer",
  probe = TRUE,
  strategy = c("fastest", "first", "stable"),
  all = TRUE,
  node_stats = NULL,
  network_policy = NULL,
  node_policy = NULL,
  probe_concurrency = 1L,
  probe_cache_seconds = 3600L
)

Arguments

replica

Replica policy. Aggregation records currently use the current records.

service

ESGF URL service to download from. Default: "HTTPServer".

probe

Whether to lightly probe URLs before ranking them.

strategy

Candidate ranking strategy.

all

Reserved for API symmetry with EsgResultFile.

node_stats

Optional data node history from Downloader$data_nodes().

network_policy

Optional network options from Downloader$network_policy.

node_policy

Optional data-node cooldown policy from Downloader$node_policy.

probe_concurrency

Maximum concurrent URL probes. Default: 1.

probe_cache_seconds

Seconds to reuse fresh data-node probe history before probing a URL again. Default: 3600.

Returns

A data.table download plan.

---

Method repair_urls()

Replace unreachable service URLs with reachable replica records.

$repair_urls() probes the current records for the selected service, queries ESGF replicas by master_id for records whose URL is missing or unreachable, probes candidate replica URLs, and returns a new result with repaired records in the original row order. The original result is not modified. Aggregation results are not downloadable as files; repaired aggregation URLs are intended for service access such as OPeNDAP. The returned result records the original query URL plus the replica lookup query URL in $query_url("all").

Usage

EsgResultAggregation$repair_urls(
  service = c("OPENDAP", "HTTPServer"),
  index_node = NULL,
  probe = NULL
)

Arguments

service

Service URL to repair. One of "OPENDAP" or "HTTPServer". Default: "OPENDAP".

index_node

Optional ESGF search index node used to look up replicas. If NULL, the current result index node is used.

probe

Optional named list of probe settings. Supported fields are level, timeout, concurrency, network_policy, cache_seconds, and cache_failures_seconds. Default level is "data_node".

Returns

A new EsgResultAggregation object.

---

Method download()

Enqueue and run aggregation downloads using a Downloader.

Usage

EsgResultAggregation$download(
  downloader = NULL,
  store = NULL,
  replica = c("current", "auto"),
  service = "HTTPServer",
  probe = TRUE,
  strategy = c("fastest", "first", "stable"),
  probe_concurrency = NULL,
  probe_cache_seconds = 3600L,
  session_label = NULL,
  run = TRUE,
  ...
)

Arguments

downloader

Optional persistent Downloader. If NULL, store$downloader() is used when store is supplied.

store

Optional EsgStore providing a bound downloader.

replica

Replica policy. Aggregation records currently use the current records.

service

ESGF URL service to download from. Default: "HTTPServer".

probe

Whether to lightly probe URLs before ranking them.

strategy

Candidate ranking strategy.

probe_concurrency

Maximum concurrent URL probes when probe = TRUE. Default comes from the downloader worker count.

probe_cache_seconds

Seconds to reuse fresh data-node probe history before probing a URL again. Default: 3600.

session_label

Optional download session label.

run

Whether to run the queued session immediately. Default: TRUE.

...

Additional arguments passed to Downloader$run().

Returns

The created downloader session ID.

---

Method print()

Print a summary of the current dataset

Usage

EsgResultAggregation$print(n = 10L)

Arguments

n

An integer indicating how many items to print. If NULL, all items will be printed. Default: 10L.

Returns

The EsgResultAggregation object itself, invisibly.

---

Method open_dataset()

Open aggregation files as an EsgDataset for remote data access via OPeNDAP

Usage

EsgResultAggregation$open_dataset(
  which = NULL,
  fallback = c("ask", "auto", "error"),
  store = NULL,
  downloader = NULL,
  progress = getOption("epwshiftr.progress", interactive())
)

Arguments

which

Aggregation records to open. Use integer indices or aggregation record IDs. If NULL, all aggregation records are opened. Default: NULL.

fallback

What to do if OPeNDAP is unavailable. One of:

• "ask": Interactively ask the user (default). In a non-interactive session this raises an error.

• "auto": Automatically download files via HTTPServer.

• "error": Raise an error.

store

Optional EsgStore used for recoverable HTTP fallback.

downloader

Optional persistent Downloader used for recoverable HTTP fallback.

progress

Whether to show progress bars while opening OPeNDAP records and any HTTP fallback downloads. By default the package option epwshiftr.progress is used, falling back to interactive().

Returns

An EsgDataset object with the connection already opened.

---

Method clone()

The objects of this class are cloneable with this method.

Usage

EsgResultAggregation$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.