Query CMIP6 data using ESGF search RESTful API

The Earth System Grid Federation (ESGF) is an international collaboration for the software that powers most global climate change research, notably assessments by the Intergovernmental Panel on Climate Change (IPCC).

The ESGF search service exposes RESTful APIs that can be used by clients to query the contents of the underlying search index, and return results matching the given constraints. The documentation of the APIs can be found using this link

EsgfQuery is the workhorse for dealing with ESGF search services.

query_esgf(host = "https://esgf-node.llnl.gov/esg-search")

Arguments

host: The URL to the ESGF Search API service. This should be the URL of the ESGF search service excluding the final endpoint name. Usually this is http://<hostname>/esg-search. Default is set to the LLNL (Lawrence Livermore National Laboratory) Index Node, which is "https://esgf-node.llnl.gov/esg-search".

`EsgfQuery` object

query_esgf() returns an EsgfQuery object, which is an R6 object with quite a few methods that can be classified into 3 categories:

Value listing: methods to list all possible values of facets, shards, etc.
Parameter getter & setter: methods to get the query parameter values or set them before sending the actual query to the ESGF search services.
Query responses: methods to collect results for the query response.

Value listing

When creating an EsgfQuery object, a facet listing query is sent to the index node to get all available facets and shards for the default project (CMIP6). EsgfQuery object provides three value-listing methods to extract data from the response of the facet listing query:

EsgfQuery$list_all_facets(): List all available facet names.
EsgfQuery$list_all_shards(): List all available shards.
EsgfQuery$list_all_values(): List all available values of a specific facet.

Parameter getter & setter

The ESGF search services support a lot of parameters. The EsgfQuery contains dedicated methods to set values for most of them, including:

Most common keywords: facets, offset, limit, fields, type, replica, latest, distrib and shards.
Most common facets: project, activity_id, experiment_id, source_id, variable_id, frequency, variant_label, nominal_resolution and data_node.

All methods act in a similar way:

If input is given, the corresponding parameter is set and the updated EsgfQuery object is returned.
- This makes it possible to chain different parameter setters, e.g. EsgfQuery$project("CMIP6")$frequency("day")$limit(1) sets the parameter project, frequency and limit sequentially.
- For parameters that want character inputs, you can put a preceding ! to negate the constraints, e.g. EsgfQuery$project(!"CMIP6") searches for all projects except for CMIP6.
If no input is given, the current parameter value is returned. For example, directly calling EsgfQuery$project() returns the current value of the project parameter. The returned value can be two types:
- NULL, i.e. there is no constraint on the corresponding parameter
- An EsgfQueryParam object which is essentially a list of three elements:
  - value: The input values
  - negate: Whether there is a preceding ! in the input
  - name: The parameter name

Despite methods for specific keywords and facets, you can specify arbitrary query parameters using EsgfQuery$params() method. For details on the usage, please see the documentation.

Query responses

The query is not sent unless related methods are called:

EsgfQuery$count(): Count the total number of records that match the query.
- You can return only the total number of matched record by calling EsgfQuery$count(facets = FALSE)
- You can also count the matched records for specified facets, e.g. EsgfQuery$count(facets = c("source_id", "activity_id"))
EsgfQuery$collect(): Collect the query results and format it into a data.table

Other helpers

EsgfQuery object also provide several other helper functions:

EsgfQuery$build_cache(): By default, EsgfQuery$build_cache() is called when initialize a new EsgfQuery object. So in general, there is no need to call this separately. Basically, EsgfQuery$build_cahce() sends a facet listing query to the index node and stores the response internally. The response contains all available facets and shards and is used as a source for validating user input for parameter setters.
EsgfQuery$url(): Returns the actual query URL or the wget script URL which can be used to download all files matching the given constraints..
EsgfQuery$response(): Returns the actual response of EsgfQuery$count() and EsgfQuery$collect(). It is a named list generated from the JSON response using jsonlite::fromJSON().
EsgfQuery$print(): Print a summary of the current EsgfQuery object including the host URL, the built time of facet cache and all query parameters.

Author

Hongyuan Jia

Methods

Method `new()`

Create a new EsgfQuery object

When initialization, a facet listing query is sent to the index node to get all available facets and shards. This information will be used to validate inputs for activity_id, scource_id facets and etc.

Usage

EsgfQuery$new(host = "https://esgf-node.llnl.gov/esg-search")

Arguments

host: The URL to the ESGF Search API service. This should be the URL of the ESGF search service excluding the final endpoint name. Usually this is http://<hostname>/esg-search. Default is to ses the LLNL (Lawrence Livermore National Laboratory) Index Node, which is "https://esgf-node.llnl.gov/esg-search".

Returns

An EsgfQuery object.

Examples

\dontrun{
q <- EsgfQuery$new(host = "https://esgf-node.llnl.gov/esg-search")
q
}

Method `build_cache()`

Build facet cache used for input validation

A facet cache is data that is fetched using a facet listing query to the index node. It contains all available facets and shards that can be used as parameter values within a specific project.

By default, $build_cache() is called when initialize a new EsgfQuery object for the default project (CMIP6). So in general, there is no need to call this method, unless that you want to rebuild the cache again with different projects after calling $project().

Usage

EsgfQuery$build_cache()

Returns

The modified EsgfQuery object.

Examples

\dontrun{
q$build_cache()
}

Method `list_all_facets()`

List all available facet names

Usage

EsgfQuery$list_all_facets()

Returns

A character vector.

Examples

\dontrun{
q$list_all_facets()
}

Method `list_all_shards()`

List all available shards

Usage

EsgfQuery$list_all_shards()

Returns

A character vector.

Examples

\dontrun{
q$list_all_shards()
}

Method `list_all_values()`

List all available values of a specific facet

Usage

EsgfQuery$list_all_values(facet)

Arguments

facet: A single string giving the facet name.

Returns

A named character vector.

Examples

\dontrun{
q$list_all_values()
}

Method `project()`

Get or set the project facet parameter.

Usage

EsgfQuery$project(value = "CMIP6")

Arguments

value

The parameter value. Default: "CMIP6". There are two options:

If value is not given, current value is returned.
A character vector or NULL. Note that you can put a preceding ! to negate the facet constraints. For example, $project(!c("CMIP5", "CMIP6")) searches for all projects except for CMIP5 and CMIP6.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$project()

# set the parameter
q$project("CMIP6")

# negate the project constraints
q$project(!"CMIP6")

# remove the parameter
q$project(NULL)
}

Method `activity_id()`

Get or set the activity_id facet parameter.

Usage

EsgfQuery$activity_id(value)

Arguments

value

The parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A character vector or NULL. Note that you can put a preceding ! to negate the facet constraints. For example, $activity_id(!c("C4MIP", "GeoMIP")) searches for all activity_ids except for C4MIP and GeoMIP.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$activity_id()

# set the parameter
q$activity_id("ScenarioMIP")

# negate the constraints
q$activity_id(!c("CFMIP", "ScenarioMIP"))

# remove the parameter
q$activity_id(NULL)
}

Method `experiment_id()`

Get or set the experiment_id facet parameter.

Usage

EsgfQuery$experiment_id(value)

Arguments

value

The parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A character vector or NULL. Note that you can put a preceding ! to negate the facet constraints. For example, $experiment_id(!c("ssp126", "ssp245")) searches for all experiment_ids except for ssp126 and ssp245.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$experiment_id()

# set the parameter
q$experiment_id(c("ssp126", "ssp585"))

# negate the constraints
q$experiment_id(!c("ssp126", "ssp585"))

# remove the parameter
q$experiment_id(NULL)
}

Method `source_id()`

Get or set the source_id facet parameter.

Usage

EsgfQuery$source_id(value)

Arguments

value

The parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A character vector or NULL. Note that you can put a preceding ! to negate the facet constraints. For example, $source_id(!c("CESM2", "CESM2-FV2")) searches for all source_ids except for CESM2 and CESM2-FV2.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$source_id()

# set the parameter
q$source_id(c("BCC-CSM2-MR", "CESM2"))

# negate the constraints
q$source_id(!c("BCC-CSM2-MR", "CESM2"))

# remove the parameter
q$source_id(NULL)
}

Method `variable_id()`

Get or set the variable_id facet parameter.

Usage

EsgfQuery$variable_id(value)

Arguments

value

The parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A character vector or NULL. Note that you can put a preceding ! to negate the facet constraints. For example, $variable_id(!c("tas", "pr")) searches for all variable_ids except for tas and pr.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$variable_id()

# set the parameter
q$variable_id(c("tas", "pr"))

# negate the constraints
q$variable_id(!c("tas", "pr"))

# remove the parameter
q$variable_id(NULL)
}

Method `frequency()`

Get or set the frequency facet parameter.

Usage

EsgfQuery$frequency(value)

Arguments

value

The parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A character vector or NULL. Note that you can put a preceding ! to negate the facet constraints. For example, $frequency(!c("day", "mon")) searches for all frequencys except for day and mon.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$frequency()

# set the parameter
q$frequency(c("1hr", "day"))

# negate the constraints
q$frequency(!c("1hr", "day"))

# remove the parameter
q$frequency(NULL)
}

Method `variant_label()`

Get or set the variant_label facet parameter.

Usage

EsgfQuery$variant_label(value)

Arguments

value

The parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A character vector or NULL. Note that you can put a preceding ! to negate the facet constraints. For example, $variant_label(!c("r1i1p1f1", "r2i1p1f1")) searches for all variant_labels except for r1i1p1f1 and r2i1p1f1.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$variant_label()

# set the parameter
q$variant_label(c("r1i1p1f1", "r1i2p1f1"))

# negate the constraints
q$variant_label(!c("r1i1p1f1", "r1i2p1f1"))

# remove the parameter
q$variant_label(NULL)
}

Method `nominal_resolution()`

Get or set the nominal_resolution facet parameter.

Usage

EsgfQuery$nominal_resolution(value)

Arguments

value

The parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A character vector or NULL. Note that you can put a preceding ! to negate the facet constraints. For example, $nominal_resolution(!c("50 km", "1x1 degree")) searches for all nominal_resolutions except for 50 km and 1x1 degree.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$nominal_resolution()

# set the parameter
q$nominal_resolution(c("100 km", "1x1 degree"))

# negate the constraints
q$nominal_resolution(!c("100 km", "1x1 degree"))

# remove the parameter
q$nominal_resolution(NULL)
}

Method `data_node()`

Get or set the data_node parameter.

Usage

EsgfQuery$data_node(value)

Arguments

value

The parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A character vector or NULL. Note that you can put a preceding ! to negate the facet constraints. For example, $data_node(!c("cmip.bcc.cma.cn", "esg.camscma.cn")) searches for all data_nodes except for cmip.bcc.cma.cn and esg.camscma.cn.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$data_node()

# set the parameter
q$data_node("esg.lasg.ac.cn")

# negate the constraints
q$data_node(!"esg.lasg.ac.cn")

# remove the parameter
q$data_node(NULL)
}

Method `facets()`

Get or set the facets parameter for facet counting query.

Note that $facets() only affects $count() method when sending a query of facet counting.

Usage

EsgfQuery$facets(value)

Arguments

value

The facet parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A character vector or NULL. The special notation "*" can be used to indicate that all available facets should be considered.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$facets()

# set the facets
q$facets(c("activity_id", "source_id"))

# use all available facets
q$facets("*")
}

Method `fields()`

Get or set the fields parameter.

By default, all available metadata fields are returned for each query. $facets() can be used to limit the number of fields returned in the query response.

Usage

EsgfQuery$fields(value = "*")

Arguments

value

The facet parameter value. Default: "*". There are two options:

If value is not given, current value is returned.
A character vector or NULL. The special notation "*" can be used to indicate that all available fields should be considered.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$fields()

# set the fields
q$fields(c("activity_id", "source_id"))

# use all available fields
q$fields("*")

# remove the parameter
# act the same as above because the default `fields` in ESGF search
# services is `*` if `fields` is not specified
q$fields(NULL)
}

Method `shards()`

Get or set the shards parameter.

By default, a distributed query targets all ESGF Nodes. $shards() can be used to execute a distributed search that targets only one or more specific nodes.

All available shards can be retrieved using $list_all_shards() method.

Usage

EsgfQuery$shards(value)

Arguments

value

The facet parameter value. There are two options:

If value is not given, current value is returned.
A character vector or NULL.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$shards()

# set the parameter
q$shards("localhost:8983/solr/datasets")

# negate the constraints
q$shards(!"localhost:8983/solr/datasets")

# only applicable for distributed queries
q$distrib(FALSE)$shards("localhost:8983/solr/datasets") # Error

# remove the parameter
q$shards(NULL)
}

Method `replica()`

Get or set the replica parameter.

By default, a query returns all records (masters and replicas) matching the search criteria, i.e. $replica(NULL). To return only master records, use $replica(FALSE); to return only replicas, use $replica(TRUE).

Usage

EsgfQuery$replica(value)

Arguments

value

The facet parameter value. Default: NULL. There are two options:

If value is not given, current value is returned.
A flag or NULL.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$replica()

# set the parameter
q$replica(TRUE)

# remove the parameter
q$replica(NULL)
}

Method `latest()`

Get or set the latest parameter.

By default, a query to the ESGF search services returns only the very last, up-to-date version of the matching records, i.e. $latest(TRUE). You can use $latest(FALSE) to return all versions.

Usage

EsgfQuery$latest(value = TRUE)

Arguments

value

The facet parameter value. Default: TRUE. There are two options:

If value is not given, current value is returned.
A flag.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$latest()

# set the parameter
q$latest(TRUE)
}

Method `type()`

Get or set the type parameter.

There are three types in total: Dataset, File or Aggregation.

Usage

EsgfQuery$type(value = "Dataset")

Arguments

value

The facet parameter value. Default: "Dataset". There are two options:

If value is not given, current value is returned.
A string.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$type()

# set the parameter
q$type("Dataset")
}

Method `limit()`

Get or set the limit parameter.

$limit() can be used to limit the number of records to return. Note that the maximum number of records to return per query for ESGF search services is 10,000. A warning is issued if input value is greater than that. In this case, limit will be reset to 10,000.

Usage

EsgfQuery$limit(value = 10L)

Arguments

value

The facet parameter value. Default: 10. There are two options:

If value is not given, current value is returned.
An integer.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$limit()

# set the parameter
q$limit(10L)

# `limit` is reset to 10,000 if input is greater than that
q$limit(10000L) # warning
}

Method `offset()`

Get or set the offset parameter.

If the query returns records that exceed the limit number, $offset() can be used to paginate through the available results.

Usage

EsgfQuery$offset(value = 0L)

Arguments

value

The facet parameter value. Default: 0. There are two options:

If value is not given, current value is returned.
An integer.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$offset()

# set the parameter
q$offset(0L)
}

Method `distrib()`

Get or set the distrib facet

By default, the query is sent to all ESGF Nodes, i.e. $distrib(TRUE). $distrib(FALSE) can be used to execute the query only on the target node.

Usage

EsgfQuery$distrib(value = TRUE)

Arguments

value

The facet parameter value. Default: TRUE. There are two options:

If value is not given, current value is returned.
A flag.

Returns

If value is given, the modified EsgfQuery object.
Otherwise, an EsgfQueryParam object which is essentially a list of three elements:
- value: input values.
- negate: Whether there is a preceding !.
- name: Parameter name.

Examples

\dontrun{
# get current value
q$distrib()

# set the parameter
q$distrib(TRUE)
}

Method `params()`

Get or set other parameters.

$params() can be used to specify other parameters that do not have a dedicated method, e.g. version, master_id, etc. It can also be used to overwrite existing parameter values specified using methods like $activity_id().

Usage

EsgfQuery$params(...)

Arguments

...

Parameter values to set. There are three options:

If not given, existing parameters that do not have a dedicated method are returned.
If NULL, all existing parameters that do not have a dedicated method are removed.
A named vector, e.g. $params(score = 1, table_id = "day") will set score to 1 and table_id to day. The ! notation can still be used to negate the constraints, e.g. $params(table_id = !c("3hr", "day")) searches for all table_id except for 3hr and day.

Returns

If parameters are specified, the modified EsgfQuery object, invisibly.
Otherwise, an empty list for $params(NULL) or a list of EsgfQueryParam objects.

Examples

\dontrun{
# get current values
# default is an empty list (`list()`)
q$params()

# set the parameter
q$params(table_id = c("3hr", "day"), member_id = "00")
q$params()

# reset existing parameters
q$frequency("day")
q$params(frequency = "mon")
q$frequency() # frequency value has been changed using $params()

# negating the constraints is also supported
q$params(table_id = !c("3hr", "day"))

# use NULL to remove all parameters
q$params(NULL)$params()
}

Method `url()`

Get the URL of actual query or wget script

Usage

EsgfQuery$url(wget = FALSE)

Arguments

wget: Whether to return the URL of the wget script that can be used to download all files matching the given constraints. Default: FALSE.

Returns

A single string.

Examples

\dontrun{
q$url()

# get the wget script URL
q$url(wget = TRUE)

# You can download the wget script using the URL directly. For
# example, the code below downloads the script and save it as
# 'wget.sh' in R's temporary folder:
download.file(q$url(TRUE), file.path(tempdir(), "wget.sh"), mode = "wb")

}

Method `count()`

Send a query of facet counting and fetch the results

Usage

EsgfQuery$count(facets = TRUE)

Arguments

facets

NULL, a flag or a character vector. There are three options:

If NULL or FALSE, only the total number of matched records is returned.
If TRUE, the value of $facets() is used to limit the facets. This is the default value.
If a character vector, it is used to limit the facets.

Returns

If facets equals NULL or FALSE, or $facets() returns NULL, an integer.
Otherwise, a named list with the first element always being total which is the total number of matched records. Other elements have the same length as input facets and are all named integer vectors.

Examples

\dontrun{
# get the total number of matched records
q$count(NULL) # or q$count(facets = FALSE)

# count records for specific facets
q$facets(c("activity_id", "source_id"))$count()

# same as above
q$count(facets = c("activity_id", "source_id"))
}

Method `collect()`

Send the actual query and fetch the results

$collect() sends the actual query to the ESGF search services and returns the results in a data.table::data.table. The columns depend on the value of query type and fields parameter.

Usage

EsgfQuery$collect()

Returns

A data.table.

Examples

\dontrun{
q$fields("source_id")
q$collect()
}

Method `response()`

Get the response of last sent query

The response of the last sent query is always stored internally and can be retrieved using $response(). It is a named list generated from the JSON response using jsonlite::fromJSON().

Usage

EsgfQuery$response()

Returns

A named list.

Examples

\dontrun{
q$response()
}

Method `print()`

Print a summary of the current EsgfQuery object

$print() gives the summary of current EsgfQuery object including the host URL, the built time of facet cache and all query parameters.

Usage

EsgfQuery$print()

Returns

The EsgfQuery object itself, invisibly.

Examples

\dontrun{
q$print()
}

Examples


## ------------------------------------------------
## Method `EsgfQuery$new`
## ------------------------------------------------

if (FALSE) {
q <- EsgfQuery$new(host = "https://esgf-node.llnl.gov/esg-search")
q
}

## ------------------------------------------------
## Method `EsgfQuery$build_cache`
## ------------------------------------------------

if (FALSE) {
q$build_cache()
}

## ------------------------------------------------
## Method `EsgfQuery$list_all_facets`
## ------------------------------------------------

if (FALSE) {
q$list_all_facets()
}

## ------------------------------------------------
## Method `EsgfQuery$list_all_shards`
## ------------------------------------------------

if (FALSE) {
q$list_all_shards()
}

## ------------------------------------------------
## Method `EsgfQuery$list_all_values`
## ------------------------------------------------

if (FALSE) {
q$list_all_values()
}

## ------------------------------------------------
## Method `EsgfQuery$project`
## ------------------------------------------------

if (FALSE) {
# get current value
q$project()

# set the parameter
q$project("CMIP6")

# negate the project constraints
q$project(!"CMIP6")

# remove the parameter
q$project(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$activity_id`
## ------------------------------------------------

if (FALSE) {
# get current value
q$activity_id()

# set the parameter
q$activity_id("ScenarioMIP")

# negate the constraints
q$activity_id(!c("CFMIP", "ScenarioMIP"))

# remove the parameter
q$activity_id(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$experiment_id`
## ------------------------------------------------

if (FALSE) {
# get current value
q$experiment_id()

# set the parameter
q$experiment_id(c("ssp126", "ssp585"))

# negate the constraints
q$experiment_id(!c("ssp126", "ssp585"))

# remove the parameter
q$experiment_id(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$source_id`
## ------------------------------------------------

if (FALSE) {
# get current value
q$source_id()

# set the parameter
q$source_id(c("BCC-CSM2-MR", "CESM2"))

# negate the constraints
q$source_id(!c("BCC-CSM2-MR", "CESM2"))

# remove the parameter
q$source_id(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$variable_id`
## ------------------------------------------------

if (FALSE) {
# get current value
q$variable_id()

# set the parameter
q$variable_id(c("tas", "pr"))

# negate the constraints
q$variable_id(!c("tas", "pr"))

# remove the parameter
q$variable_id(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$frequency`
## ------------------------------------------------

if (FALSE) {
# get current value
q$frequency()

# set the parameter
q$frequency(c("1hr", "day"))

# negate the constraints
q$frequency(!c("1hr", "day"))

# remove the parameter
q$frequency(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$variant_label`
## ------------------------------------------------

if (FALSE) {
# get current value
q$variant_label()

# set the parameter
q$variant_label(c("r1i1p1f1", "r1i2p1f1"))

# negate the constraints
q$variant_label(!c("r1i1p1f1", "r1i2p1f1"))

# remove the parameter
q$variant_label(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$nominal_resolution`
## ------------------------------------------------

if (FALSE) {
# get current value
q$nominal_resolution()

# set the parameter
q$nominal_resolution(c("100 km", "1x1 degree"))

# negate the constraints
q$nominal_resolution(!c("100 km", "1x1 degree"))

# remove the parameter
q$nominal_resolution(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$data_node`
## ------------------------------------------------

if (FALSE) {
# get current value
q$data_node()

# set the parameter
q$data_node("esg.lasg.ac.cn")

# negate the constraints
q$data_node(!"esg.lasg.ac.cn")

# remove the parameter
q$data_node(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$facets`
## ------------------------------------------------

if (FALSE) {
# get current value
q$facets()

# set the facets
q$facets(c("activity_id", "source_id"))

# use all available facets
q$facets("*")
}

## ------------------------------------------------
## Method `EsgfQuery$fields`
## ------------------------------------------------

if (FALSE) {
# get current value
q$fields()

# set the fields
q$fields(c("activity_id", "source_id"))

# use all available fields
q$fields("*")

# remove the parameter
# act the same as above because the default `fields` in ESGF search
# services is `*` if `fields` is not specified
q$fields(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$shards`
## ------------------------------------------------

if (FALSE) {
# get current value
q$shards()

# set the parameter
q$shards("localhost:8983/solr/datasets")

# negate the constraints
q$shards(!"localhost:8983/solr/datasets")

# only applicable for distributed queries
q$distrib(FALSE)$shards("localhost:8983/solr/datasets") # Error

# remove the parameter
q$shards(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$replica`
## ------------------------------------------------

if (FALSE) {
# get current value
q$replica()

# set the parameter
q$replica(TRUE)

# remove the parameter
q$replica(NULL)
}

## ------------------------------------------------
## Method `EsgfQuery$latest`
## ------------------------------------------------

if (FALSE) {
# get current value
q$latest()

# set the parameter
q$latest(TRUE)
}

## ------------------------------------------------
## Method `EsgfQuery$type`
## ------------------------------------------------

if (FALSE) {
# get current value
q$type()

# set the parameter
q$type("Dataset")
}

## ------------------------------------------------
## Method `EsgfQuery$limit`
## ------------------------------------------------

if (FALSE) {
# get current value
q$limit()

# set the parameter
q$limit(10L)

# `limit` is reset to 10,000 if input is greater than that
q$limit(10000L) # warning
}

## ------------------------------------------------
## Method `EsgfQuery$offset`
## ------------------------------------------------

if (FALSE) {
# get current value
q$offset()

# set the parameter
q$offset(0L)
}

## ------------------------------------------------
## Method `EsgfQuery$distrib`
## ------------------------------------------------

if (FALSE) {
# get current value
q$distrib()

# set the parameter
q$distrib(TRUE)
}

## ------------------------------------------------
## Method `EsgfQuery$params`
## ------------------------------------------------

if (FALSE) {
# get current values
# default is an empty list (`list()`)
q$params()

# set the parameter
q$params(table_id = c("3hr", "day"), member_id = "00")
q$params()

# reset existing parameters
q$frequency("day")
q$params(frequency = "mon")
q$frequency() # frequency value has been changed using $params()

# negating the constraints is also supported
q$params(table_id = !c("3hr", "day"))

# use NULL to remove all parameters
q$params(NULL)$params()
}

## ------------------------------------------------
## Method `EsgfQuery$url`
## ------------------------------------------------

if (FALSE) {
q$url()

# get the wget script URL
q$url(wget = TRUE)

# You can download the wget script using the URL directly. For
# example, the code below downloads the script and save it as
# 'wget.sh' in R's temporary folder:
download.file(q$url(TRUE), file.path(tempdir(), "wget.sh"), mode = "wb")

}

## ------------------------------------------------
## Method `EsgfQuery$count`
## ------------------------------------------------

if (FALSE) {
# get the total number of matched records
q$count(NULL) # or q$count(facets = FALSE)

# count records for specific facets
q$facets(c("activity_id", "source_id"))$count()

# same as above
q$count(facets = c("activity_id", "source_id"))
}

## ------------------------------------------------
## Method `EsgfQuery$collect`
## ------------------------------------------------

if (FALSE) {
q$fields("source_id")
q$collect()
}

## ------------------------------------------------
## Method `EsgfQuery$response`
## ------------------------------------------------

if (FALSE) {
q$response()
}

## ------------------------------------------------
## Method `EsgfQuery$print`
## ------------------------------------------------

if (FALSE) {
q$print()
}

Query CMIP6 data using ESGF search RESTful API

Arguments

EsgfQuery object

Value listing

Parameter getter & setter

Query responses

Other helpers

Author

Methods

Public methods

Method new()

Usage

Arguments

Returns

Examples

Method build_cache()

Usage

Returns

Examples

Method list_all_facets()

Usage

Returns

Examples

Method list_all_shards()

Usage

Returns

Examples

Method list_all_values()

Usage

Arguments

Returns

Examples

Method project()

Usage

Arguments

Returns

Examples

Method activity_id()

Usage

Arguments

Returns

Examples

Method experiment_id()

Usage

Arguments

Returns

Examples

Method source_id()

Usage

Arguments

Returns

Examples

Method variable_id()

Usage

Arguments

Returns

Examples

Method frequency()

Usage

Arguments

Returns

Examples

Method variant_label()

Usage

Arguments

Returns

Examples

Method nominal_resolution()

Usage

Arguments

Returns

Examples

Method data_node()

Usage

Arguments

Returns

Examples

Method facets()

Usage

Arguments

`EsgfQuery` object

Method `new()`

Method `build_cache()`

Method `list_all_facets()`

Method `list_all_shards()`

Method `list_all_values()`

Method `project()`

Method `activity_id()`

Method `experiment_id()`

Method `source_id()`

Method `variable_id()`

Method `frequency()`

Method `variant_label()`

Method `nominal_resolution()`

Method `data_node()`

Method `facets()`

Method `fields()`

Method `shards()`

Method `replica()`

Method `latest()`

Method `type()`

Method `limit()`

Method `offset()`

Method `distrib()`

Method `params()`

Method `url()`

Method `count()`

Method `collect()`

Method `response()`

Method `print()`