Skip to content

Dataset Config

TOML Example (Click to Expand) 
[dataset]
  name = "dsg_rpm_2_v2.1"
  dataset_id = "load-profiles"
  version = "2.1"
  version_status = "Active"
  data_format = "parquet"
  schema_info = "columns: scenario_year, timestamp, rs_number, rpm_bus_num, total_electricity_kwh"
  weather_years = [2012]
  model_years = [2020, 2025, 2030, 2035, 2040, 2045]
  units = ["kWh"]
  scenarios = ["demand_high", "demand_moderate", "demand_stress"]
  sensitivities = []
  relevant_links = ["https://testlink1.com"]
  description = """
      This is a short description for dsg_rpm_2_v2.1
  """
  comments = "long_format with scenario_year for each scenario and year"
  resource_url = "https://api.hpc.nrel.gov/esif/docs/repo.html"
  geographic_extent = "RS-A"

  [dataset.location]
    system = "ESIFRepoAPI"
    description = "API docs can be found here https://api.hpc.nrel.gov/esif/docs/repo.html"
    url = "https://esif.hpc.nrel.gov/esif/api/repo/files"
    project = "1d3562e0-2a60-4e0a-9496-1032ee4b7db6"
    dataset = "7732b2af-f8fa-48c4-a0d9-a618ca3c374f"
    keyword = ""
    tag = []
    classification = []
    ids = []

  [dataset.registration_author]
    username = "mmooney"
    first_name = "Meghan"
    last_name = "Mooney"
    email = "Meghan.Mooney@nrel.gov"

  [dataset.source_code]
    location = "https://github.com/NLR/mycode/"
    branch = "test"

  [dataset.temporal_info]
    extent = "8760 for all project model years"
    fidelity = "hourly"

  [dataset.spatial_info]
    extent = "LA City"
    fidelity = "receiving station level (nodal within the city)"

Schema Definition

schemas

DatasetCreate

Dataset Checkin Schema.

Attributes:

Name Type Description
name str

A short name.
display_name str | None

The dataset display name.
description str

The description of the scheduled dataset.
version str

Dataset version.
hash_value str

The hash value of this dataset used for integrity check.
version_status VersionStatus

Dataset version status.
previous_version str | None

Previous version of this dataset.
data_format str | None

Data format, or a list of formats separated by commas.
schema_info str | None

The schema description of the dataset.
location dict

The dataset location on data system.
registration_author UserCreate

The person who registered this dataset.
weather_years list[int]

The weather year(s) of the dataset.
model_years list[int]

The model year(s) of the dataset.
units list[str]

The units of the dataset.
temporal_info TemporalInfo

The temportal metadata of the dataset.
spatial_info SpatialInfo

The spatial metadata of the dataset.
scenarios list[str]

The list of scenario names the dataset relates to.
sensitivities list[str]

The sensitivities of the dataset.
source_code SourceCode

The source code that produces the dataset.
relevant_links list[str]

Relevant links to this dataset.
comments str

Registration comments about this dataset.
resource_url str

The resource URL for this dataset.
other dict

Other metadata info about the dataset.

Config:

  • protected_namespaces: ()

Fields:

comments

Registration comments about this dataset

data_format

data format, or a list of formats separated by commas

description

The description of the scheduled dataset

display_name

The dataset display name

hash_value

The hash value of this dataset used for integrity check.

location

The dataset location on data system

model_years

The model year(s) of the dataset

name

A short name

other

other metadata info about the dataset

previous_version

Previous version of this dataset

registration_author

The person who registered this dataset

Relevant links to this dataset

resource_url

The resource URL for this dataset

scenarios

The list of scenario names the dataset relates to

schema_info

The schema description of the dataset

sensitivities

The sensitivities of the dataset

source_code

The source code that produces the dataset

spatial_info

The spatial metadata of the dataset

temporal_info

The temportal metadata of the dataset

units

The units of the dataset

version

Dataset version

version_status

Dataset version status

weather_years

The weather year(s) of the dataset

SpatialInfo

Dataset spatial information.

Attributes:

Name Type Description
extent str

The spatial extent of the dataset.
fidelity str

The fidelity of the dataset in space.
other dict

Other info about spatial characteristics of data.

Fields:

extent

The spatial extent of the dataset

fidelity

The fidelity of the dataset in space

other

other info about spatial characteristics of data

TemporalInfo

Dataset temporal information.

Attributes:

Name Type Description
extent str

The temporal extent of the dataset.
fidelity str

The fidelity of the dataset in time.
other dict

Other info about temporal characteristics of data.

Fields:

extent

The temporal extent of the dataset

fidelity

The fidelity of the dataset in time

other

other info about temporal characteristics of data

VersionStatus

Active

Inactivate

Unresolved

schemas

UserCreate

User base model.

Attributes:

Name Type Description
email EmailStr

Email address.
first_name str | None

First name.
last_name str | None

Last name.
organization str | None

Organization name.

Fields:

email

Email address

first_name

First name

last_name

Last name

organization

Organization name

schemas

SourceCode

Source Model Schema.

Attributes:

Name Type Description
location str

The location of the source code.
branch str | None

The git branch of source code.
tag str | None

The git tag of source code.
image str | None

The location of container image.

Fields:

branch

The git branch of source code

image

The location of container image

location

The location of the source code

tag

The git tag of source code