Skip to content

Dataset Schema

Generated by scripts/generate_docs.py (scripts/regen_docs.sh). Do not edit by hand.

Methods of client.dataset_schema (reportnet_api.resources.dataset_schema_ext.DatasetSchemaResource). Every method also exists on AsyncReportnetClient as await a_client.dataset_schema.<name>(...) with the same arguments. Roles are copied from the API spec; where it states none, this page says so rather than guessing.

Back to index

Methods

export_dataflow_schemas_zip

Downloads the full dataflow schema export ZIP: one .schema/ .qcrules/.unique/.integrity/.extintegrations file set per dataset schema in the dataflow, plus the global datasetSchemaNames.names and datasetSchemaIds.ids files.

HTTP: GET /dataschema/export?dataflowId=...

Call: client.dataset_schema.export_dataflow_schemas_zip(...) (async client: await a_client.dataset_schema.export_dataflow_schemas_zip(...))

Argument Type Required Default Description
dataflow_id int yes
dest str \| os.PathLike[str] yes

Returns: None

Allowed roles:

Not stated in the API spec.

DatasetSchemaControllerImpl#exportSchemas (controller method mapped to this route) is annotated @ApiOperation(hidden = true), so it never reaches spec/api-docs.json and can't come from the generator — this is a permanent hand-written addition, not a stopgap. Requires DATAFLOW_CUSTODIAN or DATAFLOW_STEWARD on the dataflow.

The Accept: */* override is required: RequestCore.build_headers()'s default Accept: application/json gets rejected before auth is even checked — confirmed against the live production API, this specific route 406s behind what looks like a gateway/proxy layer in front of the actual controller. Oddly, Accept: application/octet-stream (matching the controller's declared produces) 406s too with an empty body, while Accept: */* passes through cleanly to the real auth/role check.

Live-tested notes:

  • Hidden endpoint (absent from the spec), hand-written. Needs DATAFLOW_CUSTODIAN or DATAFLOW_STEWARD: an ApiKey (Reporter-level) gets 403, a Bearer JWT of such a user returns the ZIP. Sends Accept: */*.

export_field_schemas

Allowed roles: CUSTODIAN, STEWARD, EDITOR WRITE

HTTP: GET /dataschema/v1/{datasetSchemaId}/exportFieldSchemas

Call: client.dataset_schema.export_field_schemas(...) (async client: await a_client.dataset_schema.export_field_schemas(...))

Argument Type Required Default Description
dataset_schema_id str yes Dataset schema id
dataset_id int \| None no None Dataset id
table_schema_id str \| None no None Table schema id

Returns: str

Allowed roles:

Not stated in the API spec.

Live-tested notes:

  • The default Accept: application/json gets a 406 from this route (Spring content negotiation against the controller's produces); Accept: */* gets through. The generated method now sends a matching Accept itself (mocked tests only, not re-tested live).
  • dataset_id and table_schema_id are both typed optional, but the backend needs both together (one alone gives 400 or 500); with both it returns CSV text.

export_field_schemas_from_dataset

Allowed roles: CUSTODIAN, STEWARD, EDITOR WRITE

HTTP: GET /dataschema/v1/dataset/{datasetId}/exportFieldSchemas

Call: client.dataset_schema.export_field_schemas_from_dataset(...) (async client: await a_client.dataset_schema.export_field_schemas_from_dataset(...))

Argument Type Required Default Description
dataset_id int yes Dataset id

Returns: str

Allowed roles:

Not stated in the API spec.

Live-tested notes:

  • The default Accept: application/json gets a 406 from this route (Spring content negotiation against the controller's produces); Accept: */* gets through. The generated method now sends a matching Accept itself (mocked tests only, not re-tested live).
  • Typed to return str, but the real body is a binary ZIP containing a per-table CSV, so decoding it as text is lossy. Not yet migrated to the dest= download treatment.

find_data_schema_by_dataset_id

Get dataset schema by dataset id

HTTP: GET /dataschema/v1/datasetId/{datasetId}

Call: client.dataset_schema.find_data_schema_by_dataset_id(...) (async client: await a_client.dataset_schema.find_data_schema_by_dataset_id(...))

Argument Type Required Default Description
dataset_id int yes Dataset id

Returns: DataSetSchemaVO

Allowed roles:

Dataset type Roles
Reporting dataset STEWARD, LEAD REPORTER, CUSTODIAN, OBSERVER, REPORTER WRITE, REPORTER READ, NATIONAL COORDINATOR, STEWARD SUPPORT
Data collection CUSTODIAN, STEWARD, OBSERVER, STEWARD SUPPORT
Test dataset CUSTODIAN, STEWARD, OBSERVER, STEWARD SUPPORT
Reference dataset CUSTODIAN, STEWARD, OBSERVER, STEWARD SUPPORT
Design dataset CUSTODIAN, STEWARD, EDITOR READ, EDITOR WRITE
EU dataset CUSTODIAN, STEWARD, OBSERVER, STEWARD SUPPORT

get_simple_schema

Get dataset schema by dataset id

HTTP: GET /dataschema/v1/getSimpleSchema/dataset/{datasetId}

Call: client.dataset_schema.get_simple_schema(...) (async client: await a_client.dataset_schema.get_simple_schema(...))

Argument Type Required Default Description
dataset_id int yes Dataset id
dataflow_id int \| None no None Dataflow id
provider_id int \| None no None Provider id

Returns: SimpleDatasetSchemaVO

Allowed roles:

Dataset type Roles
Reporting dataset STEWARD, LEAD REPORTER, CUSTODIAN, REPORTER WRITE, REPORTER READ, NATIONAL COORDINATOR
Data collection CUSTODIAN, STEWARD
Test dataset CUSTODIAN, STEWARD, STEWARD SUPPORT
Design dataset CUSTODIAN, STEWARD, EDITOR WRITE
EU dataset CUSTODIAN, STEWARD

Live-tested notes:

  • dataflow_id is typed optional (as in the spec) but the backend requires it: without it the call fails with 400 Required Long parameter 'dataflowId' is not present.

get_table_schemas_ids

Get list of dataset table schemas by dataset id

HTTP: GET /dataschema/v1/getTableSchemasIds/{datasetId}

Call: client.dataset_schema.get_table_schemas_ids(...) (async client: await a_client.dataset_schema.get_table_schemas_ids(...))

Argument Type Required Default Description
dataset_id int yes Dataset id
dataflow_id int \| None no None Dataflow id
provider_id int \| None no None Provider id

Returns: list[TableSchemaIdNameVO]

Allowed roles:

Dataset type Roles
Reporting dataset STEWARD, LEAD REPORTER, CUSTODIAN, REPORTER WRITE, REPORTER READ, NATIONAL COORDINATOR
Data collection CUSTODIAN, STEWARD
Test dataset CUSTODIAN, STEWARD, STEWARD SUPPORT
Reference dataset CUSTODIAN, STEWARD
Design dataset CUSTODIAN, STEWARD, EDITOR WRITE
EU dataset CUSTODIAN, STEWARD

Live-tested notes:

  • dataflow_id is typed optional (as in the spec) but the backend requires it: omitting it gets the same 400 as get_simple_schema.

import_dataflow_schemas_zip

add the dataset schemas of an export ZIP (export_dataflow_schemas_zip()'s file set) to a dataflow.

HTTP: POST /dataschema/import?dataflowId=...

Call: client.dataset_schema.import_dataflow_schemas_zip(...) (async client: await a_client.dataset_schema.import_dataflow_schemas_zip(...))

Argument Type Required Default Description
dataflow_id int yes
file str \| os.PathLike[str] yes

Returns: None

Allowed roles:

Not stated in the API spec.

DatasetSchemaControllerImpl#importSchemas (hidden), multipart file. Requires DATAFLOW_CUSTODIAN or DATAFLOW_STEWARD on the dataflow, and the dataflow in DESIGN status (403 "Invalid dataflow status" otherwise). The file name must end in .zip (checked here, raises ValueError): the backend ignores anything else.

Asynchronous: answers 200 once the work is started. The import (DataschemaServiceImpl#importSchemas) adds one new design dataset per schema, named IMPORTED_<name> (IMPORTED_<name> (1), ... when taken), never replaces existing ones, gives every schema, table, record and field a new id (remapping links, unique constraints, integrity checks, QC rules and external integrations to them), and then runs the SQL QC rule check. A second import into the same dataflow while one runs answers 423. Use import_dataflow_schemas_zip_and_wait() to wait for it.

Not retried: if the connection times out or drops after the upload was sent, this raises SchemaImportError (chained to the transport error), because the import may have started anyway; check the dataflow's design datasets before trying again.

import_dataflow_schemas_zip_and_wait

Import a schema ZIP into a dataflow and wait until it looks done.

Call: client.dataset_schema.import_dataflow_schemas_zip_and_wait(...) (async client: await a_client.dataset_schema.import_dataflow_schemas_zip_and_wait(...))

Argument Type Required Default Description
dataflow_id int yes
file str \| os.PathLike[str] yes
poll_interval float no 5.0
settle float no 120.0
timeout float no 900.0

Returns: list[dict[str, Any]]

Allowed roles:

Not an endpoint of its own; the rights are those of the underlying calls.

Returns the design datasets the import created: the designDatasets entries of dataflow_datasets() that are new and named IMPORTED_<name>, sorted by id. Raises ValueError if the ZIP holds no .schema file, and SchemaImportError (a JobError) if the dataflow details can't be read first, if the import stalls, or after timeout seconds.

The backend gives no completion signal through the API (its IMPORT_DATASET_SCHEMA_COMPLETED_EVENT / _FAILED_EVENT only reach the web app's websocket, and the import lock is only on an internal route), so this is a heuristic in two steps:

  1. until the dataflow has as many new IMPORTED_ design datasets as the ZIP has .schema files (the import creates them first, ~4 s apart);
  2. until the dataflow's schema export (export_dataflow_schemas_zip) matches the uploaded ZIP and has stopped changing. The import fills the datasets one after another (tables ~2 s each, a dataset's schema document only after its last table), then the unique constraints, QC and integrity rules, external integrations and the SQL rule check, so a quiet export is not a finished one. For each new IMPORTED_<name> dataset (paired with <name> in the ZIP) the export must have the same tables, the same number of fields per table, and the same number of unique constraints, QC rules, integrity rules and external integrations (counts only: the SQL rule check disables and un-verifies rules that fail it, so their enabled/verified flags are not compared). Once it matches, it must also stay unchanged for two polls. A 5xx, a timeout or a broken ZIP from the export counts as "not done yet".

If the export stays unchanged for settle seconds while it still lacks content, the import has stalled or failed and SchemaImportError says what is missing (for example "IMPORTED_X has 0/14 tables").

A 5xx or a transient transport error (timeout, dropped connection) while polling does not end the wait: the import carries on server side, and a retry would import a second set. The timeout's SchemaImportError names the last such error and is chained to it. Any other error while waiting (e.g. 401 once a login token expires) raises a chained SchemaImportError saying the import may still be running: check the dataflow's design datasets before retrying. timeout counts from the end of the upload.

Limits: the backend reports completion only on the web app's websocket, so this is still a heuristic. An item the import legitimately leaves out (a rule it cannot recreate, an integration it cannot create) shows up as a stall error naming the count, not as a silent success. An import that fails before creating datasets (invalid names in the ZIP, ...) ends in the timeout. ValueError if the ZIP holds no readable .schema file. Same roles as import_dataflow_schemas_zip(). Polls every poll_interval seconds.

import_field_schemas

Allowed roles: CUSTODIAN, STEWARD, EDITOR WRITE

HTTP: POST /dataschema/v1/{datasetSchemaId}/importFieldSchemas

Call: client.dataset_schema.import_field_schemas(...) (async client: await a_client.dataset_schema.import_field_schemas(...))

Argument Type Required Default Description
dataset_schema_id str yes Dataset schema id
file str \| os.PathLike[str] yes
dataset_id int \| None no None Dataset id
replace bool \| None no None Replace current definition
table_schema_id str \| None no None Table schema id

Returns: None

Allowed roles:

Not stated in the API spec.

Live-tested notes:

  • With replace=True fields are destroyed and recreated even for a no-op change, so field ids change; re-fetch ids via find_data_schema_by_dataset_id afterwards. A read right after a renaming import came back with no fields (lag vs failure unresolved).

update_manually_editable_by_dataset_id

updateManuallyEditableByDatasetId

HTTP: PUT /dataschema/updateManuallyEditable/{datasetId}

Call: client.dataset_schema.update_manually_editable_by_dataset_id(...) (async client: await a_client.dataset_schema.update_manually_editable_by_dataset_id(...))

Argument Type Required Default Description
dataset_id int yes
manually_editable bool yes

Returns: None

Allowed roles:

Not stated in the API spec.