Dataset Schema
Generated by
scripts/generate_docs.py(scripts/regen_docs.sh). Do not edit by hand.
Methods of client.dataset_schema (reportnet_api.resources.dataset_schema_ext.DatasetSchemaResource). Every method also exists on AsyncReportnetClient as await a_client.dataset_schema.<name>(...) with the same arguments. Roles are copied from the API spec; where it states none, this page says so rather than guessing.
Methods
- export_dataflow_schemas_zip
- export_field_schemas
- export_field_schemas_from_dataset
- find_data_schema_by_dataset_id
- get_simple_schema
- get_table_schemas_ids
- import_dataflow_schemas_zip
- import_dataflow_schemas_zip_and_wait
- import_field_schemas
- update_manually_editable_by_dataset_id
export_dataflow_schemas_zip
Downloads the full dataflow schema export ZIP: one .schema/ .qcrules/.unique/.integrity/.extintegrations file set per dataset schema in the dataflow, plus the global datasetSchemaNames.names and datasetSchemaIds.ids files.
HTTP: GET /dataschema/export?dataflowId=...
Call: client.dataset_schema.export_dataflow_schemas_zip(...) (async client: await a_client.dataset_schema.export_dataflow_schemas_zip(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataflow_id |
int |
yes | ||
dest |
str \| os.PathLike[str] |
yes |
Returns: None
Allowed roles:
Not stated in the API spec.
DatasetSchemaControllerImpl#exportSchemas (controller method
mapped to this route) is annotated @ApiOperation(hidden = true),
so it never reaches spec/api-docs.json and can't come from the
generator — this is a permanent hand-written addition, not a stopgap.
Requires DATAFLOW_CUSTODIAN or DATAFLOW_STEWARD on the dataflow.
The Accept: */* override is required: RequestCore.build_headers()'s
default Accept: application/json gets rejected before auth is even
checked — confirmed against the live production API, this specific
route 406s behind what looks like a gateway/proxy layer in front of
the actual controller. Oddly, Accept: application/octet-stream
(matching the controller's declared produces) 406s too with an
empty body, while Accept: */* passes through cleanly to the real
auth/role check.
Live-tested notes:
- Hidden endpoint (absent from the spec), hand-written. Needs
DATAFLOW_CUSTODIANorDATAFLOW_STEWARD: an ApiKey (Reporter-level) gets 403, a Bearer JWT of such a user returns the ZIP. SendsAccept: */*.
export_field_schemas
Allowed roles: CUSTODIAN, STEWARD, EDITOR WRITE
HTTP: GET /dataschema/v1/{datasetSchemaId}/exportFieldSchemas
Call: client.dataset_schema.export_field_schemas(...) (async client: await a_client.dataset_schema.export_field_schemas(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataset_schema_id |
str |
yes | Dataset schema id | |
dataset_id |
int \| None |
no | None |
Dataset id |
table_schema_id |
str \| None |
no | None |
Table schema id |
Returns: str
Allowed roles:
Not stated in the API spec.
Live-tested notes:
- The default
Accept: application/jsongets a 406 from this route (Spring content negotiation against the controller'sproduces);Accept: */*gets through. The generated method now sends a matchingAcceptitself (mocked tests only, not re-tested live). dataset_idandtable_schema_idare both typed optional, but the backend needs both together (one alone gives 400 or 500); with both it returns CSV text.
export_field_schemas_from_dataset
Allowed roles: CUSTODIAN, STEWARD, EDITOR WRITE
HTTP: GET /dataschema/v1/dataset/{datasetId}/exportFieldSchemas
Call: client.dataset_schema.export_field_schemas_from_dataset(...) (async client: await a_client.dataset_schema.export_field_schemas_from_dataset(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataset_id |
int |
yes | Dataset id |
Returns: str
Allowed roles:
Not stated in the API spec.
Live-tested notes:
- The default
Accept: application/jsongets a 406 from this route (Spring content negotiation against the controller'sproduces);Accept: */*gets through. The generated method now sends a matchingAcceptitself (mocked tests only, not re-tested live). - Typed to return
str, but the real body is a binary ZIP containing a per-table CSV, so decoding it as text is lossy. Not yet migrated to thedest=download treatment.
find_data_schema_by_dataset_id
Get dataset schema by dataset id
HTTP: GET /dataschema/v1/datasetId/{datasetId}
Call: client.dataset_schema.find_data_schema_by_dataset_id(...) (async client: await a_client.dataset_schema.find_data_schema_by_dataset_id(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataset_id |
int |
yes | Dataset id |
Returns: DataSetSchemaVO
Allowed roles:
| Dataset type | Roles |
|---|---|
| Reporting dataset | STEWARD, LEAD REPORTER, CUSTODIAN, OBSERVER, REPORTER WRITE, REPORTER READ, NATIONAL COORDINATOR, STEWARD SUPPORT |
| Data collection | CUSTODIAN, STEWARD, OBSERVER, STEWARD SUPPORT |
| Test dataset | CUSTODIAN, STEWARD, OBSERVER, STEWARD SUPPORT |
| Reference dataset | CUSTODIAN, STEWARD, OBSERVER, STEWARD SUPPORT |
| Design dataset | CUSTODIAN, STEWARD, EDITOR READ, EDITOR WRITE |
| EU dataset | CUSTODIAN, STEWARD, OBSERVER, STEWARD SUPPORT |
get_simple_schema
Get dataset schema by dataset id
HTTP: GET /dataschema/v1/getSimpleSchema/dataset/{datasetId}
Call: client.dataset_schema.get_simple_schema(...) (async client: await a_client.dataset_schema.get_simple_schema(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataset_id |
int |
yes | Dataset id | |
dataflow_id |
int \| None |
no | None |
Dataflow id |
provider_id |
int \| None |
no | None |
Provider id |
Returns: SimpleDatasetSchemaVO
Allowed roles:
| Dataset type | Roles |
|---|---|
| Reporting dataset | STEWARD, LEAD REPORTER, CUSTODIAN, REPORTER WRITE, REPORTER READ, NATIONAL COORDINATOR |
| Data collection | CUSTODIAN, STEWARD |
| Test dataset | CUSTODIAN, STEWARD, STEWARD SUPPORT |
| Design dataset | CUSTODIAN, STEWARD, EDITOR WRITE |
| EU dataset | CUSTODIAN, STEWARD |
Live-tested notes:
dataflow_idis typed optional (as in the spec) but the backend requires it: without it the call fails with 400Required Long parameter 'dataflowId' is not present.
get_table_schemas_ids
Get list of dataset table schemas by dataset id
HTTP: GET /dataschema/v1/getTableSchemasIds/{datasetId}
Call: client.dataset_schema.get_table_schemas_ids(...) (async client: await a_client.dataset_schema.get_table_schemas_ids(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataset_id |
int |
yes | Dataset id | |
dataflow_id |
int \| None |
no | None |
Dataflow id |
provider_id |
int \| None |
no | None |
Provider id |
Returns: list[TableSchemaIdNameVO]
Allowed roles:
| Dataset type | Roles |
|---|---|
| Reporting dataset | STEWARD, LEAD REPORTER, CUSTODIAN, REPORTER WRITE, REPORTER READ, NATIONAL COORDINATOR |
| Data collection | CUSTODIAN, STEWARD |
| Test dataset | CUSTODIAN, STEWARD, STEWARD SUPPORT |
| Reference dataset | CUSTODIAN, STEWARD |
| Design dataset | CUSTODIAN, STEWARD, EDITOR WRITE |
| EU dataset | CUSTODIAN, STEWARD |
Live-tested notes:
dataflow_idis typed optional (as in the spec) but the backend requires it: omitting it gets the same 400 asget_simple_schema.
import_dataflow_schemas_zip
add the dataset schemas of an export ZIP (export_dataflow_schemas_zip()'s file set) to a dataflow.
HTTP: POST /dataschema/import?dataflowId=...
Call: client.dataset_schema.import_dataflow_schemas_zip(...) (async client: await a_client.dataset_schema.import_dataflow_schemas_zip(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataflow_id |
int |
yes | ||
file |
str \| os.PathLike[str] |
yes |
Returns: None
Allowed roles:
Not stated in the API spec.
DatasetSchemaControllerImpl#importSchemas (hidden), multipart file.
Requires DATAFLOW_CUSTODIAN or DATAFLOW_STEWARD on the dataflow, and
the dataflow in DESIGN status (403 "Invalid dataflow status"
otherwise). The file name must end in .zip (checked here, raises
ValueError): the backend ignores anything else.
Asynchronous: answers 200 once the work is started. The import
(DataschemaServiceImpl#importSchemas) adds one new design dataset
per schema, named IMPORTED_<name> (IMPORTED_<name> (1), ... when
taken), never replaces existing ones, gives every schema, table,
record and field a new id (remapping links, unique constraints,
integrity checks, QC rules and external integrations to them), and
then runs the SQL QC rule check. A second import into the same
dataflow while one runs answers 423. Use
import_dataflow_schemas_zip_and_wait() to wait for it.
Not retried: if the connection times out or drops after the upload was
sent, this raises SchemaImportError (chained to the transport error),
because the import may have started anyway; check the dataflow's design
datasets before trying again.
import_dataflow_schemas_zip_and_wait
Import a schema ZIP into a dataflow and wait until it looks done.
Call: client.dataset_schema.import_dataflow_schemas_zip_and_wait(...) (async client: await a_client.dataset_schema.import_dataflow_schemas_zip_and_wait(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataflow_id |
int |
yes | ||
file |
str \| os.PathLike[str] |
yes | ||
poll_interval |
float |
no | 5.0 |
|
settle |
float |
no | 120.0 |
|
timeout |
float |
no | 900.0 |
Returns: list[dict[str, Any]]
Allowed roles:
Not an endpoint of its own; the rights are those of the underlying calls.
Returns the design datasets the import created: the designDatasets
entries of dataflow_datasets() that are new and named
IMPORTED_<name>, sorted by id. Raises ValueError if the ZIP holds
no .schema file, and SchemaImportError (a JobError) if the
dataflow details can't be read first, if the import stalls, or after
timeout seconds.
The backend gives no completion signal through the API (its
IMPORT_DATASET_SCHEMA_COMPLETED_EVENT / _FAILED_EVENT only reach
the web app's websocket, and the import lock is only on an internal
route), so this is a heuristic in two steps:
- until the dataflow has as many new
IMPORTED_design datasets as the ZIP has.schemafiles (the import creates them first, ~4 s apart); - until the dataflow's schema export (
export_dataflow_schemas_zip) matches the uploaded ZIP and has stopped changing. The import fills the datasets one after another (tables ~2 s each, a dataset's schema document only after its last table), then the unique constraints, QC and integrity rules, external integrations and the SQL rule check, so a quiet export is not a finished one. For each newIMPORTED_<name>dataset (paired with<name>in the ZIP) the export must have the same tables, the same number of fields per table, and the same number of unique constraints, QC rules, integrity rules and external integrations (counts only: the SQL rule check disables and un-verifies rules that fail it, so theirenabled/verifiedflags are not compared). Once it matches, it must also stay unchanged for two polls. A 5xx, a timeout or a broken ZIP from the export counts as "not done yet".
If the export stays unchanged for settle seconds while it still lacks
content, the import has stalled or failed and SchemaImportError
says what is missing (for example "IMPORTED_X has 0/14 tables").
A 5xx or a transient transport error (timeout, dropped connection)
while polling does not end the wait: the import carries on server side,
and a retry would import a second set. The timeout's SchemaImportError
names the last such error and is chained to it. Any other error while
waiting (e.g. 401 once a login token expires) raises a chained
SchemaImportError saying the import may still be running: check the
dataflow's design datasets before retrying. timeout counts from the
end of the upload.
Limits: the backend reports completion only on the web app's
websocket, so this is still a heuristic. An item the import legitimately
leaves out (a rule it cannot recreate, an integration it cannot
create) shows up as a stall error naming the count, not as a silent
success. An import that fails before creating datasets (invalid names
in the ZIP, ...) ends in the timeout. ValueError if the ZIP holds no
readable .schema file. Same roles as import_dataflow_schemas_zip().
Polls every poll_interval seconds.
import_field_schemas
Allowed roles: CUSTODIAN, STEWARD, EDITOR WRITE
HTTP: POST /dataschema/v1/{datasetSchemaId}/importFieldSchemas
Call: client.dataset_schema.import_field_schemas(...) (async client: await a_client.dataset_schema.import_field_schemas(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataset_schema_id |
str |
yes | Dataset schema id | |
file |
str \| os.PathLike[str] |
yes | ||
dataset_id |
int \| None |
no | None |
Dataset id |
replace |
bool \| None |
no | None |
Replace current definition |
table_schema_id |
str \| None |
no | None |
Table schema id |
Returns: None
Allowed roles:
Not stated in the API spec.
Live-tested notes:
- With
replace=Truefields are destroyed and recreated even for a no-op change, so field ids change; re-fetch ids viafind_data_schema_by_dataset_idafterwards. A read right after a renaming import came back with no fields (lag vs failure unresolved).
update_manually_editable_by_dataset_id
updateManuallyEditableByDatasetId
HTTP: PUT /dataschema/updateManuallyEditable/{datasetId}
Call: client.dataset_schema.update_manually_editable_by_dataset_id(...) (async client: await a_client.dataset_schema.update_manually_editable_by_dataset_id(...))
| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
dataset_id |
int |
yes | ||
manually_editable |
bool |
yes |
Returns: None
Allowed roles:
Not stated in the API spec.