provenaclient.clients.datastore_client

Created Date: Thursday June 6th 2024 +1000 Author: Peter Baker —– Last Modified: Thursday June 6th 2024 1:39:55 pm +1000 Modified By: Peter Baker —– Description: Datastore L2 Client. —– HISTORY: Date By Comments ———- — ———————————————————

Classes

`DatastoreEndpoints`	An ENUM containing the datastore-api endpoints.
`DatasetReviewSubClient`	This class interface just captures that the client has an instantiated auth
`DatastoreClient`	This class interface just captures that the client has an instantiated auth

Module Contents

class provenaclient.clients.datastore_client.DatastoreEndpoints[source]

Bases: str, ProvenaInterfaces.DataStoreAPI.Enum

An ENUM containing the datastore-api endpoints.

GET_REGISTRY_ITEMS_FETCH_DATASET = '/registry/items/fetch-dataset'

POST_REGISTER_MINT_DATASET = '/register/mint-dataset'

GET_HEALTH_CHECK = '/'

POST_METADATA_VALIDATE_METADATA = '/metadata/validate-metadata'

GET_METADATA_DATASET_SCHEMA = '/metadata/dataset-schema'

POST_REGISTER_UPDATE_METADATA = '/register/update-metadata'

PUT_REGISTER_REVERT_METADATA = '/register/revert-metadata'

POST_REGISTER_VERSION = '/register/version'

POST_REGISTRY_ITEMS_LIST = '/registry/items/list'

POST_REGISTRY_ITEMS_GENERATE_PRESIGNED_URL = '/registry/items/generate-presigned-url'

POST_REGISTRY_CREDENTIALS_GENERATE_READ_ACCESS_CREDENTIALS = '/registry/credentials/generate-read-access-credentials'

POST_REGISTRY_CREDENTIALS_GENERATE_WRITE_ACCESS_CREDENTIALS = '/registry/credentials/generate-write-access-credentials'

DELETE_RELEASE_SYS_REVIEWERS_DELETE = '/release/sys-reviewers/delete'

POST_RELEASE_SYS_REVIEWERS_ADD = '/release/sys-reviewers/add'

GET_RELEASE_SYS_REVIEWERS_LIST = '/release/sys-reviewers/list'

POST_RELEASE_APPROVAL_REQUEST = '/release/approval-request'

PUT_RELEASE_ACTION_APPROVAL_REQUEST = '/release/action-approval-request'

GET_CHECK_ACCESS_CHECK_GENERAL_ACCESS = '/check-access/check-general-access'

GET_CHECK_ACCESS_CHECK_ADMIN_ACCESS = '/check-access/check-admin-access'

GET_CHECK_ACCESS_CHECK_READ_ACCESS = '/check-access/check-read-access'

GET_CHECK_ACCESS_CHECK_WRITE_ACCESS = '/check-access/check-write-access'

GET_ADMIN_CONFIG = '/admin/config'

GET_ADMIN_SENTRY_DEBUG = '/admin/sentry-debug'

class provenaclient.clients.datastore_client.DatasetReviewSubClient(auth: provenaclient.clients.client_helpers.AuthManager, config: provenaclient.clients.client_helpers.Config)[source]

Bases: provenaclient.clients.client_helpers.ClientService

This class interface just captures that the client has an instantiated auth manager which allows for helper functions abstracted for L2 clients.

_auth

_config

_build_endpoint(endpoint: DatastoreEndpoints) → str[source]

async delete_dataset_reviewer(reviewer_id: str) → None[source]

Delete a reviewer existing within the datastore.

Parameters:: reviewer_id (str) – Id of an existing reviewer within the system.

async add_dataset_reviewer(reviewer_id: str) → None[source]

Add a reviewer to the datastore.

Parameters:: reviewer_id (str) – Valid Id of a reviewer.

async approval_request(approval_request_payload: ProvenaInterfaces.DataStoreAPI.ReleaseApprovalRequest) → ProvenaInterfaces.DataStoreAPI.ReleaseApprovalRequestResponse[source]

Submit a request for approval of dataset through the datastore.

Parameters:: approval_request_payload (ReleaseApprovalRequest) – An object that requires the dataset id, approver id and notes
Returns:: Contains details of the approval request.
Return type:: ReleaseApprovalRequestResponse

async action_approval_request(action_approval_request_payload: ProvenaInterfaces.DataStoreAPI.ActionApprovalRequest) → ProvenaInterfaces.DataStoreAPI.ActionApprovalRequestResponse[source]

Action an approval request from a dataset approval request via the datastore.

Parameters:: action_approval_request (ActionApprovalRequest) – The dataset id, your decision of approval and any extra information you want to add (notes).
Returns:: The details of the approval action and the relevant dataset details.
Return type:: ActionApprovalRequestResponse

class provenaclient.clients.datastore_client.DatastoreClient(auth: provenaclient.clients.client_helpers.AuthManager, config: provenaclient.clients.client_helpers.Config)[source]

Bases: provenaclient.clients.client_helpers.ClientService

This class interface just captures that the client has an instantiated auth manager which allows for helper functions abstracted for L2 clients.

review: DatasetReviewSubClient

_auth

_config

_build_endpoint(endpoint: DatastoreEndpoints) → str[source]

async get_health_check() → provenaclient.models.HealthCheckResponse[source]

Health check the API

Returns:: Response
Return type:: HealthCheckResponse

async validate_metadata(metadata_payload: ProvenaInterfaces.RegistryModels.CollectionFormat) → provenaclient.utils.helpers.StatusResponse[source]

Validates provided dataset info with the datastore API.

Parameters:: metadata_payload (CollectionFormat) – A structured format containing all necessary information to register a new dataset, including associations, approvals, and dataset-specific information.
Returns:: Response indicating whether your dataset metadata setup is valid and correct.
Return type:: StatusResponse

async update_metadata(handle_id: str, reason: str, metadata_payload: ProvenaInterfaces.RegistryModels.CollectionFormat) → ProvenaInterfaces.DataStoreAPI.UpdateMetadataResponse[source]

Updates existing dataset metadata through datastore API.

Returns:: The updated metadata response from datastore.
Return type:: UpdateMetadataResponse

async revert_metadata(metadata_payload: provenaclient.models.RevertMetadata) → provenaclient.utils.helpers.StatusResponse[source]

Reverts the metadata for a dataset to a previous identified historical version.

Parameters:: metadata_payload (RevertMetadata) – The revert request, passed through to the registry API and requires dataset id, history id and reason for reverting.
Returns:: Response indicating whether your dataset metadata setup is valid and correct.
Return type:: StatusResponse

async version_dataset(version_dataset_payload: ProvenaInterfaces.RegistryAPI.VersionRequest) → ProvenaInterfaces.RegistryAPI.VersionResponse[source]

Creates a new versioning of an existing dataset within Provena: through the Datastore.

Parameters:: version_dataset_payload (VersionRequest) – The request which includes the item ID and reason for versioning.
Returns:: Response of the versioning of the dataset, containing new version ID and job session ID.
Return type:: VersionResponse

async list_datasets(list_request: ProvenaInterfaces.RegistryAPI.NoFilterSubtypeListRequest) → ProvenaInterfaces.RegistryAPI.DatasetListResponse[source]

Gets datasets within the datastore in a paginated fashion.

Parameters:: list_request (NoFilterSubtypeListRequest) – Contains parameters for the specified sorting criteria, optional pagination key and amount of records to fetch.
Returns:: Response of fetching datasets from datastore API.
Return type:: ListRegistryResponse

async generate_presigned_url(presigned_url: ProvenaInterfaces.DataStoreAPI.PresignedURLRequest) → ProvenaInterfaces.DataStoreAPI.PresignedURLResponse[source]

Generates a presigned url for an existing dataset within the datastore.

Parameters:: presigned_url (PresignedURLRequest) – Contains the dataset id + file path + length of expiry of URL.
Returns:: A response containing the presigned url.
Return type:: PresignedURLResponse

async generate_read_access_credentials(read_access_credentials: ProvenaInterfaces.DataStoreAPI.CredentialsRequest) → ProvenaInterfaces.DataStoreAPI.CredentialResponse[source]

Creates a read-access for a certain subdirectory of the S3 bucket.

Parameters:: read_access_credentials (CredentialsRequest) – Contains the dataset id + console session URL required flag (boolean)
Returns:: The AWS credentials creating read level access into the subset of the bucket requested in the S3 location object.
Return type:: CredentialResponse

async generate_write_access_credentials(write_access_credentials: ProvenaInterfaces.DataStoreAPI.CredentialsRequest) → ProvenaInterfaces.DataStoreAPI.CredentialResponse[source]

Creates a write-access for a certain subdirectory of the S3 bucket.

Parameters:: write_access_credentials (CredentialsRequest) – Contains the dataset id + console session URL required flag (boolean)
Returns:: The AWS credentials creating write level access into the subset of the bucket requested in the S3 location object.
Return type:: CredentialResponse

async fetch_dataset(id: str) → ProvenaInterfaces.DataStoreAPI.RegistryFetchResponse[source]

Fetches a dataset from the datastore based on the provided ID.

Parameters:

id (str) – The unique identifier of the dataset to be retrieved. For example: “10378.1/1451860”

Returns:

A interactive python datatype of type RegistryFetchResponse containing the dataset details.

Return type:

RegistryFetchResponse

Raises:

CustomTimeoutException – Raised if the request times out.
Exception – General exception for handling unexpected errors.
ValueError – Raised if there is an issue in parsing the response into the expected model.

async mint_dataset(dataset_info: ProvenaInterfaces.RegistryModels.CollectionFormat) → ProvenaInterfaces.DataStoreAPI.MintResponse[source]

Creates a new dataset in the datastore with the provided dataset information.

Parameters:

dataset_info (CollectionFormat) – A structured format containing all necessary information to register a new dataset, including associations, approvals, and dataset-specific information.

Returns:

A interactive python datatype of type MintResponse containing the newly created dataset details.

Return type:

MintResponse

Raises:

BadRequestException – Raised if the server returns a 400 status code, indicating a bad request.
CustomTimeoutException – Raised if the request times out.
Exception – General exception for handling unexpected errors.
ValueError – Raised if there is an issue in parsing the response into the expected model.