dask#
- class predict_backend.persistence.dask_persistence.DaskPersistence(handler, name, data=None, allow_filter=False)#
Bases:
Persistence
Persistence instance class for use with dask dataframes (and more) which are published to dask clusters.
- Parameters:
handler (
DaskHandler
) – a predict_backend.persistence.data_store_handler_dask.DaskHandler objectname (
str
) – the object namedata – the object to be persisted
allow_filter (
bool
) –
- extension_type = 'dask'#
- class predict_backend.persistence.data_store_handler_dask.DaskHandler(client)#
Bases:
DataStoreHandler
Convenience Class for persisting datasets/objects to a Dask Cluster
- Parameters:
client (
Client
) – a dask.distributed Client object
- download(dname, client)#
Download an object from a dask cluster
- Parameters:
dname (
str
) – the dataset nameclient (
Client
) – the dask.distributed Client object
- Returns:
the persisted object
- upload(data, dname)#
Upload an object to a dask cluster
- Parameters:
data – the data/object to be uploaded
dname – the dataset name
- Returns: