Azure Storage

Accessing a Azure Storage with the shell tool azcopy. This can be either a Azure Blob Storage or a Azure Delta Lake Storage.

Installation

You need to install azcopy. Take a look at Get started with Azcopy.

$ pip install mara-storage[azure-blob]

Configuration examples

Note

Currently some of the functions require a SAS token, and some of the functions require a account key. It is recommended to provide both a SAS token and a account key.

import pathlib
import mara_storage.storages
mara_storage.config.storages = lambda: {
    'data': mara_storage.storages.AzureStorage(
        account_name='account-name',
        container_name='container-name',
        sas='sp=racwdlm&st=2022-05-11T10:04:05Z&se=2023-05-11T18:04:05Z&spr=https&sv=2020-08-04&sr=c&sig=u7tqxugyv5MbyrtFdEUp22tnou4wifBoUfIaLDazeRT%3D',

        # optional
        storage_type = 'dfs'  # use a dfs client instead of 'blob' (default value)
        )
}


API reference

This section contains database specific API in the module.

Configuration

class mara_storage.storages.AzureStorage(account_name: str, container_name: str, sas: Optional[str] = None, storage_type: str = 'blob', account_key: Optional[str] = None, spa_tenant: Optional[str] = None, spa_application: Optional[str] = None, spa_client_secret: Optional[str] = None)
__init__(account_name: str, container_name: str, sas: Optional[str] = None, storage_type: str = 'blob', account_key: Optional[str] = None, spa_tenant: Optional[str] = None, spa_application: Optional[str] = None, spa_client_secret: Optional[str] = None)

Connection information for a Azure sstorage bucket

Possible authentication methods:

SAS => “Shared access signature”, see https://docs.microsoft.com/en-us/azure/storage/common/storage-sas-overview SPA => “Service principal”

Parameters

  • account_name – The storage account name

  • container_name – The container name within the storage

  • storage_type – The storage type. Supports ‘blob’ or ‘dfs’.

  • sas – The SAS token

  • account_key – The storage account key

  • spa_tenant – The service principal tenant id

  • spa_application – The service principal application id

  • spa_client_secret – The service principal client secret

build_uri(path: Optional[str] = None, storage_type: Optional[str] = None)

Returns a URI for a path on the storage