Resolve container configuration from TOML

Description

Resolves a logical container key to a concrete Azure container name and corresponding AzureStorageConfig object based on env and a TOML configuration file.

Usage

containerio_config(
  container_key,
  env = options::opt("storage_env"),
  toml_file = "container-config.toml",
  auth_type = options::opt("auth_type")
)

Arguments

Argument	Description
container_key	Character string. Logical container key defined in the [containers] section of the TOML file (e.g. "raw", "analytics").
env	Storage environment. One of "local", "staging", "prod".
toml_file	Character string. Path to the TOML configuration file. Defaults to "container-config.toml".
auth_type	Azure auth method. One of "device-code", "sas-token", "client-credentials". If NULL, falls back to session auth method, then "device-code".

Value

A named list with the following elements:

container : Character string. The resolved Azure container name or NULL when env = "local" (local filesystem mode).

azure_config : A Python AzureStorageConfig object created via reticulate, suitable for passing to containerio::StorageHandler(), or NULL when env = "local" (local filesystem mode).

Details

The function looks up the container definition under [containers] and maps it to an Azure configuration defined under [azure_auth] for the selected env. It then constructs an AzureStorageConfig object that can be passed directly to containerio::StorageHandler().

The TOML file must contain [azure_auth] and [containers] sections. Each container maps an environment (staging, prod) to a concrete container name and an Azure auth configuration.

Auth can be referenced in two ways.

Tag indirection (shared auth across containers). azure_auth.<env> is a string referencing a [azure_auth.<tag>] section:

[azure_auth.raw_staging]
storage_account = "<your-storage-account>"
tenant_id = "<your-tenant-id>"
client_id = "<client-id-for-client-cred-auth-staging>"
client_secret_env = "<name-of-env-var-with-secret-for-client-cred-auth-staging>"
device_code_client_id = "<device-code-client-id-for-staging>"

[azure_auth.raw_prod]
storage_account = "<your-storage-account>"
tenant_id = "<your-tenant-id>"
client_id = "<client-id-for-client-cred-auth-prod>"
client_secret_env = "<name-of-env-var-with-secret-for-client-cred-auth-prod>"
device_code_client_id = "<device-code-client-id-for-prod>"

[containers.input1]
name.staging = "<container-name-input-from-source1-staging>"
name.prod = "<container-name-input-from-source1-prod>"
azure_auth.staging = "raw_staging"
azure_auth.prod = "raw_prod"

[containers.input2]
name.staging = "<container-name-input-from-source2-staging>"
name.prod = "<container-name-input-from-source2-prod>"
azure_auth.staging = "raw_staging"
azure_auth.prod = "raw_prod"

Inline nested auth (per-container). Alternatively, azure_auth.<env> can be a nested table that defines the auth fields directly under the container, avoiding a separate [azure_auth] section:

[containers.input1.azure_auth.staging]
storage_account = "<your-storage-account>"
tenant_id = "<your-tenant-id>"
client_id = "<client-id-for-client-cred-auth-staging>"
client_secret_env = "<name-of-env-var-with-secret-for-client-cred-auth-staging>"
device_code_client_id = "<device-code-client-id-for-staging>"

client_secret_env is the name of the environment variable that holds the client secret.

If the resolved azure_auth does not define a storage_account, the function falls back to AzureStorageConfig$from_env(), which reads configuration from environment variables.

Errors

The function stops with an error if:

• The specified container_key does not exist
• The container has no azure_auth defined for the selected env
• The referenced azure_auth does not exist

Examples

cfg <- containerio_config(container_key = "input1", env = "staging")

handler <- init_storage_handler(
  container = cfg$container,
  config = cfg$azure_config
)