MinIO

Write files to a MinIO-compatible object storage bucket in CSV or JSONL format. Use Zeotap to export model results, audience lists, or enriched data as files in MinIO for downstream consumption. MinIO provides S3-compatible object storage that can run on-premises, in private clouds, or as a managed service.

Prerequisites

A running MinIO server or MinIO-compatible object storage service
A bucket created on the MinIO server
An access key and secret key with read/write permissions on the target bucket
Network connectivity from Zeotap to the MinIO endpoint

Authentication

MinIO uses Access Keys authentication, which is compatible with the S3 API.

In the MinIO Console, navigate to Identity > Users or Access Keys
Create a new access key or use an existing one with write permissions on the target bucket
Enter the Access Key and Secret Key in Zeotap

Required Permissions

The access key needs the following permissions on the target bucket:

s3:PutObject — write files to the bucket
s3:GetBucketLocation — verify bucket region
s3:ListBucket — list objects for verification
s3:DeleteObject — clean up test objects during connection testing

You can assign these via a MinIO policy:


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetBucketLocation",
        "s3:ListBucket",
        "s3:DeleteObject"
      ],
      "Resource": [
        "arn:aws:s3:::your-bucket-name",
        "arn:aws:s3:::your-bucket-name/*"
      ]
    }
  ]
}

Configuration

Field	Type	Required	Description
Endpoint URL	Text	Yes	The MinIO server endpoint (hostname and port, without protocol). For example: `minio.example.com:9000`
Bucket Name	Text	Yes	The MinIO bucket to write files to (e.g., `my-data-bucket`)
Region	Text	No	The region for the MinIO server. Most MinIO deployments use `us-east-1`. Default: `us-east-1`
Use SSL (HTTPS)	Toggle	No	Enable HTTPS for the connection to the MinIO server. Default: On
Path Prefix	Text	No	Optional prefix for all object keys within the bucket. Default: `zeotap/`

Target Settings

Field	Type	Required	Description
File Format	Select	Yes	Output file format: CSV or JSONL (Newline Delimited JSON). Default: CSV
Enable Gzip Compression	Toggle	No	Compress files with gzip before uploading. Default: On

Supported Operations

Sync Modes

Mode	Supported	Description
Insert	Yes	Write the full current dataset as new files on every run
Mirror	Yes	Write changed rows as new files with an `_operation` column (`added` / `changed` / `removed`)
Snapshot	Yes	Write the complete current dataset as new files on every run — no delta, no `_operation` column

Audience Sync Modes

Mode	Supported	Description
Add	Yes	Write new and updated members as new files (departed members excluded)
Mirror	Yes	Write membership changes as new files with an `_operation` column
Snapshot	Yes	Write the complete current membership as new files on every run — no delta, no `_operation` column

Features

Field Mapping: No — MinIO writes all mapped fields as file columns
Schema Introspection: No — MinIO is a file-based destination without schema discovery

How It Works

Zeotap writes data as files to the configured MinIO bucket using the S3-compatible API:

Data is serialized into the selected format (CSV or JSONL)
If gzip compression is enabled, the file is compressed before upload
Files are uploaded to {bucket}/{path_prefix}/{YYYY-MM-DD}/{batch_id}.{ext}[.gz]
Each sync run produces a new set of date-partitioned files
MinIO is accessed via path-style addressing (e.g., http://minio:9000/bucket/key)

Sync mode semantics

Insert writes the full current dataset as new files on every run.
Mirror writes the full dataset on the first run, then only the rows that changed on later runs. Every row in a mirror-mode file carries an _operation column — added, changed, or removed — so downstream consumers can apply updates and deletions. On the first run all rows are marked added. In CSV files _operation is always the last column; treat the name as reserved in mirror mode.
Add (audience syncs) writes only new and updated audience members. Members who left the audience since the previous run are excluded from the files.
Snapshot writes the complete current dataset as new files on every run and never diffs. There is no _operation column (the files are the whole dataset) and unchanged rows are included every time. Records that left are simply absent from the next drop rather than delivered tagged removed, so treat each run’s output as authoritative and replace your prior copy rather than merging into it. Unlike Insert, snapshot is guaranteed to read the full model output on every warehouse: it never adopts native change tracking (Snowflake streams, Databricks CDF), which can only surface deltas.

Rate Limits

MinIO does not impose API rate limits by default. Performance is bounded by the MinIO server’s hardware, network bandwidth, and configuration. For large-scale deployments, consider:

Distributed MinIO with erasure coding for throughput
Network bandwidth between Zeotap and the MinIO cluster
Disk I/O capacity of the MinIO server nodes

Best Practices

Use HTTPS in production: Enable the Use SSL toggle for any non-local deployment to encrypt data in transit
Set a path prefix: Use a descriptive prefix like zeotap/ to organize exported data separately from other objects in the bucket
Use JSONL for nested data: If your data contains complex or nested fields, JSONL preserves structure better than CSV
Enable compression: Gzip compression reduces upload size and storage costs with minimal CPU overhead
Create dedicated credentials: Use a dedicated MinIO access key for Zeotap with minimal required permissions

Troubleshooting

Connection refused

Verify the MinIO endpoint URL is correct and includes the port number (default 9000). Ensure the MinIO server is running and the endpoint is reachable from Zeotap’s network.

Access denied

Verify the access key has s3:PutObject permission on the bucket. Check that the MinIO policy attached to the user or access key grants access to the specific bucket and path prefix.

Bucket not found

Ensure the bucket name is correct and exists on the MinIO server. Bucket names are case-sensitive in MinIO. Create the bucket via the MinIO Console or mc mb command before configuring the destination.

SSL/TLS errors

If your MinIO server uses a self-signed certificate, ensure the certificate is trusted by Zeotap’s runtime environment. For testing, you can disable SSL, but always use HTTPS in production.

Region mismatch

Most MinIO deployments use the default region us-east-1. If your MinIO server is configured with a different region, set the Region field to match. A mismatch causes authentication failures.

Large file uploads timing out

For very large syncs, ensure sufficient network bandwidth between Zeotap and the MinIO server. Consider enabling gzip compression to reduce upload size, or increasing the MinIO server’s request timeout settings.