In-Warehouse Log
Zeotap writes every operational outcome to an observability log table in your own data warehouse. Because the history lives in your warehouse, you can query it with SQL, join it against your other data, and build dashboards in whatever BI tool you already use — without leaving your data platform.
The in-warehouse log is on by default. There is nothing to enable.
Where the Log Lives
The log is written to the observability_events table in the audit_logs schema of your warehouse.
Zeotap writes it to your workspace’s configured event-log warehouse. If you have not designated one, it uses your first connected warehouse.
What Gets Logged
One row is written for each operational outcome across the platform — sync, journey, loader, store feed, realtime audience snapshot rebuild, identity graph, and trait runs, plus event delivery and contract activity. Both successful completions and failures are recorded, along with:
- Runs that stall and are later detected as failed
- Consecutive-failure signals
- Automatic-pause signals
Table Schema
Each row describes one observability event:
| Column | Description |
|---|---|
occurred_at | When the event happened |
event_type | The kind of event (for example, a run completion or failure) |
severity | Relative importance of the event |
resource_type | The type of resource (sync, journey, loader, store_feed, audience_snapshot, identity_graph, trait, and so on) |
resource_id | Identifier of the specific resource |
resource_name | Human-readable name of the resource |
run_id | Identifier of the run this event belongs to |
status | Outcome of the run — completed, failed or cancelled |
error_message | Failure detail, when the event represents a failure |
metrics | Structured metrics for the run (row counts, duration, rejected rows, and so on), stored as JSON |
attributes | Additional structured context, stored as JSON |
Column Values
status is completed, failed or cancelled for a run; pause, resume and destination events carry the resource’s own state instead (paused, active, auth_expired or connected). severity is info, warning or critical.
resource_type is one of sync, realtime_sync, journey, loader, store_feed, identity_graph, trait, event_forwarding, contract, connection, warehouse or audience_snapshot. A connection row’s resource_id is a destination’s id.
event_type names the signal itself:
| Event type | What it records |
|---|---|
run.completed | A run finished cleanly |
run.failed | A run failed |
run.cancelled | A run was cancelled |
run.zombie | A run stopped heartbeating and was reclaimed |
run.deferred | A batched journey run was spilled to its journey’s next wake |
sync.consecutive_failures | A sync’s failure streak crossed its threshold |
sync.auto_paused | A sync was paused automatically after repeated failures — a batch sync (sync) or a realtime sync (realtime_sync) |
sync.resumed | A realtime sync paused automatically, whose pause was reported, was resumed. Written within about a minute of the resume |
sync.rows_capped | A run suppressed rows under a frequency cap |
connection.auth_expired | A destination’s stored credential stopped authenticating. Recorded once, when it happens, not on every later attempt |
connection.restored | A destination became connected — after its credential expired, after an error, or on its first successful connection test |
event.delivery_failed | An event delivery failed |
event.volume | Event volume for a window |
contract.violation | An event breached a contract |
snapshot.rebuilt | A realtime audience’s membership was rebuilt and served |
snapshot.held | A realtime audience’s new membership was set aside as unusually large |
journey.wake_skipped | A journey was due but could not start — an active run, or the workspace concurrency cap |
warehouse.reclaim_failed | Deleting a resource could not drop the warehouse objects it owned |
warehouse.objects_reaped | A reaper sweep dropped orphaned warehouse objects |
A realtime sync’s sync.auto_paused row is written within about a minute of the pause, and its occurred_at is when the sync was paused. Its error_message is the reason shown on the sync’s page, its metrics carry consecutive_failures, and its attributes carry the sync’s destination_id and audience_id. It does not carry the destination’s error response: look at the sync’s failures for that.
Realtime audiences write at two granularities under the one audience_snapshot resource type. run.* events carry the workspace id, because one rebuild scans for every audience at once; snapshot.rebuilt and snapshot.held carry the audience id, with that audience’s membership size and its enter and exit counts under metrics.
Example Queries
Failures in the last 24 hours, most recent first:
SELECT occurred_at, resource_type, resource_name, error_message
FROM audit_logs.observability_events
WHERE status = 'failed'
AND occurred_at >= CURRENT_TIMESTAMP - INTERVAL '24' HOUR
ORDER BY occurred_at DESC;Run counts by resource type and status over the last week:
SELECT resource_type,
status,
COUNT(*) AS runs
FROM audit_logs.observability_events
WHERE occurred_at >= CURRENT_TIMESTAMP - INTERVAL '7' DAY
GROUP BY resource_type, status
ORDER BY resource_type, status;The metrics and attributes columns are JSON — use your warehouse’s JSON functions to extract individual values (for example, row counts or duration).
Sync Audit Tables
observability_events records one row per run. For the detail of an individual sync run, down to each record it sent, Zeotap also writes sync audit tables to the cdp_audit schema of the warehouse the sync reads from. On BigQuery, that is the cdp_audit dataset in the source’s project, or in the working dataset’s project when the source has one.
| Table | One row per | What it holds |
|---|---|---|
cdp_audit.sync_runs | Sync run | Run outcome, timing and row counts |
cdp_audit.sync_changelog | Record sent in a run | Whether each record was delivered, and the error when it was not |
cdp_audit.sync_snapshot | Record currently in the destination | The last run that sent each record, and when |
Model and audience syncs write to all three tables. Journey destination sends write a row per send to sync_runs (with the journey’s id as sync_id) and record the members a frequency cap held back in sync_changelog, but do not record per-record delivery results or update sync_snapshot. Profile store feeds write to their own tables in the same schema: feed_runs, feed_changelog and feed_snapshot.
The tables are created the first time a sync runs from that warehouse source, so they do not exist until then. Snowflake stores the names in upper case (CDP_AUDIT.SYNC_RUNS). Every other warehouse uses lower case.
sync_runs
| Column | Description |
|---|---|
run_id | Identifier of the run, as shown on the sync’s run history |
sync_id | Identifier of the sync (for a journey send, the journey) |
sync_type | model, audience, journey or campaign |
workspace_id | Identifier of the workspace |
status | running while the run is in progress, then completed, failed or cancelled |
started_at / completed_at | When the run started and finished |
rows_added / rows_updated / rows_deleted | Records added, changed and removed in the destination |
total_rows | Records the destination accepted. Failed and suppressed records are not counted, so do not use it as the number of records attempted |
batches_total / batches_failed | Batches sent to the destination, and how many of them failed |
error_message | Why the run failed, when it did |
triggered_by | Who or what started the run, when recorded |
sync_changelog
| Column | Description |
|---|---|
run_id / sync_id | The run and the sync the record belongs to |
batch_number | The batch the record was sent in, or 0 for a record a frequency cap held back |
primary_key_value | The record’s primary key |
operation | added, changed or removed |
status | success, failed, or suppressed for a record that was not sent because its address is suppressed or a frequency cap held it back |
error_message | The destination’s error for a failed record, or why a record was suppressed (for example suppressed by frequency cap) |
created_at | When the entry was written |
sync_snapshot
| Column | Description |
|---|---|
sync_id | The sync |
primary_key_value | The record’s primary key |
last_operation | added or changed |
last_run_id | The last run that sent the record |
last_synced_at | When that run sent it |
A row_data column exists on sync_changelog and sync_snapshot but is always empty: the audit records which records were sent and how each one fared, not their contents.
Example Queries
The last ten runs of one sync:
SELECT run_id, status, started_at, completed_at, total_rows, error_message
FROM cdp_audit.sync_runs
WHERE sync_id = '<sync id>'
ORDER BY started_at DESC
LIMIT 10;The records a run failed to deliver, with the destination’s error:
SELECT primary_key_value, operation, error_message
FROM cdp_audit.sync_changelog
WHERE run_id = '<run id>'
AND status = 'failed';When a whole batch is refused by the destination, no sync_changelog rows are written for that batch, so an empty result does not prove a run had no failures. Check the run’s batches_failed and error_message in sync_runs as well.
On BigQuery, qualify the dataset with your project when it is not your default, for example `my-project.cdp_audit.sync_runs`.
Best Practices
- Build a health dashboard — Point your BI tool at
audit_logs.observability_eventsfor an always-current view of pipeline health, entirely within your own warehouse. - Keep it for trend analysis — Because the log persists in your warehouse, it is well suited to long-range trend analysis that live alerting does not cover.
- Pair it with alerting — Use the Alerting section for real-time notification, and the in-warehouse log for investigation and reporting after the fact.
Next Steps
- External Export (OTLP) — Stream telemetry to your monitoring platform
- Alerting — Turn these events into real-time notifications