FirstSpirit

The FirstSpirit loader imports content documents from the FirstSpirit CaaS (Content as a Service) platform into your warehouse. Each CaaS collection you select becomes its own table: the FirstSpirit system fields land as columns, and the complete JSON document lands in a data column you can unnest downstream.

Use it to bring released page, dataset and media metadata into the same warehouse as your customer data — so content can be joined to behaviour, modelled, and used in audiences and journeys.

This loader is the read-side counterpart of the FirstSpirit destination. Both use the same CaaS base URL, database and API key, so a single CaaS setup covers both directions.

Prerequisites

• A FirstSpirit CaaS Platform instance and its REST base URL (for FirstSpirit Cloud, your project’s CaaS endpoint).
• The CaaS database name — the tenant/project store that holds your content.
• The names of the CaaS collections you want to import. CaaS Connect creates them as <project-uuid>.<state>.content (and <project-uuid>.<state>.files for media), so a typical value is <project-uuid>.release.content.
• A CaaS API key whose permission allows GET on those collections (see Authentication).
• Content that has been published at least once from FirstSpirit — CaaS collections only exist after FirstSpirit has written into them.
• A connected warehouse to load the data into.
• CaaS Connect 3.83.9 or later is strongly recommended if you plan to use incremental sync. Earlier versions had a defect (CAAS-2536, fixed in 3.83.9) where the changeInfo field on a CaaS document was not correctly updated even though the editorial content was current. Incremental sync keys off changeInfo.revision, so on an older module a changed document can be missed. Check your module version with your FirstSpirit administrator, or use Always Full Refresh.

Authentication

Zeotap authenticates with a CaaS API key, sent as a Bearer token on every request.

1. In CaaS, open the apikeys collection — in the CaaS admin database for a global key, or in your own database for a local key.
2. Create (or pick) a key for Zeotap.
3. Make sure its permission covers the collections you want to import, and that the permission’s methods list includes GET.
4. In Zeotap, add a new FirstSpirit loader and paste the key into the API Key field.

The key must be readable, not just writable

This is the one thing that differs from the FirstSpirit destination. A destination only ever writes, so a key with PUT/PATCH/DELETE in its methods is enough. A loader reads, so:

• A write-only key (no GET in methods) returns a 4xx on every request and the connection test fails.
• A key scoped with permissionMode: PREFIX to a specific collection path can read that collection but cannot list the database’s collections. That is expected and supported: the connection test tolerates a 403 on the database root and verifies read access on each configured collection instead, and collection auto-discovery is best-effort only. You always name your collections explicitly.

The key is stored encrypted and is only used to read from the configured database.

Configuration

Choosing collections

CaaS Connect names the collections it maintains after the FirstSpirit project UUID plus the publishing state:

.preview.content holds unpublished drafts. Importing it loads content your editors have written but have not released — including copy, prices and offers that were never meant to be visible. If you use preview data for a content-QA table, keep it in a separate loader and do not model it alongside released content.

apikeys and gql-apps are reserved by CaaS and are never offered as streams; apikeys in particular holds credentials.

Copy collection names verbatim from CaaS. A misspelled name is not silently ignored — the run fails with a “collection does not exist” error, because for a loader an absent collection means there is nothing to read.

Available Streams

Each collection you configure becomes one stream and one table. If your API key is broad enough to list the database’s collections, Zeotap also suggests the ones it finds; if it is not (the common case for a scoped read key), only the collections you named are offered.

A CaaS collection is schemaless — page references, datasets, media and project properties live side by side, and your own FirstSpirit templates add arbitrary fields. So every collection lands with the same stable column set: a header of FirstSpirit system fields, plus the whole document.

Every column is nullable. Which system fields a document carries depends on its type: a page reference has uid and route but no schema; a dataset has schema, entity_type and locale_* but no uid; media metadata has neither route nor locale. change_revision is nullable too — see Sync Modes.

The interesting content — formData, metaFormData, template, page, and everything your templates define — is in data. Extract it downstream with your warehouse’s JSON functions. For example, in BigQuery:

SELECT
  _id,
  display_name,
  locale_language,
  JSON_VALUE(data, '$.page.formData.pt_headline.value') AS headline,
  route,
  change_revision
FROM <your_collection_table>
WHERE fs_type = 'PageRef'

routes is JSON text as well, so it unnests the same way (JSON_QUERY_ARRAY(routes) in BigQuery, PARSE_JSON(routes) in Snowflake, routes::jsonb in Postgres).

Sync Modes

How incremental sync works

FirstSpirit stamps every CaaS document with changeInfo.revision: the repository-wide revision number, which increases monotonically as content is published. It is the cursor:

1. Each run reads the revision it stopped at last time.
2. It asks CaaS only for documents whose changeInfo.revision is greater than that value — so a document read once is never read again.
3. When the run finishes, the new high-watermark is the highest revision it saw.

A single FirstSpirit publish bumps the repository revision once, which means many documents share one revision value. Zeotap therefore keeps the filter fixed for the whole run and only records the new watermark once every page of a collection has been read. That is deliberate: recording a partial position mid-run could leave the rest of a shared revision permanently filtered out.

The caveat: documents without changeInfo

changeInfo is optional on a CaaS document. A document that does not carry it can never satisfy a “greater than” comparison, so it is invisible to incremental sync — permanently, not just for one run.

This matters in three situations:

• Documents written to CaaS by something other than CaaS Connect (a custom importer, a migration script, a GraphQL app).
• Collections created before changeInfo was populated.
• CaaS Connect older than 3.83.9, where changeInfo was not always updated even for current content (CAAS-2536).

Handle it one of two ways:

• Turn on Always Full Refresh and set the loader’s sync mode to Full refresh. Every run reads every document and replaces the table. This is the safe default when you are not sure. Stream discovery then stops offering a cursor field — but this does not change the loader’s own sync mode, and it does not delete a cursor already stored. Set the loader’s sync mode to Full refresh as well. If you leave it on Incremental with this toggle on, no watermark is ever recorded and every run appends another complete copy of the collection.
• Run incremental, plus a periodic full refresh. Keep incremental for freshness and schedule a second, full-refresh loader (weekly, say) over the same collections as a reconciliation pass.

Deletions are also worth noting: an element removed in FirstSpirit disappears from CaaS, and there is no tombstone document, so an incremental run cannot see the removal. Only a full refresh drops the row.

How It Works

• Paging. Zeotap reads each collection page by page (up to 100 documents per request, the CaaS maximum), sorted by revision and then document id so page boundaries are stable. It stops when a page comes back empty or shorter than the page size.
• Batching and checkpointing. Each page is written to your warehouse as a batch, and the incremental watermark for a collection is recorded once that collection has been read in full. Streams are read independently, so one collection’s failure does not lose another’s progress.
• Restarts. If a run fails partway, the next run restarts that collection from the last recorded watermark. Incremental runs append, so a restart can re-deliver rows that were already written — deduplicate on _id (and change_revision) in your models if that matters.
• Sorting needs an index. The read sorts and filters on changeInfo.revision. On a large collection without a MongoDB index on that field, CaaS may answer 408 (see Troubleshooting).

Rate Limits

CaaS does not publish a rate limit, because it is normally deployed per customer or per tenant — the throughput you get depends on your own instance’s sizing, and on whether it is FirstSpirit Cloud or self-hosted.

In practice:

• The loader issues one request per 100 documents, so a 50,000-document collection is ~500 requests per full refresh. Incremental runs are far smaller.
• If your instance is shared with the live storefront or a rendering front end, schedule the loader outside peak hours, or ask your administrator whether a dedicated CaaS read endpoint is available.
• A 429 response is surfaced with a clear message; reduce the schedule frequency, or lower Page Size if your instance struggles with large pages.
• Ask your FirstSpirit administrator for the limits that apply to your deployment before scheduling a frequent full refresh over a large collection.

Table names

The warehouse table is derived from the collection name, so 3ddbe70b-1c6c-….release.content becomes _3ddbe70b_1c6c_…_release_content — dots and hyphens become underscores, and a name starting with a digit gains a leading underscore. Check the table list after the first run rather than guessing the name.

Troubleshooting

Next Steps

• Create a Model over the loaded content table.
• Build an Audience that joins content to customer behaviour.
• Push audiences or content back with the FirstSpirit destination.