SAP S/4HANA Loader

The SAP S/4HANA loader pulls master and transactional data out of SAP S/4HANA through its OData APIs and lands it in your warehouse: business partners and their addresses, sales order headers and line items, and product (material) master data.

It is the read-side twin of the SAP S/4HANA destination and shares the same credential shape — one communication arrangement configures both directions.

Every column this loader creates comes from SAP’s published EDMX service metadata, not from prose documentation, so property names, types, nullability and decimal precision match what your system actually returns.

Prerequisites

• An SAP S/4HANA system (Cloud or on-premise) reachable over HTTPS from Zeotap.

• The three OData services activated and exposed:

  • API_BUSINESS_PARTNER
  • API_SALES_ORDER_SRV
  • API_PRODUCT_SRV

  On S/4HANA Cloud these are published through a Communication Arrangement. On-premise, activate them in transaction /IWFND/MAINT_SERVICE.

• A communication user (or an OAuth 2.0 client) with read authorization on the entity sets you plan to sync. The loader only ever issues GET requests — it never writes, and it never fetches an X-CSRF-Token, because CSRF protection applies to modifying requests only.

• A connected Warehouse with write permissions on the target schema.

Authentication

Two methods are supported, selected with the Authentication Method field.

Basic Authentication

1. Create a communication user in your S/4HANA system (S/4HANA Cloud: Maintain Communication Users).
2. Assign it a communication arrangement that exposes the three APIs above with the inbound direction and read authorization.
3. Enter the Username and Password in Zeotap.

Basic authentication cannot be refreshed — if the password is rotated or the user is locked, the run fails immediately with an HTTP 401 rather than retrying.

OAuth 2.0 Client Credentials

1. In SAP BTP (or the communication arrangement’s OAuth 2.0 section) create an OAuth client with the client_credentials grant.
2. Copy the Client ID, Client Secret and Token URL (typically https://<subdomain>.authentication.<region>.hana.ondemand.com/oauth/token).
3. Enter all three in Zeotap.

The access token is minted once per run and cached. If the token expires mid-run, SAP answers a page with HTTP 401; the loader re-mints the token once and replays the same page, so a long extract is not lost to an expiring token.

Configuration

OData version

The built-in stream catalog targets the OData V2 services (/sap/opu/odata/sap/API_…), whose entity sets are A_-prefixed — A_BusinessPartner, A_SalesOrder, A_SalesOrderItem, A_Product, A_BusinessPartnerAddress. This is deliberate and it is the honest position: the OData V4 equivalents live under /sap/opu/odata4/… with service-group paths that vary per system and release, and their entity sets are not A_-prefixed, so a single hard-coded V4 catalog would be wrong for most systems.

If you need to read from a V4 service, set OData Version to v4 and supply the Service Path Override. The loader then:

• sends OData V4-shaped cursor literals (bare ISO-8601 with a Z, no datetime'…' prefix),
• reads the V4 response envelope (value + @odata.nextLink).

The entity-set name still comes from the selected stream, so a V4 service whose entity set is named differently will return HTTP 404. In that case the built-in catalog cannot serve that service, and a Generic REST API loader pointed at the entity set is the right tool.

Available Streams

Default selected streams: Business Partners, Sales Orders, Products.

Each stream declares the full property set of its entity type — 44 to 77 columns — and $select requests exactly those, so nothing is fetched that the target table has no column for.

Column types

The loader converts each value against its declared column type before emitting, because the OData wire formats do not match warehouse types:

Three consequences worth knowing:

• CreationTime and LastChangeTime are strings, not times. They are Edm.Time — a duration since midnight — so they land as PT13H20M-style text. Combine them with CreationDate / LastChangeDate in your own model if you need a full timestamp.
• IsNaturalPerson and VarblPurOrdUnitIsActive are strings, not booleans. SAP types them Edm.String (a one-character flag) despite the names, and this loader declares what the metadata says.
• An unset SAP date becomes NULL. SAP renders the initial ABAP date 0000-00-00 as /Date(-62135596800000)/; that sentinel becomes NULL rather than a year-1 date. Genuine pre-1970 dates (birth dates) are negative millisecond values and are converted correctly.

Sync Modes

Full refresh replaces the target table on every run and works for every stream.

Incremental appends rows whose change stamp is at or after the stored high-watermark, and is available for three of the five streams:

For the two full-refresh-only streams: ValidityStartDate / ValidityEndDate on A_BusinessPartnerAddress are business validity dates — when the address is valid for the partner — not a record-modification stamp. Filtering on them would silently omit edits to currently-valid addresses, so the loader does not offer them as a cursor. If address or line-item volume makes a full refresh expensive, schedule those two streams less often than the others (a separate loader per cadence).

Newly created records, and how this loader handles them

SAP leaves a business partner’s change stamp initial (blank) until the record is first edited, so filtering on LastChangeDate alone would make a created-but-never-edited partner permanently invisible to incremental runs.

The loader closes that gap: for business_partner the incremental filter is a disjunction over both stamps —

$filter=(LastChangeDate ge datetime'…' or CreationDate ge datetime'…')

— and the high-watermark is the maximum across both, so a record is picked up whichever stamp moved. Both properties are filterable in the published service metadata.

sales_order and product use LastChangeDateTime, a technical UTC change stamp that SAP maintains from creation, so they need no equivalent.

A periodic full refresh is still worth scheduling — not for this reason, but for the two below.

Two further limitations that apply to every incremental stream here:

• Deletes are invisible. OData exposes no tombstone for these entity sets, so a business partner or sales order deleted in SAP stays in your warehouse until the next full refresh. A_Product.IsMarkedForDeletion is a soft deletion flag and is loaded as a column, so soft-deleted products are detectable.

• The filter is inclusive (ge, not gt). SAP change stamps are not unique, so an exclusive filter would permanently drop every row sharing the watermark instant with the previous run’s last row. The trade-off is that boundary rows are re-read.

• Incremental runs append; they do not merge. Re-read boundary rows are inserted again, so the raw table can hold more than one version of a record. Nothing deduplicates them for you. Do it in your modelling layer, keeping the latest version per key:

  SELECT * FROM raw_business_partner
  QUALIFY ROW_NUMBER() OVER (
    PARTITION BY BusinessPartner ORDER BY LastChangeDate DESC
  ) = 1

  This matters most for business_partner, whose cursor has day granularity — a daily schedule re-reads the whole boundary day on every run.

How It Works

1. Authentication. Basic credentials are attached to every request, or an OAuth client-credentials token is minted once and sent as a Bearer header. Both are re-applied on next-link requests, so a server-supplied continuation URL is never followed unauthenticated.
2. Request shape. Every request carries Accept: application/json and $format=json — OData V2 defaults to XML, so omitting either returns a document the loader cannot read.
3. Paging. The primary loop is $filter + $orderby … asc + $top + $skip. If SAP returns a next link (d.__next in V2, @odata.nextLink in V4) the loader follows it instead of computing the next $skip: SAP caps page size server-side, so a page shorter than $top is not by itself the end of the collection. The chain terminating — or a genuinely short page when no link was ever offered — ends the stream.
4. Stable ordering. $orderby is the cursor ascending followed by the entity’s key fields. $skip is an offset into a result set, so rows tied on the cursor could otherwise be reshuffled between requests and be duplicated or skipped at a page boundary.
5. No $inlinecount / $count. Neither is ever requested — each would force SAP to compute a full COUNT alongside every page, which on a large sales-order table costs more than the page itself.
6. Batching and memory. Each page is converted and emitted in sub-batches of 500 rows and then released. The loader never buffers a whole result set, so a multi-million-row extract runs in constant memory.
7. Cursor checkpointing. The connector reports the highest change stamp it has emitted; the loader pipeline persists it only after the rows are durable in the warehouse. A run that fails midway resumes from the last written position, never from an over-optimistic one.
8. Cursor literals are version- and type-correct. OData is unforgiving here, and a wrong literal is worse than an error — it can be ignored:
   • V2 Edm.DateTime: datetime'2024-03-01T00:00:00' — no Z; appending one yields HTTP 400.
   • V2 Edm.DateTimeOffset: datetimeoffset'2024-03-01T10:11:12.1234567Z'.
   • V4: a bare literal with no prefix and no quotes.

Rate Limits

SAP publishes no request-rate limit for the S/4HANA OData APIs — the constraint is your system’s own capacity, and on-premise systems in particular share ICM work processes with interactive users.

Practical guidance:

• The loader issues one request per page, sequentially, with no concurrency inside a stream. A 1,000-row page size means one request per 1,000 rows.
• S/4HANA Cloud enforces a per-user session / dialog-work-process ceiling rather than a request count. Give the loader its own communication user so a large extract cannot starve other integrations.
• On-premise, point the loader at a dedicated application server or gateway node if one is available, and schedule large full refreshes outside business hours.
• Lower Page Size if you see gateway timeouts; raise it (up to 5000) to reduce round trips on a fast system. SAP silently caps the page below your $top when its own limit is lower and signals the remainder with a next link, which the loader follows — so an over-large Page Size costs correctness nothing.

Troubleshooting

authentication failed (HTTP 401): the server returned an HTML page rather than an OData error Basic authentication was rejected and SAP returned its logon page instead of an OData envelope. Check the username and password, and check the communication user is not locked. Basic authentication is never retried — the same credentials would fail again.

access denied (HTTP 403) — the communication user is authenticated but lacks read authorization for this entity set The credentials are valid but the user has no read authorization on this entity set. On S/4HANA Cloud, confirm the communication arrangement includes the API and that the inbound service is marked as active. On-premise, check the authorization object for the service.

OData service not found at /sap/opu/odata/sap/API_BUSINESS_PARTNER The service is not activated. On-premise, add it in /IWFND/MAINT_SERVICE. On Cloud, add the API to the communication arrangement. If you set a Service Path Override, verify the path — including the release segment (…/0001) on V4 service groups.

OData request failed (HTTP 400) mentioning a filter or property Usually one of three things: an incremental run against a V4 service still configured as v2 (or the reverse — the cursor literal syntax differs and SAP rejects the wrong one); a Service Path Override pointing at a service whose entity set does not carry the cursor property; or a system where the property has been made non-filterable by an extension. Run the stream as a full refresh to confirm, then fix the version or the path.

OData request failed (HTTP 404) mid-run, after the connection test passed The Service Path Override is set and applies to every selected stream, so the second stream’s entity set does not exist in that service. Create one loader per overridden stream, or clear the override to use the per-stream defaults.

Incremental runs return zero rows even though records changed Three causes, in order of likelihood: (1) the cursor is at or past the newest change stamp — check the stream’s last cursor value on the loader’s Runs tab; (2) for business_partner, the changed records have a blank LastChangeDate (see the incremental risk) — run a full refresh; (3) system time skew, if the SAP system’s UTC clock is behind the watermark you already stored.

New records never appear, but edits to old records do On business_partner this should not happen — the incremental filter ORs CreationDate in alongside LastChangeDate precisely so never-edited records are still picked up. If you do see it, confirm your service exposes CreationDate as filterable (a customer extension can mark a property non-filterable), and check whether a service_path override is pointing at a service where it is absent. Run a full refresh to recover the missing rows.

Numeric columns are NULL in the warehouse but have values in SAP Amounts and quantities arrive as quoted JSON strings in OData V2 ("1234.567"). The loader parses them into real numbers; a NULL means the value was not parseable at all. Check whether an extension field has been overlaid on the property, and confirm the OData Version matches the service.

Timestamps land as 1970-01-01 or NULL /Date(0)/ is a genuine 1970-01-01 in SAP. NULL means the value was the initial ABAP date 0000-00-00 (/Date(-62135596800000)/), which is a sentinel for “not set” and is intentionally not stored as a year-1 date.

A run is slower than expected on a large table Raise Page Size toward 5000 to cut round trips, and make sure the stream is running incrementally where it can. A_SalesOrderItem has no cursor, so it is always a full table scan — split it into its own loader on a slower schedule.

Gateway timeouts (HTTP 504 or a dropped connection) on the first page The $orderby on a very large table can be expensive if the underlying field is not indexed. Lower Page Size, and involve your SAP Basis team about an index on the change-stamp field.

Next Steps

• Model the raw tables into your customer schema — see Models.
• Join business_partner with sales_order on SoldToParty = BusinessPartner to build purchase-behaviour traits.
• Push segments back into SAP with the SAP S/4HANA destination.