Data Extract API
Introduction
The Data Extract API provides built-in endpoints for programmatically transferring your solution data into data warehouses and data lakes for further analysis and storage. This API is designed to facilitate seamless integration of your Appfarm solution data into existing data transformation workflows.
Common data warehouse software that you can use with this API includes:
Snowflake
Databricks
Google BigQuery
Amazon Redshift
As a developer, you're responsible for creating and managing your own processes for calling the endpoints and extracting your data. This is typically managed through ETL (extract, transform, load) or ELT (extract, load, transform) software.
Key features
Object class-level export: Data export can be enabled per object class, allowing for targeted data extraction. A unique endpoint is generated for each object class.
Efficient extraction: The Data Extract API endpoints stream data and use cursor-based pagination, enabling efficient extraction of large datasets.
Secure access: Calling a Data Extract API endpoint requires an API key generated within a service account with explicit data export permissions.
Usage
Configuration
Enabling the Data Extract API
Premium feature
The Data Extract API is a premium feature. Availability is determined by your subscription. Please check your current subscription or contact customer success to ensure you have access to this functionality.
To enable the Data Extract API for a specific object class:
In the object class properties, check the Enable Endpoint box to enable data export for that specific object class.
Hover over the blue dot next to Enable Endpoint and note the object class' unique endpoint.
The Data Extract API is available in all environments when enabled in a solution.
Supported data
The API primarily supports exporting data from standard object classes.
File object classes are supported, but only metadata is exported (file content URLs are not included).
Time series data is not supported by this API and requires separate APIs for access.
Pagination
The Data Extract API uses cursor-based pagination:
Each response includes a
next_cursor
for fetching the next page of objects.By default, 1,000 objects are returned per response.
The
limit
query parameter can adjust the number of returned objects (max 50,000).
Making requests
Endpoint structure
Each object class has a unique endpoint for data extraction. The subdomain in the endpoint URL is specific to your deployment environment (e.g., Development, Test, Production).
For Production, the general structure of the endpoint is as follows:
Example of an object class endpoint across environments:
Authentication
Requests to the Data Extract API require a Bearer token for authentication. To set up authentication:
Use an existing service account or create a new one in Appfarm Create.
Ensure the service account has a role with Read permissions for the object class you want to access.
Generate an API key for the service account. Make sure the API key has the Data Export scope.
Use this API key as a Bearer token in your requests for authentication.
For more details on service accounts and API keys, refer to the Service accounts documentation.
Query parameters
cursor
(optional): The ID of an object to determine the starting point for the next set of results.limit
(optional): The maximum number of objects to return (default: 1,000, max: 50,000).
Example request with query parameters:
Response format
The API returns a JSON object containing:
data
: An array of objects (e.g., transactions).next_cursor
: The ID of the next object, or null if there are no more objects.
Example response:
Last updated