Developer Documentation

Enterprise REST API & Cloud Ingestion

HLP Data delivers verified datasets through programmatic REST APIs and automated Amazon S3 cloud synchronization. Feeds are pre-audited against standardized schemas and updated on predictable schedules.

Commercial API Status: Active / Enterprise by Request

Base URL: https://api.hlpdata.com/v1

Request API Access

1. Authentication

All API requests must include a valid commercial API key in the HTTP Authorization header using the Bearer scheme:

HTTP Request Header

Authorization: Bearer hlp_live_xxxxxxxxxxxxxxxxxxxxxxxx

Accept: application/json

2. Core Endpoints

Endpoints return deterministic JSON responses structured according to our verified marketplace schemas.

GET /v1/datasets

Catalog Index

Returns metadata for all available datasets, latest observation timestamps, total record counts, and schema definitions.

GET /v1/datasets/:id/records

Paginated Records

Retrieve filtered, paginated records. Query parameters support limit (default 100, max 1000), offset, observed_after, and field-level filters.

GET /v1/datasets/:id/latest-export

Pre-signed S3 Download

Generates a temporary, pre-signed AWS S3 URL for downloading the complete latest census snapshot in CSV or Parquet format.

3. Implementation Examples

Easily integrate HLP Data feeds into Python, Polars, Pandas, or shell automation scripts:

Python (Polars / Requests)

import requests

import polars as pl

# Authenticated feed request

headers = {"Authorization": "Bearer hlp_live_YOUR_KEY"}

res = requests.get("https://api.hlpdata.com/v1/datasets/continente-pt/records?limit=1000", headers=headers)

df = pl.DataFrame(res.json()["data"])

# Analyze retail markdowns

print(df.select(["product_name", "price", "discount_pct"]))

Preferred Enterprise Delivery: Direct S3 Data Lake Sync

For hedge funds and enterprise analysts consuming high-volume datasets (500K+ rows), direct S3 bucket synchronization eliminates API rate limits and HTTP timeout risks.

Automated push to buyer bucket: s3://buyer-data-lake/hlpdata/<DATASET>/<DATE>/data_file.csv
Automated receipt handshake (approved.txt verification).
SHA-256 checksums and UTF-8-SIG byte order validation included with every delivery.

Configure S3 Cloud Delivery