You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the interest of simplifying a deployment, we should add an integrated OLAP database (DuckDB) to the aggregator. Emitted events should be ingested into this DB. This makes any kind of downstream system optional for analytics. The aggregator shoud offer an API endpoint like:
GET aggregator:8080/api/export?from=2025-01-01&to=2025-02-01&metrics=bytesin,bytesout,storage
Content-negotiation should be used to determine the exact format of the data export, with CSV and JSON(L) being possible options. This would allow customers to download data for the relevant billing period and generate bills/reports via e.g. Jupyter by talking to the aggregator directly. Operators will no longer need to maintain a dedicated database deployment.
It should be possible to deactivate this OLAP module, in case users prefer collecting aggregated data in an external system.
The text was updated successfully, but these errors were encountered:
arov00
changed the title
Load data into an integrated OLAP database. Allow downloading a CSV or JSONL data report via API endpoint
Load data into an integrated OLAP database. Allow downloading a CSV or JSONL data export via API endpoint
Jan 20, 2025
In the interest of simplifying a deployment, we should add an integrated OLAP database (DuckDB) to the aggregator. Emitted events should be ingested into this DB. This makes any kind of downstream system optional for analytics. The aggregator shoud offer an API endpoint like:
Content-negotiation should be used to determine the exact format of the data export, with CSV and JSON(L) being possible options. This would allow customers to download data for the relevant billing period and generate bills/reports via e.g. Jupyter by talking to the aggregator directly. Operators will no longer need to maintain a dedicated database deployment.
It should be possible to deactivate this OLAP module, in case users prefer collecting aggregated data in an external system.
The text was updated successfully, but these errors were encountered: