Atoti AutoPivot

Atoti AutoPivot is a standalone application for online analysis (OLAP) of CSV files.

Atoti AutoPivot discovers the structure of CSV files, field separator, column names, column types, and loads data in memory with a high throughput parallel CSV source. Atoti AutoPivot exposes the data as a cube with hierarchies and metrics that can be manipulated in the Atoti UI frontend or directly from the Microsoft Excel Pivot Table, using the XMLA protocol and MDX query language.

This project is packaged using Spring Boot 3.5.4 and requires Java 21.

Quick Start

# Build the project
mvn clean install

# Run with sample data
mvn spring-boot:run

# Access the UI
# Open http://localhost:9090/ui in your browser

Dependencies

Java: 21 or higher
Atoti Server: 6.1.13
Spring Boot: 3.5.4
Maven for building the project

Launching Atoti AutoPivot

Building the Project

Build the project with Maven:

mvn clean install

Running the Application

Option 1: Run with Maven (recommended for development)

mvn spring-boot:run

Option 2: Run the JAR file

java -jar target/autopivot-6.1.13.jar

Option 3: Run with custom parameters

java -jar target/autopivot-6.1.13.jar --fileName=./my-data.csv --server.port=8080

Once running, Atoti UI will be available at http://localhost:9090/ui (or the port you configured).

Sample Data

The project includes sample CSV files in the sample-data/ directory:

risks.csv - Financial risk data (default in application.yml)
OfficeSupplies.csv - Office supply orders
Titanic.csv - Titanic passenger data

You can switch between them by editing the fileName property in src/main/resources/application.yml or passing it as a parameter.

Performance

The multithreaded CSV source usually parses CSV data at several hundreds of MB/s. Of course this kind of throughput can only be reached with fast storage, a local SSD drive for instance or network storage accessed through a 10Gbps network at least.

Atoti AutoPivot is powered by the Atoti Server technology, the in-memory analytical platform developed by ActiveViam. Atoti Server runs on all sizes of hardware, from laptops to large servers with hundreds of cores and tens of terabytes of memory. When used in fire and forget mode, AutoPivot targets files up to a few hundreds of gigabytes.

CSV Format

Atoti AutoPivot expects a standard CSV file, with headers (column names) on the first row.

Configuration Options

Configuration options can be set in src/main/resources/application.yml or passed as command-line parameters.

Common Configuration Options:

Required:

fileName - Path to the CSV file to load (mandatory)
```
--fileName=./sample-data/risks.csv
```

Optional:

charset - Character encoding of the CSV file (default: UTF-8)
server.port - HTTP server port (default: 9090)
datastore.partitioningField - Field to use for data partitioning
pivot.cache.size - Size of the ActivePivot LRU aggregate cache (default: 10000)
activeviam.dataexportservice.rootpath - Root path for data exports

Examples:

Run with a different CSV file:

java -jar target/autopivot-6.1.13.jar --fileName=./my-data.csv

Run on a different port:

java -jar target/autopivot-6.1.13.jar --server.port=8080

Combine multiple options:

java -jar target/autopivot-6.1.13.jar --fileName=./data.csv --server.port=8080 --charset=ISO-8859-1

Tweaking The Project

Atoti AutoPivot tries to guess what's in the data and do everything automatically. More generally it illustrates Atoti cubes can be configured programmatically and started on the fly, a very powerful concept for Atoti developers that can be reused beyond the simple usage of AutoPivot.

Here are some entry points to jump into the code, starting from src/main/java:

com.activeviam.apps.autopivot.csv.discover.CSVDiscovery - Logic to discover the CSV separator character and the data types of the columns
com.activeviam.apps.autopivot.cfg.AutoPivotGenerator - Logic to create an Atoti cube (hierarchies, aggregates...) based on the file format
com.activeviam.apps.autopivot.cfg - Package containing the Spring configuration of the AutoPivot application
com.activeviam.apps.autopivot.AutoPivotApplication - Main Spring Boot application class
src/main/resources/application.yml - Configuration options of the AutoPivot application

Licensing

The code of the Atoti AutoPivot application is open source, licensed under the Apache License 2.0. The AutoPivot application depends on the Atoti Server (commercial) software, the Atoti Server jar files distributed by ActiveViam must be available in the maven repository for the application to build. Running the Atoti AutoPivot application requires a license for the Atoti software. To use the Atoti UI frontend, the Atoti license must have the Atoti UI option enabled.