ilum/mageai repository overview

⁠Ilum Mage AI

⁠Overview

The ilum/mageai image extends the official Mage AI platform with native Spark integration, enabling seamless data pipeline orchestration and transformation workflows on Ilum.

• Spark Integration — pre-installed PySpark with SQL and Connect modules
• Data Platform — modern data pipeline tool with notebook-style interface
• Java Runtime — includes OpenJDK 17 JRE for Spark execution
• Multi-arch support — built for linux/amd64, linux/arm64, and linux/arm/v7

⁠TL;DR / Quick Start

# Run Mage AI with default configuration
docker run -d \
  --name ilum-mageai \
  -p 6789:6789 \
  -v $(pwd)/mage_data:/home/src \
  ilum/mageai:<version>

Access the UI at http://localhost:6789 and start building data pipelines.

⁠Supported Tags

• <version>: Mage AI 0.9.76 + PySpark + Java 17

⁠Environment Variables

⁠Core Configuration

Mage AI configuration is primarily handled via io_config.yaml and metadata.yaml files. Standard Mage AI environment variables are supported.

⁠Spark Configuration

PySpark can be configured through standard Spark environment variables:

⁠Data Persistence (Volumes)

# docker-compose example
volumes:
  - ./mage_project:/home/src

TIP: Mount your Mage project directory to /home/src to persist pipelines, data files, and configuration across container restarts.

⁠Network Configuration

⁠Security

⁠Non-root Architecture

The image inherits the user configuration from the base mageai/mageai image. When mounting host volumes, ensure appropriate permissions are set.

⁠Secret Hygiene

CAUTION: Never pass sensitive credentials directly in environment variables!

Recommended methods:

1. Kubernetes Secrets — mount credentials as files or environment variables.
2. Mage Secrets — use Mage AI's built-in secrets management for pipeline credentials.

⁠Vulnerability Scanning

Before production deployment, scan the image:

# Trivy
trivy image ilum/mageai:<version>

⁠Building the Image

Run command from the docker/ilum-mageai directory:

docker buildx build \
  --platform linux/amd64,linux/arm/v7,linux/arm64 \
  -t ilum/mageai:<version> \
  --push .

⁠Image Verification

⁠Layer Analysis (Dive)

dive ilum/mageai:<version>

⁠Software Bill of Materials (SBOM)

# Syft
syft ilum/mageai:<version> -o spdx-json > sbom.json

⁠Image Contents — Details

⁠Dockerfile Components

• Base: mageai/mageai:0.9.76
• Java Runtime: OpenJDK 17 JRE (headless)
• PySpark: Latest version with sql and connect modules

⁠Support and Feedback

• Issues: Bug Tracker⁠
• Documentation: Ilum Docs⁠
• Slack: Join our Community⁠