ubuntu/airflow

Verified Publisher

By Canonical

Updated about 1 month ago

Ubuntu rock for Apache Airflow - a workflow management platform for data engineering pipelines.

Image
Integration & delivery
Machine learning & AI
Data science
0

3.6K

ubuntu/airflow repository overview

ubuntu/airflow is a rock for airflow maintained by Canonical.

Apache Airflow is a platform to programmatically author, schedule, and monitor workflows. This rock packages Airflow built from source on Ubuntu 24.04 with Python and a wide set of providers pre-installed (Amazon, GCP, Kubernetes, MySQL, Postgres, and more), so common integrations work out of the box without requiring additional pip installation inside the container.

[Learn more about airflow]

Tags and Architectures

NOTE: These tags follow a specific naming convention that respects the concept of channels.

TagsSupported untilVersionBaseArchitectures
3.1-24.04_edge10/20263.124.04arm64, amd64
View deprecated tags
TagsSupport ended onUpgrade Path
3-24.04_edge04/2026 -

Get started

This rock's entrypoint is Pebble, a container-optimized init process that enables the orchestration of a collection of local service processes as an organized set.

[Learn more about Pebble]

Inspect

To view the Pebble services and checks defined in this rock:

docker run --rm ubuntu/airflow:3-24.04_edge plan
Run

To run this rock, use any of the usual container runtimes, for example, docker:

docker run --name airflow-container \
    -p 8080:8080 -e AIRFLOW_HOME=/opt/airflow -v /path/to/dags:/opt/airflow/dags -v /path/to/airflow.cfg:/opt/airflow/airflow.cfg \
    ubuntu/airflow:3-24.04_edge

Pebble (PID 1) launches the airflow service, which runs airflow standalone. This initializes a local SQLite metadata database, creates a default admin user, and starts the webserver and scheduler in a single process group (intended for development use). The generated admin password is printed in the container logs. Access the Airflow web UI at http://localhost:8080.

Airflow service

The airflow service is configured with:

command: /usr/bin/airflow standalone
startup: enabled

airflow standalone initializes a local metadata DB, creates a default admin user, and starts the webserver, scheduler, and other core components in one process group (intended for development).

Configuration & State
  • Runtime state (AIRFLOW_HOME) lives at /opt/airflow. This includes airflow.cfg, airflow.db (Database by default for standalone), logs, and dags/.
  • It is required to mount volumes to persist that data or to inject custom dags. For example: -v ./dags:/opt/airflow/dags -v ./airflow.cfg:/opt/airflow/airflow.cfg
Providers

The rock is built from Airflow with many providers pre-installed (Amazon, GCP, Kubernetes, MySQL, Postgres, etc.), so common integrations work out of the box without requiring pip install inside the container.

Full Configuration Reference:

Airflow offers a large number of configuration options that can be set via environment variables or airflow.cfg. For the complete list of configurable settings, consult the official configuration reference

Get logs

To view the logs of the running container, run:

docker exec airflow-container pebble logs

Or, for a specific service:

docker exec airflow-container pebble logs <service>
Monitor health checks

Rocks may have predefined health checks that you can list by running:

docker exec airflow-container pebble checks

The overall health of your container can be inspected via:

docker exec airflow-container pebble health

Configuration

Environment variables
OptionDescriptionDefault
AIRFLOW_HOMERoot directory for Airflow state inside the container (DB, logs, DAGs, config)./opt/airflow
AIRFLOW__CORE__LOAD_EXAMPLESLoad the example DAGs shipped with Airflow. Disable in production environments.None
AIRFLOW__CORE__EXECUTORExecutor class used by Airflow. Supported values include LocalExecutor, CeleryExecutor, KubernetesExecutor, or the full import path to a custom executor class.None
AIRFLOW__DATABASE__SQL_ALCHEMY_CONNSQLAlchemy connection string for the Airflow metadata database. airflow standalone defaults to local SQLite; override for a production Postgres or MySQL database.None
AIRFLOW__CORE__AUTH_MANAGERAuth manager class. Refer to the Airflow Auth Manager documentation for available options and required configuration.None
AIRFLOW__API__SECRET_KEYSecret key for the API server. Use a random value; all instances in a cluster must share the same key.None
AIRFLOW__API__BASE_URLBase URL of the API server.None
AIRFLOW__CORE__MAX_ACTIVE_TASKS_PER_DAGMaximum number of task instances allowed to run concurrently per DAG run.None
Volumes
OptionDescriptionDefault
-v <path>:/opt/airflow/dagsMount a local DAGs directory. Files are auto-discovered by the scheduler.None
-v <path>:/opt/airflow/logsPersist task logs outside the container.None
-v <path>:/opt/airflow/airflow.cfgInject a custom airflow.cfg in place of the auto-generated one.None
-v <path>:/requirements.txt(Optional) Mount additional Python dependencies to install at runtime.None
Ports
OptionDescriptionDefault
-p 8080:8080Expose port 8080 on the host for the Airflow web UI (HTTP).None
-p 8794:8794Expose port 8794 on the host for the Airflow triggerer log server.None
-p 8793:8793Expose port 8793 on the host for the Airflow worker log server.None

[Report a bug] [Contribute]

Canonical is the provider of this container image. It is the user's responsibility to ensure that any use of this image complies with any relevant licenses for all software contained within, as well as with Canonical’s IP Policy.

Tag summary

Content type

Image

Digest

sha256:bfb468cd1

Size

702.9 MB

Last updated

about 1 month ago

docker pull ubuntu/airflow:3.1-24.04_edge

This week's pulls

Pulls:

38

Last week