Tutorial Install AirFlow
1. Pastikan kalian sudah memiliki Git dan telah ada di environment kalian. Jika belum
tambahkan terlebih dahulu, dan pastikan memiliki Docker desktop.
2. Clone repo berikut:
https://github.com/coder2j/airflow-docker?tab=readme-ov-file
3. Buka Docker desktop dan pastikan berjalan sebagaimana mestinya.
4. Masuk kedalam direktori airflow menggunakan Terminal.
5. Buat direktori ./dags ./logs ./plugins dengan kode berikut:
mkdir dags
mkdir logs
mkdir plugins
6. Rubah isi dari docker-compose.yaml menjadi berikut:
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
# Basic Airflow cluster configuration for CeleryExecutor with Redis and
PostgreSQL.
#
# WARNING: This configuration is for local development. Do not use it in a
production deployment.
#
# This configuration supports basic configuration using environment variables or
an .env file
# The following variables are supported:
#
# AIRFLOW_IMAGE_NAME - Docker image name used to run Airflow.
# Default: apache/airflow:master-python3.8
# AIRFLOW_UID - User ID in Airflow containers
# Default: 50000
# AIRFLOW_GID - Group ID in Airflow containers
# Default: 50000
# _AIRFLOW_WWW_USER_USERNAME - Username for the administrator account.
# Default: airflow
# _AIRFLOW_WWW_USER_PASSWORD - Password for the administrator account.
# Default: airflow
#
# Feel free to modify this file to suit your needs.
---
x-airflow-common:
&airflow-common
build: .
image: ${AIRFLOW_IMAGE_NAME:-extending_airflow:terbaru}
environment:
&airflow-common-env
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__CORE__SQL_ALCHEMY_CONN:
postgresql+psycopg2://airflow:airflow@postgres/airflow
AIRFLOW__CORE__FERNET_KEY: ''
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: 'true'
AIRFLOW__CORE__LOAD_EXAMPLES: 'false'
AIRFLOW__SCHEDULER__MIN_FILE_PROCESS_INTERVAL: 10
volumes:
- ./dags:/opt/airflow/dags
- ./logs:/opt/airflow/logs
- ./plugins:/opt/airflow/plugins
user: "${AIRFLOW_UID:-50000}:${AIRFLOW_GID:-50000}"
depends_on:
postgres:
condition: service_healthy
services:
postgres:
image: postgres:16
environment:
POSTGRES_USER: airflow
POSTGRES_PASSWORD: airflow
POSTGRES_DB: airflow
volumes:
- postgres-db-volume:/var/lib/postgresql/data
ports:
- 5432:5432
healthcheck:
test: ["CMD", "pg_isready", "-U", "airflow"]
interval: 5s
retries: 5
restart: always
airflow-webserver:
<<: *airflow-common
command: webserver
ports:
- 8080:8080
healthcheck:
test: ["CMD", "curl", "--fail", "http://localhost:8080/health"]
interval: 10s
timeout: 10s
retries: 5
restart: always
airflow-scheduler:
<<: *airflow-common
command: scheduler
restart: always
airflow-init:
<<: *airflow-common
command: version
environment:
<<: *airflow-common-env
_AIRFLOW_DB_UPGRADE: 'true'
_AIRFLOW_WWW_USER_CREATE: 'true'
_AIRFLOW_WWW_USER_USERNAME: ${_AIRFLOW_WWW_USER_USERNAME:-
airflow}
_AIRFLOW_WWW_USER_PASSWORD: ${_AIRFLOW_WWW_USER_PASSWORD:-
airflow}
volumes:
postgres-db-volume:
7. Set user permission dengan current user kalian dengan kode berikut:
$AIRFLOW_UID=[System.Security.Principal.WindowsIdentity]::GetCurrent().User.V
alue; "AIRFLOW_UID=$AIRFLOW_UID`nAIRFLOW_GID=0" | Out-File -FilePath .env
8. Jadikan file .env kalian bertipe utf-8 dengan kode berikut:
Get-Content .env | Out-File -FilePath .env -Encoding utf8
9. Jalankan filenya dengan menggunakan docker compose:
docker-compose up -d
10. cek apakah file docker berjalan dengan kode berikut:
docker ps
11. buka airflow di website dengan domain adalah sebagai berikut:
localhost:8080
12. login dengan username dan password adalah airflow
