Monitoring Apache Airflow DAGs with a dead man's switch

Airflow has retries, SLAs and on-failure callbacks, but they all run inside Airflow. If the scheduler is stuck, the metadata database is full, or someone pauses a DAG, those callbacks never fire. An external heartbeat catches exactly that case: it alerts when the DAG didn't finish.

Ping at the end of the DAG

Add a final task that runs only when everything upstream succeeded:

from airflow.operators.python import PythonOperator
import urllib.request

def ping(**_):
    urllib.request.urlopen("https://cronpup.com/p/YOUR-UUID", timeout=10)

done = PythonOperator(task_id="heartbeat", python_callable=ping, trigger_rule="all_success")
extract >> transform >> load >> done

Report failures immediately

Use an on_failure_callback on the DAG to call https://cronpup.com/p/YOUR-UUID/fail, so a failed run alerts at once instead of waiting for the deadline.

Match the schedule

Give the monitor the same cron expression and timezone as the DAG's schedule, and a grace period a little longer than the DAG's usual runtime (check its duration history in the Airflow UI).

One monitor per DAG that matters

Not every DAG needs one. Start with the ones someone would notice missing a day later: billing exports, reports, backups. Create the monitors; 10 are free.