← All Categories
📊
Data Engineering & ETL Cron Expressions
Cron expressions for data engineering workflows — Airflow DAGs, dbt transformations, Spark batch jobs, Snowflake reloads, Kafka consumer checks, and ETL pipelines.
Featured Patterns
0 2 * * *
Apache Airflow DAG Daily at 2 AM
Apache Airflow DAG schedule using cron preset '@daily' equivalent — runs at 2:00 AM every day. Set in your DAG definition: schedule='0 2 * * *'. Avoids peak hours for heavy data pipeline execution.
Airflow ETL nightly pipelinedbt models run via Airflow DAG
0 * * * *
Apache Airflow DAG Hourly
Apache Airflow DAG with hourly schedule — equivalent to '@hourly' preset. schedule='0 * * * *' in your DAG. Suitable for near-real-time data pipelines, hourly metric aggregation, and incremental data loads.
Hourly data ingestion pipelineIncremental Airflow ETL task
0 6 * * *
dbt Daily Transformation Run at 6 AM
dbt (data build tool) scheduled to run daily at 6:00 AM, ahead of business hours. Triggered via dbt Cloud job schedule, Airflow, or Prefect. Transforms raw data into analytics-ready models before analysts arrive.
dbt Cloud job daily scheduledbt models fresh before 9am
0 0 * * 0
dbt Weekly Full Refresh — Sunday Midnight
dbt full-refresh run every Sunday at midnight to rebuild all incremental models from scratch. Prevents data drift accumulation. Use dbt Cloud's 'Full Refresh' flag with this cron schedule.
dbt full-refresh incremental modelsWeekly data mart rebuild
0 3 * * *
dbt Daily Run — 3 AM
Executes the full dbt model DAG every day at 3 AM. Runs all dbt models in dependency order, tests data quality assertions, and generates documentation for changed models.
dbt daily run crondbt model refresh 3am scheduler
0 1 * * *
Apache Spark Nightly Batch at 1 AM
Apache Spark batch job triggered nightly at 1:00 AM via Airflow, Oozie, or a cron-triggered submit script. Processes daily data volumes during off-peak hours with minimal cluster contention.
Spark job nightly data processingHadoop batch ETL 1am trigger
0 0 * * *
ETL Extract Phase — Daily at Midnight
ETL pipeline extract phase running daily at midnight to pull fresh data from source systems before transform and load phases. The classic nightly ETL start time to maximize the processing window.
Nightly ETL extract from CRMDatabase-to-data-lake extract
0 4 * * *
Snowflake Data Reload — Daily at 4 AM
Snowflake data reload task running daily at 4:00 AM using Snowflake Tasks or an external scheduler. Loads staged data from S3/Azure Blob/GCS into Snowflake tables before business-hour query traffic.
Snowflake COPY INTO from S3Snowflake Task daily execution
*/5 * * * *
Kafka Consumer Lag Check Every 5 Minutes
Automated Kafka consumer group lag monitoring every 5 minutes. Checks consumer offset lag across topics, triggers alerts if lag exceeds threshold, and logs metrics to monitoring systems.
Kafka consumer lag alertingConsumer group offset monitoring
0 2 * * *
Daily at 2:00 AM
Runs once per day at 2:00 AM UTC. Popular for backup jobs run during low-traffic hours.
Full database backupNightly data export
0 * * * *
Every Hour (at :00)
Runs once at the top of every hour (:00), 24 times per day. The standard hourly cron schedule.
Hourly reportsCache invalidation
Related Patterns
*/10 * * * *
Every 10 Minutes
Runs every 10 minutes, 144 times per day. A good balance between frequency and resource usage.
*/15 * * * *
Every 15 Minutes
Runs every 15 minutes (at :00, :15, :30, :45), 96 times per day. Common for lightweight periodic tasks.
*/20 * * * *
Every 20 Minutes
Runs every 20 minutes (at :00, :20, :40), 72 times per day.
*/30 * * * *
Every 30 Minutes
Runs twice per hour (at :00 and :30), 48 times per day. Very common for periodic background jobs.
*/45 * * * *
Every 45 Minutes
Runs every 45 minutes. Note: because 45 doesn't divide 60 evenly, runs occur at :00 and :45 each hour.
15 * * * *
Hourly at :15
Runs at 15 minutes past every hour (:15), 24 times per day. Useful when you want to offset from other hourly jobs.
30 * * * *
Hourly at :30
Runs at 30 minutes past every hour (:30), 24 times per day.
45 * * * *
Hourly at :45
Runs at 45 minutes past every hour (:45), 24 times per day.