📊 Python Data Workflows – ⏰ Scheduling Jobs 🐍

Posted on: July 10, 2026

Description:

Running a data pipeline manually every day quickly becomes repetitive.

Whether you’re generating reports, processing CSV files, or updating dashboards, automation ensures these tasks happen consistently without human intervention.

This is where job scheduling becomes useful.

Why Schedule Jobs?

Many data workflows need to run on a regular basis.

Examples include:

Daily sales reports
Hourly API synchronisation
Weekly dashboard updates
Nightly ETL pipelines

Instead of executing scripts manually, a scheduler can run them automatically.

Using the `schedule` Library

The schedule library provides a simple way to execute Python functions at fixed intervals.

schedule.every(10).seconds.do(process_sales_data)

You can also schedule jobs:

Every minute
Every hour
Every day
Every Monday
At a specific time

Running the Scheduler

The scheduler continuously checks whether a task is due.

while True:
    schedule.run_pending()
    time.sleep(1)

Whenever the scheduled time arrives, the corresponding function is executed automatically.

What About Cron?

On Linux and macOS, scheduled jobs are commonly managed using cron.

Example:

0 8 * * * python sales_report.py

This runs the script every day at 8:00 AM.

The schedule library is excellent for learning and small automation tasks, while cron is commonly used in production environments.

Key Takeaways

Scheduling removes repetitive manual work.
Automated workflows improve consistency.
The schedule library is easy to learn.
Cron is widely used for production scheduling.

Code Snippet:

import schedule
import time
import pandas as pd


# -------------------------------
# Job Function
# -------------------------------
def process_sales_data():
    print("\n🚀 Running Scheduled Job...")

    df = pd.read_csv("sales_data.csv")

    # Remove duplicate rows
    df = df.drop_duplicates()

    # Convert sales to numeric
    df["sales"] = pd.to_numeric(
        df["sales"],
        errors="coerce"
    )

    # Fill missing values
    df["sales"] = df["sales"].fillna(
        df["sales"].median()
    )

    total_sales = df["sales"].sum()
    average_sales = df["sales"].mean()

    print(f"💰 Total Sales: {total_sales:.2f}")
    print(f"📊 Average Sales: {average_sales:.2f}")

    df.to_csv(
        "processed_sales_data.csv",
        index=False
    )

    print("💾 Processed report generated successfully.")


# -------------------------------
# Schedule Job
# -------------------------------
schedule.every(10).seconds.do(process_sales_data)

print("⏰ Scheduler Started...")
print("Press Ctrl + C to stop.\n")


# -------------------------------
# Keep Scheduler Running
# -------------------------------
while True:
    schedule.run_pending()
    time.sleep(1)

← →	move
↑	rotate
↓	soft drop
Space	hard drop
P	pause / resume

📊 Python Data Workflows – ⏰ Scheduling Jobs 🐍

Description:

Why Schedule Jobs?

Using the `schedule` Library

Running the Scheduler

What About Cron?

Key Takeaways

Code Snippet:

Comments

Add Your Comment

📊 Python Data Workflows – ⏰ Scheduling Jobs 🐍

Description:

Why Schedule Jobs?

Using the schedule Library

Running the Scheduler

What About Cron?

Key Takeaways

Code Snippet:

Comments Show Comments

Add Your Comment

Related Posts

📊 Python Data Workflows – 🔄 ETL Pipeline 🐍

📊 Python Data Workflows – ⚙️ Mini Project 🐍

📊 Python Data Workflows – ✅ Data Validation 🐍

7-Day AI Crash Course

Using the `schedule` Library

Comments