Python for Automation & Scripting

Published 1 month ago

Automation has become an essential skill for developers, system administrators, and even non-technical professionals. Python, being a powerful and easy-to-learn language, is widely used for automation and scripting. This blog explores three major areas of Python automation:

Automating File Management
Web Scraping with BeautifulSoup & Selenium
Task Automation with Python

1. Automating File Management

Managing files and directories manually can be tedious. Python provides powerful libraries like os, shutil, and pathlib to automate file management tasks.

1.1 Working with Files and Directories

The os and pathlib modules help interact with the filesystem.

import os
from pathlib import Path

# Create a new directory
os.makedirs("backup", exist_ok=True)

# List files in a directory
files = os.listdir(".")
print("Files:", files)

# Rename a file
os.rename("old_file.txt", "new_file.txt")

# Delete a file
os.remove("unwanted_file.txt")

1.2 Copying and Moving Files

The shutil module helps in copying and moving files.

import shutil

# Copy a file
shutil.copy("source.txt", "destination.txt")

# Move a file
shutil.move("file.txt", "backup/file.txt")

1.3 Automating File Cleanup

You can automate the deletion of old log files or temporary files using Python.

import os
import time

# Delete files older than 7 days
folder = "logs"
cutoff_time = time.time() - (7 * 24 * 60 * 60)  # 7 days in seconds

for filename in os.listdir(folder):
    file_path = os.path.join(folder, filename)
    if os.path.isfile(file_path) and os.path.getmtime(file_path) < cutoff_time:
        os.remove(file_path)
        print(f"Deleted {file_path}")

2. Web Scraping with BeautifulSoup & Selenium

Web scraping helps automate data collection from websites. Python provides two powerful libraries for this:

BeautifulSoup (for static web pages)
Selenium (for dynamic web pages with JavaScript)

2.1 Web Scraping with BeautifulSoup

BeautifulSoup is ideal for parsing HTML and extracting data from static web pages.

import requests
from bs4 import BeautifulSoup

# Fetch the webpage
url = "https://example.com"
response = requests.get(url)

# Parse HTML content
soup = BeautifulSoup(response.text, "html.parser")

# Extract and print all links
for link in soup.find_all("a"):
    print(link.get("href"))

2.2 Web Scraping with Selenium

Selenium is used when JavaScript renders content dynamically.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager

# Set up the browser
service = Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=service)

# Open a webpage
driver.get("https://example.com")

# Extract dynamic content
elements = driver.find_elements(By.TAG_NAME, "p")
for element in elements:
    print(element.text)

# Close the browser
driver.quit()

3. Task Automation with Python

Python can automate repetitive tasks such as sending emails, scheduling scripts, and processing data.

3.1 Automating Emails with `smtplib`

Python’s smtplib module allows sending emails automatically.

import smtplib
from email.message import EmailMessage

# Email details
sender_email = "your_email@example.com"
receiver_email = "recipient@example.com"
subject = "Automated Email"
body = "Hello, this is an automated email."

# Create EmailMessage object
msg = EmailMessage()
msg["From"] = sender_email
msg["To"] = receiver_email
msg["Subject"] = subject
msg.set_content(body)

# Send email
with smtplib.SMTP("smtp.example.com", 587) as server:
    server.starttls()
    server.login("your_email@example.com", "your_password")
    server.send_message(msg)

3.2 Scheduling Tasks with `schedule`

The schedule library helps execute scripts at specified times.

import schedule
import time

def job():
    print("Running scheduled task...")

# Schedule the task every minute
schedule.every(1).minutes.do(job)

while True:
    schedule.run_pending()
    time.sleep(1)

3.3 Automating Data Processing with Pandas

pandas helps process large datasets automatically.

import pandas as pd

# Read an Excel file
df = pd.read_excel("data.xlsx")

# Process data
df["Total"] = df["Price"] * df["Quantity"]

# Save processed data
df.to_csv("processed_data.csv", index=False)

Conclusion

Python is a powerful tool for automation and scripting. Whether you’re managing files, extracting web data, or automating tasks, Python provides efficient libraries to streamline your workflow. Start small, experiment, and integrate automation into your daily tasks to improve efficiency and productivity!

Obafemi Emmanuel