Zephyrnet Logo

Generative Data Intelligence

Big Data

Monitor Your File System With Python’s Watchdog – KDnuggets

Republished By Plato
Republished By Plato

Date:

Views: 26

Monitor Your File System With Python’s Watchdog
Image by Author | DALLE-3 & Canva

 

Python’s watchdog library makes it easy to monitor your file system and respond to these changes automatically. Watchdog is a cross-platform API that allows you to run commands in response to any changes in the file system being monitored. We can set triggers on multiple events such as file creation, modification, deletion, and movement, and then respond to these changes with our custom scripts.

Setup for Watchdog

You’ll need two modules to begin:

  • Watchdog: Run this command below in the terminal to install the watchdog. 
    pip install watchdog

     

  • Logging: It is a built-in Python module, so there is no need to externally install it.

Basic Usage

Let’s create a simple script ‘main.py’ that monitors a directory and prints a message whenever a file is created, modified, or deleted.

Step 1: Import Required Modules

First, import the necessary modules from the watchdog library:

import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

Step 2: Define Event Handler Class

We define a class MyHandler that inherits from FileSystemEventHandler. This class overrides methods like on_modified, on_created, and on_deleted to specify what to do when these events occur. The event handler object will be notified when any changes happen in the file system.

class MyHandler(FileSystemEventHandler):
    def on_modified(self, event):
        print(f'File {event.src_path} has been modified')

    def on_created(self, event):
        print(f'File {event.src_path} has been created')

    def on_deleted(self, event):
        print(f'File {event.src_path} has been deleted')

Some useful methods of FileSystemEventHandler are explained below.

  • on_any_event: Executed for any event.
  • on_created: Executed upon creation of a new file or directory.
  • on_modified: Executed upon modification of a file or when a directory is renamed.
  • on_deleted: Triggered upon the deletion of a file or directory.
  • on_moved: Triggered when a file or directory is relocated.

Step 3: Initialize and Run the Observer

The Observer class is responsible for tracking the file system for any changes and subsequently notifying the event handler. It continuously tracks file system activities to detect any updates.

if __name__ == "__main__":
    event_handler = MyHandler()
    observer = Observer()
    observer.schedule(event_handler, path='.', recursive=True)
    observer.start()

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

We start the observer and use a loop to keep it running. When you want to stop it, you can interrupt with a keyboard signal (Ctrl+C).

Step 4: Run the Script

Finally, run the script with the following command.

python main.py

Output: 

File .File1.txt has been modified
File .New Text Document (2).txt has been created
File .New Text Document (2).txt has been deleted
File .New Text Document.txt has been deleted

The above code will log all the changes in the directory to the terminal if any file/folder is created, modified, or deleted.

Advanced Usage

In the following example, we will explore how to set up a system that detects any change in Python files and run tests for it automatically. We need to install pytest with the following command.

pip install pytest

Step 1: Create a Simple Python Project With Tests

First, set up the basic structure of your project:

my_project/
│
├── src/
│   ├── __init__.py
│   └── example.py
│
├── tests/
│   ├── __init__.py
│   └── test_example.py
│
└── watchdog_test_runner.py

Step 2: Write Code in Example Python File

Create a simple Python module in src/example.py:

def add(a, b):
    return a + b

def subtract(a, b):
    return a - b

Step 3: Write the Test Cases

Next, write the test cases for functions in tests/test_example.py:

import pytest
from src.example import add, subtract

def test_add():
    assert add(1, 2) == 3
    assert add(-1, 1) == 0
    assert add(-1, -1) == -2

def test_subtract():
    assert subtract(2, 1) == 1
    assert subtract(1, 1) == 0
    assert subtract(1, -1) == 2

Step 4: Write the Watchdog Script

Now, create the watchdog_test_runner.py script to monitor changes in Python files and automatically run tests:

import time
import subprocess
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class TestRunnerHandler(FileSystemEventHandler):
    def on_modified(self, event):
        if event.src_path.endswith('.py'):
            self.run_tests()

    def run_tests(self):
        try:
            result = subprocess.run(['pytest'], check=False, capture_output=True, text=True)
            print(result.stdout)
            print(result.stderr)
            if result.returncode == 0:
                print("Tests passed successfully.")
            else:
                print("Some tests failed.")
        except subprocess.CalledProcessError as e:
            print(f"Error running tests: {e}")

if __name__ == "__main__":
    path = "."  # Directory to watch
    event_handler = TestRunnerHandler()
    observer = Observer()
    observer.schedule(event_handler, path, recursive=True)
    
    observer.start()
    print(f"Watching for changes in {path}...")

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    
    observer.join()

Step 5: Run the Watchdog Script

In the end, open a terminal, navigate to your project directory (my_project), and run the watchdog script:

python watchdog_test_runner.py

Output:

Watching for changes in ....
========================= test session starts =============================
platform win32 -- Python 3.9.13, pytest-8.2.1, pluggy-1.5.0
rootdir: F:Web Devwatchdog
plugins: anyio-3.7.1
collected 2 items

teststest_example.py ..                                                 [100%]

========================== 2 passed in 0.04s ==============================

Tests passed successfully.

This output shows that all the test cases are passed after changes were made to example.py file.

Summing Up

Python’s watchdog library is a powerful tool for monitoring your file system. Whether you’re automating tasks, syncing files, or building more responsive applications, watchdog makes it easy to react to file system changes in real time. With just a few lines of code, you can start monitoring directories and handling events to streamline your workflow.

 
 

Kanwal Mehreen Kanwal is a machine learning engineer and a technical writer with a profound passion for data science and the intersection of AI with medicine. She co-authored the ebook “Maximizing Productivity with ChatGPT”. As a Google Generation Scholar 2022 for APAC, she champions diversity and academic excellence. She’s also recognized as a Teradata Diversity in Tech Scholar, Mitacs Globalink Research Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having founded FEMCodes to empower women in STEM fields.

More On This Topic

spot_img

Latest Intelligence

spot_img

Copyright @ 2024 Plato Technologies Inc.