Project reloads every time sqlite is touched

[Jackevansevo]

This issue is pretty similar to: #2

I'm re-opening/duplicating the original issue with some additional context as I think some filtering is not properly taking effect here. I'm seeing the server restart/reload when anything in sqlite is touched (something which doesn't happen with the default StatReloader)

Is this behaviour known + intended/expected? (If not I'm happy to investigate a bit further)

---

I guess I could easily move sqlite out of my project and into a separate dir (where it won't trigger reloads)

Below is a comparison between the default/watchfiles reloaders

Screencast.from.24-09-22.10.21.08.mov

e.g. when I login to the admin panel:

❯ ./manage.py runserver
Watching for file changes with WatchfilesReloader
Performing system checks...

System check identified no issues (0 silenced).
September 24, 2022 - 13:04:54
Django version 4.1.1, using settings 'e.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
watcher: INotifyWatcher { channel: Sender { .. }, waker: Waker { inner: Waker { fd: File { fd: 7, path: "anon_inode:[eventfd]", read: true, write: true } } } }
raw-event: Event { kind: Create(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Remove(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Create(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Remove(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Create(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Modify(Data(Any)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
raw-event: Event { kind: Remove(File), paths: ["/home/jackevans/code/e/db.sqlite3-journal"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
[24/Sep/2022 13:05:05] "POST /admin/login/?next=/admin/ HTTP/1.1" 302 0
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
[24/Sep/2022 13:05:05] "GET /admin/ HTTP/1.1" 200 3888
raw-event: Event { kind: Access(Close(Write)), paths: ["/home/jackevans/code/e/db.sqlite3"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
/home/jackevans/code/e/db.sqlite3-journal changed, reloading.
Watching for file changes with WatchfilesReloader
Performing system checks...

[q0w]

Check my pr #10
You can pass custom watchfiles filter or use bult-in watchfiles PythonFilter.

[ddahan]

I have a similar issue which is quite annoying:
when uploading files via a form, the server restarts immediately when files are uploaded to my server, and my request is interrupted, the response is not return.

@q0w can I use your PR code to filter watched folders? To say something like "don't watch mediafiles".
Thanks.

[q0w]

Yes, you can. Use bultin PythonFilter or write your own. Check watchfiles docs, its simple to ignore specific folders

WATCHFILES = { "watch_filter": "watchfiles.PythonFilter"}

[ddahan]

Thanks a lot, it works as expected:

# https://watchfiles.helpmanual.io/api/filters/#custom-filters
class MediaFilesFilter(DefaultFilter):
    """Prevent a local server reload when uploading files"""

    def __call__(self, change: Change, path: str) -> bool:
        return super().__call__(change, path) and "mediafiles" not in path

In settings:

WATCHFILES = {
    "watch_filter": "django_watchfiles.custom_filters.MediaFilesFilter",
    "raise_interrupt": False,
    "debug": False,
}

Just waiting for the PR to be accepted and merged, sothat I can use the package itself rather than writing my own internal app.

[minusf]

this project is nice cause it works (as opposed to watchman) but it's a bit too raw. if django-watchfiles is the interface to watchfiles i think it needs to expose at least some basic level of control in a bit more user friendly way. the bare minimum being disabling debug messages and having a whitelist/blacklist. thank you for looking into this.

[adamchainz]

This is an alpha quality project still. When I have time I’ll work on it again.

[minusf]

thank you for doing all the heavy lifting and research. for now i removed this dependency from our projects and simply add a watcher.py module in project directory with a list of folders to watch:

from pathlib import Path

import watchfiles

from django.conf import settings
from django.utils import autoreload


class WatchfilesReloader(autoreload.BaseReloader):
    def tick(self):
        watcher = watchfiles.watch(*settings.WATCHFILES, debug=False)
        for file_changes in watcher:
            for _change, path in file_changes:
                self.notify_file_changed(Path(path))
            yield


def replaced_get_reloader() -> autoreload.BaseReloader:
    return WatchfilesReloader()


autoreload.get_reloader = replaced_get_reloader

[adamchainz]

I think dc1af91 fixed this by filtering changes coming from watchfiles.