Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds new exists logic that can optionally ignore the file extension #5154

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,23 @@ Description
filename extension (``file.1.ext``, ``file.2.ext``, etc.)


extractor.*.skip-extension
-----------------
Type
* ``bool``
Default
``false``
Description
If enabled, the file extension is ignored when checking to see if a file
already exists locally during the pre-download stage.
i.e. if downloading `./test/image_5.jpg` but `./test/image_5.png` or
`./test/image_5.avif` exists, it will behave as if `./test/image_5.jpg` exists
on disk

* ``false``: Respects the file extension when checking if the file exists
* ``true``: Ignores the file extension when checking if the file exists and
any file with the same path & name will return True

extractor.*.sleep
-----------------
Type
Expand Down
1 change: 1 addition & 0 deletions docs/gallery-dl.conf
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
"cookies-update": true,
"proxy": null,
"skip": true,
"skip-extension": false,

"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0",
"retries": 4,
Expand Down
15 changes: 13 additions & 2 deletions gallery_dl/path.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import re
import shutil
import functools
import glob
from . import util, formatter, exception

WINDOWS = util.WINDOWS
Expand Down Expand Up @@ -51,6 +52,7 @@ def __init__(self, extractor):
raise exception.FilenameFormatError(exc)

directory_fmt = config("directory")
self.skip_ext = config("skip-extension") or False
try:
if directory_fmt is None:
directory_fmt = extractor.directory_fmt
Expand Down Expand Up @@ -158,8 +160,17 @@ def open(self, mode="wb"):

def exists(self):
"""Return True if the file exists on disk"""
if self.extension and os.path.exists(self.realpath):
return self.check_file()
# Search only by file name not including file extension
# This means checking if 'image_1.jpg' is on disk will
# be True if 'image_1.png' is found
if self.skip_ext:
split_real_path = self.realpath.split(".")
any_ext_path = ".".join(split_real_path[:-1]) + ".*"
if self.extension and glob.glob(any_ext_path):
return self.check_file()
else:
if self.extension and os.path.exists(self.realpath):
return self.check_file()
return False

@staticmethod
Expand Down
Loading