v1.28.2

Extractors

Additions

• [cyberdrop] add extractor for media URLs (#2496)
• [itaku] add search extractor (#6613)
• [lofter] add initial support (#650, #2294, #4095, #4728, #5656, #6607)
• [yiffverse] add support (#6611)

Fixes

• [facebook] decode Unicode surrogate pairs in metadata values (#6599)
• [zerochan] parse API responses manually when receiving invalid JSON (#6632)
• [zerochan] fix source metadata extraction when not logged in

Improvements

• [bilibili] extract files from module_top entries (#6687)
• [bilibili] support /upload/opus URLs (#6687)
• [bluesky] default to posts timeline when reposts or quoted is enabled (#6583)
• [common] simplify HTTP error messages
• [common] detect DDoS-Guard challenge pages
• [deviantart] improve tiptap markup to HTML conversion (#6686)
  • fix KeyError: 'attrs' for links without href
  • support heading content blocks
  • support strike text markers
• [instagram] extract date metadata for stories (#6677)
• [kemonoparty:favorite] support new URL format (#6676)
• [saint] support saint2.cr URLs (#6692)
• [tapas] improve extractor hierarchy (#6680)

Options

• [cohost] add avatar and background options (#6656)

Miscellaneous

• support * wildcards for parent>child categories, for example reddit>* (#6673)
• use latest Firefox UA as default user-agent
• use random unused port for "user-agent": "browser" requests

v1.28.1

Extractors

Additions

• [bluesky] add info extractor

Fixes

• [bluesky] fix exception when encountering non-quote embeds (#6577)
• [bluesky] unescape search queries (#6579)
• [common] restore using environment proxy settings by default (#6553, #6609)
• [common] improve handling of user-agent settings (#6594)
• [e621] fix TypeError when metadata is enabled (#6587)
• [gofile] fix website token extraction (#6596)
• [inkbunny] fix re-login loop (#6618)
• [instagram] handle empty carousel_media entries (#6595)
• [kemonoparty] fix o query parameter handling (#6597)
• [nhentai] fix download URLs (#6620)
• [readcomiconline] fix chapter extraction (#6070, #6335)
• [realbooru] fix extraction (#6543)
• [rule34] fix favorite extraction (#6573)
• [zerochan] download .webp and .gif files (#6576)

Improvements

• [hentaicosplays] update domains (#6578)
• [pixiv:ranking] implement filtering results by content (#6574)
• [pixiv] include user ID in failed AJAX request warnings (#6581)

Options

• [patreon] add format-images option (#6569)
• [zerochan] add extensions option (#6576)

v1.28.0

Changes

• [common] disable using environment network settings by default (HTTP_PROXY, .netrc, …)
  • disable trust_env session attribute
  • disable Authorization header injection from .netrc auth (#5780, #6134, #6455)
  • add proxy-env option
• [ytdl] change forward-cookies default value to true (#6401, #6348)

Extractors

Additions

• [bilibili] add support for opus articles (#2824, #6443)
• [bluesky] add hashtag extractor (#4438)
• [danbooru] add artist and artist-search extractors (#5348)
• [everia] add support (#1067, #2472, #4091, #6227)
• [facebook] add support (#470, #2612, #5626, #6548)
• [hentaifoundry] add tag extractor (#6465)
• [hitomi] add index and search extractors (#2502, #6392, #3720)
• [motherless] add support (#2074, #4413, #6221)
• [noop] add noop extractor
• [rule34vault] add support (#5708, #6240)
• [rule34xyz] add support (#1078, #4960)
• [saint] add support (#4405, #6324)
• [tumblr] add search extractor (#6394)

Fixes

• [8chan] avoid performing network requests within _init() (#6387)
• [bluesky] fix downloads from non-bsky PDSs (#6406)
• [bunkr] fix album names containing <>& characters
• [flickr] use download URLs (#6360, #6464)
• [hiperdex] update domain to hipertoon.com (#6420)
• [imagechest] fix extractors (#6475, #6491)
• [instagram] fix using numeric cursor values (#6414)
• [kemonoparty] update to new site layout (#6415, #6503, #6528, #6530, #6536, #6542, #6554)
• [koharu] update domain to niyaniya.moe (#6430, #6432)
• [mangadex] apply lang option only to chapter results (#6372)
• [newgrounds] fix metadata extraction (#6463, #6533)
• [nhentai] support .webp files (#6442, #6479)
• [patreon] use legacy mobile UA when no session_id is set
• [pinterest] update API headers (#6513)
• [pinterest] detect video/audio by block content (#6421)
• [scrolller] prevent exception for posts without mediaSources (#5051)
• [tumblrgallery] fix file downloads (#6391)
• [twitter] make source metadata extraction non-fatal (#6472)
• [weibo] fix livephoto filename & extension (#6471)

Improvements

• [bluesky] support main.bsky.dev URLs (#4438)
• [bluesky] match common embed fixes (#6410, #6411)
• [boosty] update default video format list (#2387)
• [bunkr] support bunkr.cr URLs
• [common] allow passing cookies to OAuth extractors
• [common] allow overriding more default User-Agent headers (#6496)
• [philomena] switch default ponybooru filter (#6437)
• [pinterest] support story_pin_music blocks (#6421)
• [pixiv] get ugoira frame extension from meta_single_page values (#6056)
• [reddit] support user profile share links (#6389)
• [steamgriddb] disable adjust-extensions for fake-png files (#5274)
• [twitter] remove cookies migration workaround

Metadata

• [bluesky] provide author[instance] metadata (#4438)
• [instagram] fix extension of apparent .webp files (#6541)
• [pillowfort] provide count metadata (#6478)
• [pixiv:ranking] add rank metadata field (#6531)
• [poipiku] return count as proper number (#6445)
• [webtoons] extract episode_no for comic results (#6439)

Options

• [civitai] add metadata option - support fetching generation data (#6383)
• [exhentai] implement tags option (#2117)
• [koharu] implement tags option
• [rule34xyz] add format option (#1078)

Downloaders

• [ytdl] fix AttributeError caused by decodeOption() removal (#6552)

Post Processors

• [classify] rewrite - fix skipping existing files (#5213)
• enable inheriting options from global postprocessor objects
• allow postprocessors values to be a single post processor object

Cookies

• support Chromium table version 24 (#6162)
• fix GCM pad length calculation (#6162)
• try decryption with empty password as fallback

Documentation

• update recommended pip command for installing dev version (#6493)
• update gallery-dl.conf (#6501)

Options

• add -4/--force-ipv4 and -6/--force-ipv6 command-line options
• fix passing negative numbers as arguments (#5262)

Miscellaneous

• [output] use default ANSI colors only when stream is a TTY
• [util] implement defaultdict filters-environment
• [util] enable using multiple statements for all filter options (#5262)

v1.27.7

Extractors

Additions

• [civitai] add extractors for global models and images (#6310)
• [mangadex] add author extractor (#6372)
• [scrolller] add support (#295, #3418, #5051)

Fixes

• [8chan] automatically detect TOS cookie name (#6318)
• [bunkr] update to new site layout (#6344, #6352, #6368)
• [bunkr] send proper Referer headers for file downloads (#6319)
• [civitai] add uuid metadata field & use it as default archive format (#6326)
• [civitai] fix "My Reactions" results (#6263)
• [civitai] fix model file download URLs for tRPC API
• [lensdump] fix extraction (#6313)
• [pixiv] make retrieving ugoira metadata non-fatal (#6297)
• [pixiv] fix exception when processing deleted sanity_level works (#6339)
• [urlgalleries] fix extraction
• [wikimedia] fix non-English Fandom/wiki.gg articles (#6370)

Improvements

• [8chan] support /last/ thread URLs (#6318)
• [bunkr] support bunkr.ph and bunkr.ps URLs
• [newgrounds] support page numbers in URLs (#6320)
• [patreon] support /c/ prefix in creator URLs (#6348)
• [pinterest] support story pins (#6188, #6078, #4229)
• [pixiv] implement sanity_level workaround for user artworks results (#4327, #5435, #6339)

Options

• [bluesky] add quoted option (#6323)
• [pixiv] add captions option (#4327)
• [reddit] add embeds option (#6357)
• [vk] add offset option (#6328)

Downloaders

• [ytdl] implement explicit HLS/DASH handling

Post Processors

• add error event

Miscellaneous

• [cookies] convert Chromium expires_utc values to Unix timestamps
• [util] add std object to global eval namespace (#6330)
• add --print and --print-to-file command-line options (#6343)
• use child extractor fallbacks only when a non-user error occurs (#6329)

v1.27.6

Extractors

Additions

• [ao3] add subscriptions extractor (#6247)
• [boosty] add support (#2387)
• [civitai] add post extractors (#6279)
• [pixiv] support unlisted artworks (#5162)

Fixes

• [cohost] sanitize default filenames (#6262)
  • limit headline length
  • remove plainTextBody
• [deviantart] fix & improve journal/literature extraction (#6254, #6207, #6196)
  • extract journal HTML from webpage if possible
  • support converting tiptap markup to HTML
• [deviantart] fix stash folder extraction
• [flickr] update default API credentials (#6300)
• [flickr] fix ZeroDivisionError (#6252)
• [imagefap] fix {num} in default filenames for single images
• [myhentaigallery] fix tags extraction
• [patreon] extract attachments_media files (#6241, #6268)
• [pixiv] implement workaround for limit_sanity_level works (#4327, #4747, #5054, #5435, #5651, #5655)
• [pornhub] fix KeyError when album images are missing (#6299)
• [rule34us] fix extraction (#6289)
• [8chan] set TOS cookie for current and previous day

Improvements

• [bunkr] support bunkr.pk URLs (#6272)
• [civitai] use tRPC API by default (#6279)
• [civitai] improve default archive format (#6302)
• [komikcast] update domain to komikcast.cz
• [newgrounds] detect more comment embeds (#6253)
• [newgrounds] add more fallback URL formats for art-images files
• [oauth] prevent empty browser names
• [patreon] use mobile UA (#6241, #6239, #6140)
• [patreon] handle suspended accounts
• [pixiv] detect works requiring My pixiv access

Metadata

• [civitai] ensure image files have an id (#6251)
• [gelbooru_v02] unescape HTML entities in categorized tags
• [generic] ensure path metadata is always defined
• [pixiv] retrieve caption from AJAX API when empty (#4327, #5191)

Options

• [fanbox] add comments option, extend metadata option (#6287)
• [pixiv] add comments option (#6287)

Removals

• [blogger] remove micmicidol.club
• [chevereto] remove deltaporno.com
• [lolisafe] remove xbunkr.com
• [pururin] remove module
• [shimmie2] remove loudbooru.com

Post Processors

• [ugoira] fix BadZipFile exceptions (#6285)
• [ugoira] catch all exceptions when extracting ZIP archives (#6285)
• [ugoira] forward frame data as _ugoira_frame_data (#6154, #6285)

Miscellaneous

• [build] remove setuptools and requests version restrictions
• [docker] build from python:3.12-alpine
• [text] improve parse_query() performance

v1.27.5

Extractors

Additions

• [ao3] add support (#6013)
• [civitai] add support (#3706, #3787, #4129, #5995, #6220)
• [cohost] add support (#4483, #6191)

Fixes

• [8chan] update TOS cookie name
• [deviantart] work around OAuth API returning empty journal texts (#6196, #6207, #5916)
• [weasyl:favorite] fix pagination (#6113)

Improvements

• [bluesky] support video downloads (#6183)
• [deviantart] add previews option (#3782, #6124)
• [deviantart] warn about empty journal texts (#5916)
• [inkbunny:favorite] update default directory (#6115)
• [jpgfish] update domain to jpg5.su (#6231)
• [skeb] prevent 429 errors and need for request_key cookie
• [weasyl:favorite] support readable URL format (#6113)
• [wikimedia] automatically detect API endpoint when none is defined
• [zzup] support up.zzup.com galleries (#6181)

Post Processors

• [ugoira] implement storing "original" frames in ZIP archives (#6147)
• [ugoira] fix KeyError: '_ugoira_frame_index' (#6154)

Formatter

• add L conversion - returns the length of a value
• allow accessing util.NONE via global _nul

Miscellaneous

• [cookies] add cookies-select option
• [cookies:firefox] support using domain & container filters together
• [docker] prevent errors in Dockerfile build
• [tests] make #category result entries optional
• allow filtering --list-extractors results
• implement alternatives for deprecated utc datetime functions

v1.27.4

Extractors

Additions

• [sexcom] add likes extractor (#6149)
• [wikimedia] add wiki extractor (#6050)

Fixes

• [bunkr] fix file downloads (#6037)
• [cyberdrop] fix extraction
• [deviantart] fix "pagination": "manual" for cursor-based endpoints (#6079)
• [deviantart] fix "original": "images" (#6124)
• [exhentai] fix limits option (#6090)
• [flickr] make exif and context metadata extraction non-fatal (#6002, #6077)
• [flickr] make album metadata extraction non-fatal (#3441)
• [furaffinity] fix favorite pagination (#6151)
• [gofile] fix KeyError: 'childrenIds' (#5993)
• [newgrounds] fix warning for age-restricted posts (#6005)
• [toyhouse] fix extraction of image URLs
• [tumblr] fix 401 Unauthorized for likes when using api-key (#5994)
• [twitter] fix pinned Tweet extraction (#6102)
• [ytdl] fix processing playlists of playlists (#6127)

Improvements

• [bcbnsfw] use * as query when retrieving all posts (#6135)
• [bunkr] support bunkr: URL prefix (#6017)
• [e621] cache pool metadata API calls (#6001)
• [generic] better directory names (#6104)
• [koharu] improve format selection (#6088)
• [pixiv] implement downloading "original" ugoira frames (#6056)
• [pixiv] use mobile API for series (#5983)

Metadata

• [batoto] improve chapter info regex (#5988, #5997)
• [batoto] extract chapter_url metadata (#5562)
• [batoto] improve title extraction (#5988)
• [hitomi] extract extension_original metadata (#6049)
• [instagram] add post_date metadata field (#6081, #6091)
• [sankaku] restore old tags format (#6043)
• [twitter] extract type metadata (#6111)

Options

• [bunkr] add tlds option to match URLs with all possible TLDs (#5875, #6017)
• [instagram] add max-posts option (#6054)
• [instagram] add info as a possible include value
• [instagram] allow disabling cursor output
• [twitter] add info as a possible include value (#6114)
• [twitter] allow disabling cursor output (#5990)

Post Processors

• [hash] add hash post processor to compute file hash metadata (#6099)
• [metadata] add include and exclude options (#6058)
• [metadata] fix using .. in directories on Windows (#5942, #6094)
• [rename] add rename post processor to rename previously downloaded files (#5846, #6044)
• [ugoira] support converting "original" frames (#6056)
• [ugoira] add skip option (#6056)

Miscellaneous

• [cookies:firefox] extract only cookies without container by default (#5957)
• [formatter] implement A format specifier (#6036)
• [tests] fix bug when running tests in a certain order
• [util] extend CustomNone with arithmetic operators (#6007, #6009)
• add --rename and --rename-to command-line options (#5846, #6044)
• add input-files config option (#6059)

v1.27.3

Extractors

Additions

• [bunkr] support bunkr.ci and bunkrrr.org (#5970)
• [furaffinity] add submissions extractor (#5954)
• [hentaicosplays] support hentai-cosplay-xxx.com (#5959)

Fixes

• [behance] fix KeyError: 'fields' (#5965)
• [behance] fix video extraction (#5965)
• [cien] extract all files when authenticated (#5934)
• [deviantart] fix KeyError - 'category' (#5960, #5961, #5969, #5971, #5976, #5978)
• [fanbox] update pagination logic (#5949, #5951, #5956)
• [hotleak] fix AttributeError (#5950)
• [instagram] restore GraphQL API functionality (#5920)
• [twitter] update x-csrf-token header during login (#5945)

Improvements

• [bunkr] fail downloads for maintenance files (#5952)
• [zerochan] improve tag redirect handling, add redirects option (#5891)

Post Processors

• [metadata] add base-directory option (#5262, #5728)

v1.27.2

Extractors

Additions

• [agnph] add tag and post extractors (#5284, #5890)
• [aryion] add favorite extractor (#4511, #5870)
• [cien] add support (#2885, #4103, #5240)
• [instagram] add info extractor (#5262)
• [koharu] add gallery, search, and favorite extractors (#5893, #4707)
• [twitter] add info extractor (#3623)

Fixes

• [8chan] update TOS cookie name (#5868)
• [behance] fix image extraction (#5873, #5926)
• [booru] prevent crash when file URL is empty (#5859)
• [deviantart] try to work around journal/status API changes (#5916)
• [hentainexus] fix error with spread pages (#5827)
• [hotleak] fix faulty image URLs (#5915)
• [inkbunny:following] fix potentially infinite loop
• [nijie] fix image URLs of single image posts (#5842)
• [readcomiconline] fix extraction (#5866)
• [toyhouse] fix Content Warning bypass (#5820)
• [tumblr] revert to offset pagination, implement pagination option (#5880)
• [twitter] fix username-alt option name (#5715)
• [warosu] fix extraction
• [zerochan] handle KeyError - 'items' (#5826)
• [zerochan] fix error on tag redirections (#5891)
• [zerochan] fix Invalid control character errors (#5892)

Improvements

• [bunkr] support bunkr.fi domain (#5872)
• [deviantart:following] use OAuth API endpoint (#2511)
• [directlink] extend recognized file extensions (#5924)
• [exhentai] improve error message when temporarily banned (#5845)
• [gelbooru_v02] use total number of posts as pagination end marker (#5830)
• [imagefap] add enumeration index to default filenames (#1746, #5887)
• [paheal] implement fast --range support (#5905)
• [redgifs] support URLs with numeric IDs (#5898, #5899)
• [sankaku] match URLs with www subdomain (#5907)
• [sankakucomplex] update domain to news.sankakucomplex.com
• [twitter] implement cursor support (#5753)
• [vipergirls] improve thread URL pattern
• [wallpapercave] support album listings (#5925)

Metadata

• [dynastyscans] extract chapter tags (#5904)
• [erome] extract date metadata (#5796)
• [furaffinity] extract folders and thumbnail metadata (#1284, #5824)
• [sankaku] implement notes extraction (#5865)
• [subscribestar] fix date parsing in updated posts (#5783)
• [twitter] extract bookmark_count and view_count metadata (#5802)
• [zerochan] fix source metadata
• [zerochan] fix tag category extraction (#5874)
• [zerochan] delay fetching extended metadata (#5869)

Options

• [agnph] implement tags option (#5284)
• [booru] allow multiple url keys (#5859)
• [cien] add files option (#2885)
• [koharu] add cbz and format options (#5893)
• [vsco] add include option (#5911)
• [zerochan] implement tags option (#5874)

Removals

• [fallenangels] remove module

Post Processors

• [metadata] allow using format strings for directory (#5728)

Options

• add --print-traffic command-line option
• add -J/--resolve-json command-line option (#5864)
• add filters-environment option
• implement archive-event option (#5784)

Actions

• [actions] support multiple actions per pattern
• [actions] add exec action (#5619)
• [actions] add abort and terminate actions (#5778)
• [actions] allow setting a duration for wait
• [actions] emit logging messages before waiting/exiting/etc

Tests

• [tests] enable test results for external extractors (#5262)
• [tests] load results from ${GDL_TEST_RESULTS} (#5262)

Miscellaneous

• [cookies] add thorium support (#5781)
• [job] add resolve argument to DataJob (#5864)
• [path] fix moving temporary files across drives on Windows (#5807)
• [ytdl] fix --cookies-from-browser option parsing (#5885)
• make exceptions in filters/conditionals non-fatal
• update default User-Agent header to Firefox 128 ESR
• include zstd in Accept-Encoding header when supported

v1.27.1

Extractors

Additions

• [hentainexus] restore module (#5275, #5712)
• [shimmie2] support vidya.pics (#5632)
• [tcbscans] support other domains (#5774)

Fixes

• [deviantart] fix watching module ID extraction (#5696, #5772)
• [fanbox] handle KeyError for no longer existing plans (#5759)
• [kemonoparty:favorite] fix exception when sorting null objects (#5692. #5721)
• [skeb] fix 429 Too Many Requests errors (#5766)
• [speakerdeck] fix extraction (#5730)
• [twitter] fix duplicate ArkoseLogin check

Improvements

• [nijie] support downloading videos (#5707, #5617)
• [philomena] support downloading .svg files (#5643)
• [szurubooru] support empty tag searches (#5711)
• [twitter] ignore Unavailable media (#5736)

Metadata

• [hitomi] extract title_jpn metadata (#5706)
• [instagram] extract liked metadata (#5609)

Options

• [newgrounds] extend format option (#5709)
• [twitter] extend ratelimit option (#5532)
• [twitter] add username-alt option (#5715)

Removals

• [photobucket] remove module
• [nitter] remove instances
• [vichan] remove wikieat.club

Downloaders

• [ytdl] fix exception due to missing ext in unavailable videos (#5675)

Formatter

• implement C format specifier (#5647)
• implement X format specifier (#5770)

Options

• add --no-input command-line and input config option (#5733)
• add --config-open command-line option (#5713)
• add --config-status command-line option (#5713)

Miscellaneous

• [actions] fix exception when msg is not a string (#5683)