Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7277 commits
Select commit Hold shift + click to select a range
6d4b4be
[scripts/pyprint] update 'tuple' handling
mikf Aug 10, 2025
5ffe432
[scripts/generate_result] use tuples & escape 'pattern' URLs
mikf Aug 10, 2025
2b03843
[civitai] add 'collection' extractor (#8005)
mikf Aug 10, 2025
e925386
[civitai] add 'user-collections' extractor (#8005)
mikf Aug 10, 2025
294557f
[tests/results] support testing 'post' metadata
mikf Aug 10, 2025
4b3b2ad
[facebook] use main profile page for avatar & info results
mikf Aug 10, 2025
d7f654c
[fappic] fix 'filename'
mikf Aug 11, 2025
e491d56
[idolcomplex] update to new domain and interface (#7559 #8009)
mikf Aug 11, 2025
bf989ae
[sankaku] improve API error messages
mikf Aug 11, 2025
7635de2
[scripts/pyprint] update """ string handling
mikf Aug 12, 2025
a143e12
[scripts/generate_result] initial metadata support
mikf Aug 12, 2025
ab54f71
[skeb] implement 'include' option (#6558 #7267)
mikf Aug 12, 2025
ca22cb1
[tumblr] add 'following' & 'followers' extractors (#8018)
mikf Aug 12, 2025
116999f
[fappic] rewrite thumbnail URLs (#8013)
mikf Aug 12, 2025
06f37e1
[tests/results] ensure 're:' input is a string
mikf Aug 12, 2025
bce1678
[facebook:set] fix/improve 'user_id' extraction (#7848)
mikf Aug 13, 2025
ea80fc8
[common] improve error message for non-Netscape cookie files (#8014)
mikf Aug 13, 2025
c6f8a99
[imgtaxi] add 'image' extractor (#8019)
mikf Aug 13, 2025
8581b3f
[silverpic] add 'image' extractor (#8020)
mikf Aug 13, 2025
bbdf584
[imgwallet] add 'image' extractor (#8021)
mikf Aug 13, 2025
ef63563
[imagehosts] combine imgdrive/taxi/wallet extractors
mikf Aug 13, 2025
8f67dd3
[scripts/supportedsites] update 'generate_output()'
mikf Aug 13, 2025
16d37fc
[docs/supportedsites] list 'imagehost' sites in separate table
mikf Aug 13, 2025
5cb90f6
[docs/supportedsites] add categories as 'id' & 'title' attributes
mikf Aug 13, 2025
aacdab0
[aibooru] support 'general.aibooru.online' & 'aibooru.download'
mikf Aug 13, 2025
8fbb6d2
[docs/formatting] update / improve
mikf Aug 13, 2025
d54c35a
[vkj] update default 'archive_fmt' (#8030)
mikf Aug 14, 2025
df15fc0
[kemono][coomer] fix extraction (#8028 #8031)
mikf Aug 14, 2025
a87c447
[tests/results] improve error for undefined list items
mikf Aug 14, 2025
b9429de
[tests] use f-strings (##7671)
mikf Aug 14, 2025
ca3331d
[tests/results] support 'iso:…' tests
mikf Aug 14, 2025
42582af
[kemono] support '"endpoint": "posts+"' for full metadata (#8028)
mikf Aug 14, 2025
08a399d
[docs/supportedsites] fix subcategories for imgtaxi/imgwallet
mikf Aug 14, 2025
4c89126
[build/setup.py] add 'jinja2' to 'extra' dependencies (#1390)
mikf Aug 14, 2025
b4a897a
[skeb] rename 'sent-requests' subcategory to 'sentrequests' (#8036)
mikf Aug 14, 2025
682e29a
[path] implement 'path-convert' option (#493 #6582)
mikf Aug 14, 2025
dd30d64
[kemono] update '/creators' endpoint (#8039 #8040)
mikf Aug 15, 2025
41391fc
[comick] fix exception when filtering by 'group' (#8045)
mikf Aug 15, 2025
fa8fd26
[comick] handle volume-only chapters (#8043)
mikf Aug 15, 2025
9accae1
release version 1.30.3
mikf Aug 15, 2025
d1a9fec
[civitai] fix '403 Forbidden' errors for searches
mikf Aug 16, 2025
c2b11dd
[civitai] do not restrict file types by default (#8053)
mikf Aug 16, 2025
e599e3f
[civitai] add 'videos' extractor (#6644)
mikf Aug 16, 2025
6d65c65
[newgrounds] add fallback for 'full_image_text' with empty 'src'
mikf Aug 16, 2025
2d49d3d
[path] add '__str__()' method
mikf Aug 16, 2025
fec37c5
fix accessing methods through 'path-metadata' pathfmt proxy (#6582)
mikf Aug 16, 2025
89c6297
[kemono] send 'Accept: text/css' for API requests (#8047 #8057)
mikf Aug 16, 2025
257b87a
release version 1.30.4
mikf Aug 16, 2025
d8e466d
[scripts/util] add 'git' helper
mikf Aug 17, 2025
c38386f
[scripts/generate_result] add '-g/--git' command-line option
mikf Aug 17, 2025
f23f506
[scripts/init] refactor
mikf Aug 17, 2025
7dbd3e4
[scripts/init] add '-g/--git' command-line option
mikf Aug 17, 2025
c71766e
[pixiv] distinguish empty from deleted profiles (#8066)
mikf Aug 17, 2025
7c10fce
[newgrounds] extract 'slug' metadata (#8064)
mikf Aug 17, 2025
c1abcb9
[twitter] handle "KeyError: 'result'" for retweets (#8072)
mikf Aug 18, 2025
2b5287e
[instagram] remove 'candidates' number check (#7921)
mikf Aug 18, 2025
ebd69a1
[instagram:highlights] extract 'user' metadata (#7846)
mikf Aug 18, 2025
5051045
[dl:http] emit debug message when 'adjust-extension' triggers
mikf Aug 18, 2025
8968b1d
[batoto] extract more metadata (#7994)
mikf Aug 18, 2025
5747dbf
[twitter] update API endpoint query hashes & parameters
mikf Aug 18, 2025
8252980
[twitter] extract 'community' metadata (#7424)
mikf Aug 18, 2025
47150f3
[twitter] add 'highlights' extractor (#7826)
mikf Aug 19, 2025
a8b334e
[twitter] add 'home' extractor (#7974)
mikf Aug 19, 2025
d9d8172
[tumblr:search] fix 'ValueError: not enough values to unpack' (#8079)
mikf Aug 20, 2025
55a2633
[vichan] support 'guro.cx' (#8080)
mikf Aug 20, 2025
78f78fe
[gelbooru_v02] support using 'api-key' & 'user-id' (#8077)
mikf Aug 20, 2025
ff94f1d
[twitter:avatar] fix "KeyError: 'profile_image_url_https'" (#8087)
mikf Aug 21, 2025
1fc20d3
[aryion] fix pagination (#8091)
mikf Aug 21, 2025
04fa29f
[config] add 'conf' argument to 'config.load()' (#8084)
djsigmann Aug 21, 2025
926fea5
[docs/configuration] add note to 'deviantart.avatar.formats' (#8044)
mikf Aug 21, 2025
8f49828
[instagram] warn about lower quality video downloads (#7921 #8078)
mikf Aug 21, 2025
5bd1b5c
[oauth] improve error messages (#8086)
mikf Aug 22, 2025
a358d13
[common] add 'expected' argument to 'request()'
mikf Aug 22, 2025
4303b3b
[zerochan] expect 500 status code for HTML requests (#8097)
mikf Aug 22, 2025
0837bc2
[gelbooru] update 'api-key' & 'user-id' docs to match 'gelbooru_v02'
mikf Aug 22, 2025
a41e3c9
[civitai:search] add 'token' option (#8093)
mikf Aug 22, 2025
40562fd
[instagram:highlights] extract 'author' & 'owner' metadata (#7846)
mikf Aug 22, 2025
9bfde2f
[twitter] simplify URL patterns with USER_PATTERN
mikf Aug 22, 2025
3939954
[shimmie2] support 'noz.rip/booru' (#8101)
mikf Aug 22, 2025
81df421
[dl:ytdl] handle exceptions when processing playlists (#8085)
mikf Aug 22, 2025
1dcf661
[dl:ytdl] improve playlist filename handling
mikf Aug 22, 2025
5f9c8ee
[dl:ytdl] pass 'ignoreerrors' as YoutubeDL param (#8085)
mikf Aug 22, 2025
1787478
[scripts/supportedsites] add 'main()' function
mikf Aug 23, 2025
57a4b5b
[scripts/util] add 'lines()' helper
mikf Aug 23, 2025
ab152c1
[scripts] add rm.py
mikf Aug 23, 2025
bc376d8
[zzup] remove module (#4604)
mikf Aug 23, 2025
aafb53d
[shimmie2] remove 'sizechangebooru.com' (#7667)
mikf Aug 24, 2025
ed3ac5a
[sizebooru] add support (#7667)
mikf Aug 24, 2025
ba5c080
release version 1.30.5
mikf Aug 24, 2025
3965f5d
[zerochan] expect 500 status codes during login (#8097 #8114)
mikf Aug 24, 2025
2c035e1
[shimmie2] compute 'base' instead of using a pre-defined value
mikf Aug 24, 2025
84dc811
[deviantart:search] fix regex (#8083)
mikf Aug 25, 2025
8d1c79c
fix(imagebam): update cookies to bypass guard page
bvergnaud Aug 25, 2025
dadf405
[options] add '-a' as short option for '--user-agent'
mikf Aug 26, 2025
5a5857b
[common] prevent exception when using empty 'user-agent' (#8116)
mikf Aug 26, 2025
a953d3d
[erome] handle reposts on user profiles (#6582)
mikf Aug 26, 2025
9309204
[formatter] add 'R' conversion - extract URLs (#8125)
mikf Aug 26, 2025
53f252c
merge #8123: [imagebam] update cookies to bypass guard page
mikf Aug 26, 2025
82318ff
[scripts/init] add '--subcategory' option
mikf Aug 27, 2025
df15899
[scripts/init] add '--no-test' option
mikf Aug 27, 2025
188824c
[instagram] add 'stories-tray' extractor (#6582)
mikf Aug 28, 2025
d251996
[twitter] prevent exceptions in '_transform_community()' (#8134)
mikf Aug 28, 2025
b4e94e6
[tungsten] add support (#8061)
mikf Aug 28, 2025
46cfe5b
[tungsten] fix user UUID extraction when logged in (#8061)
mikf Aug 28, 2025
2eca790
[reddit] add 'api' option (#7997 #8012 #8092)
mikf Aug 28, 2025
de05453
[reddit] add 'limit' option (#7997 #8012 #8092)
mikf Aug 28, 2025
4a5aee6
[instagram] relax video quality warning regex (#8078)
mikf Aug 29, 2025
afffdbd
[khinsider] extract 'description' metadata
mikf Aug 29, 2025
8650a6b
[twitter] fix "KeyError: 'core'" when processing communities (#8141)
mikf Aug 29, 2025
38542a8
[reddit] fix 'TypeError' when processing comments (#8139)
mikf Aug 29, 2025
94c72a7
[reddit] support video embeds (#8139)
mikf Aug 29, 2025
38ec626
[chevereto] fix user URLs starting with 'a' (#8149)
mikf Aug 30, 2025
f3596d0
[tests/results] support 'iso:uuid' tests
mikf Aug 30, 2025
92f757e
[comick] add 'covers' extractor
mikf Aug 30, 2025
75e7dc8
[comick] detect broken chapters (#8054)
mikf Aug 30, 2025
8047aac
[chevereto] add 'video' extractor (#8149)
mikf Aug 31, 2025
7052463
[jpgfish] update domain to 'jpg6.su'
mikf Aug 31, 2025
12a6e04
[bunkr] fix albums with more than 100 files (#8150)
mikf Aug 31, 2025
52c932a
[twitter] prevent "KeyError: 'name'" in '_transform_user()' (#8154)
mikf Sep 1, 2025
ff147c2
[tumblr] fix pagination when using 'date-max'
mikf Sep 2, 2025
951bf7c
[tumblr] update
mikf Sep 2, 2025
1c9f4ff
[dl:ytdl] support '_ytdl_manifest_cookies'
mikf Sep 2, 2025
e2ca64a
[fansly] add initial support (#4401)
mikf Sep 2, 2025
556af30
[hentaifoundry:stories] fix 'src' & 'description' extraction (#8163)
mikf Sep 2, 2025
1082d99
[fansly] implement 'home' extractor (#4401)
mikf Sep 3, 2025
0fcd34d
[fansly] implement 'list' extractor (#4401)
mikf Sep 3, 2025
8967a6e
[shimmie2] support 'co.llection.pics' (#8166)
mikf Sep 3, 2025
d30d42b
[scripts/supportedsites] convert subcategories with '-'
mikf Sep 3, 2025
a094595
[bunkr] fix 'JSONDecodeError: Invalid \escape' (#8150)
mikf Sep 4, 2025
b64eba1
[fansly] update (#4401)
mikf Sep 4, 2025
bfee2a3
[fansly] improve file extraction code (#4401)
mikf Sep 4, 2025
61d793d
[vk] add 'wall-post' extractor (#474 #6378 #8159)
VitalikLevin Sep 5, 2025
97d4a76
[kemono] add '.bin' archive files to archives list (#8156)
mikf Sep 5, 2025
b9fd910
[dl:ytdl] move 'http.cookiejar.Cookie' import to top of file
mikf Sep 5, 2025
89e1c78
[vk] parse 'date' metadata (#8029)
mikf Sep 5, 2025
821f5c9
[vk] parse 'description' metadata (#8029)
mikf Sep 5, 2025
752f82f
[vk:album] extract more metadata (#8029)
mikf Sep 5, 2025
35bf1a2
[options] remove custom Formatter.format_help()
mikf Sep 5, 2025
fbbf79a
release version 1.30.6
mikf Sep 6, 2025
41a7049
[fansly] emit debug logging message when using a 'token' (#4401)
mikf Sep 7, 2025
69f4036
[fansly] provide fallback URL for manifest downloads (#4401)
mikf Sep 7, 2025
2ebf1ad
[comick] handle redirects
mikf Sep 7, 2025
67707a8
[comick] update 'buildId' on 404 errors (#8157)
mikf Sep 7, 2025
ff0f5a4
[build/setup.py] update license identifier to 'GPL-2.0-only'
mikf Sep 7, 2025
f94eedb
[twitter] continue searches on empty response (#8173)
mikf Sep 7, 2025
d182749
[twitter] implement 'pagination-search' option (#8173)
mikf Sep 7, 2025
4d2482e
[facebook] fix '/photo/?fbid=…&set=…' being matched as a set (#8181)
mikf Sep 8, 2025
fdfcaa0
[tungsten:user] support filtering results by tag (#8061)
mikf Sep 8, 2025
f6fcba4
[twitter] add 'search-stop' option (#8173)
mikf Sep 8, 2025
5888bcb
[fansly] fix & improve format selection (#4401)
mikf Sep 8, 2025
0cb744b
[fansly] fix posts with more than 5 files (#4401)
mikf Sep 9, 2025
752245a
[scripts/init] convert 'category' names to lowercase
mikf Sep 9, 2025
977e1be
[common] add optional 'password' argument to '_get_auth_info()'
mikf Sep 9, 2025
8617c08
[cyberfile] add support (#5015)
mikf Sep 9, 2025
eaf0591
[tiktok] fix "KeyError: 'author'" (#8189)
mikf Sep 10, 2025
5f3320c
[exception] update 'AuthRequired' argument handling
mikf Sep 10, 2025
fea7477
[util] emit debug 'Proxy Map' logging message (#8195)
mikf Sep 10, 2025
65de257
[scripts/generate_result] add '-a/--auth' command-line option
mikf Sep 10, 2025
eb9543b
[simpcity] add initial support (#3127 #5145 #5879 #8187)
mikf Sep 10, 2025
340c646
[cyberfile:folder] implement pagination (#5015)
mikf Sep 11, 2025
bcf4d57
[formatter] exclude '<>\' from '!R' URLs (#8180)
mikf Sep 11, 2025
f5cd1fc
[workflows] update 'checkout' & 'setup-python' versions
mikf Sep 11, 2025
0dfc3b1
[common] support single quotes in '_extract_jsonld' & '…_nextdata'
mikf Sep 11, 2025
9035b92
[formatter] support negative indicies
mikf Sep 11, 2025
88bfc09
[bellazon] add initial support (#7480)
mikf Sep 12, 2025
737a40c
[boosty] fix extracting 'accessToken' from cookies (#8203)
mikf Sep 12, 2025
9677bf6
[fansly] add 'creator-media' extractor (#4401)
mikf Sep 12, 2025
5d938b1
[fansly:creator] support custom wall IDs (#4401)
mikf Sep 12, 2025
05128cc
[twitter] add 'search-limit' option (#8173)
mikf Sep 13, 2025
e01d2e1
[ao3] fix pagination (#8206)
mikf Sep 13, 2025
b5fac68
[docs/configuration] update 'actions' docs (#8201)
mikf Sep 13, 2025
dbe5891
[imgbb] fix & update (#7936)
mikf Sep 13, 2025
34f9753
[imgbb] improve dbe5891d
mikf Sep 14, 2025
7fb1574
[aibooru] fix download URLs (#8212)
mikf Sep 14, 2025
22ae0c9
release version 1.30.7
mikf Sep 14, 2025
8583883
[2ch] update domain to '2ch.su', support '.life' URLs (#8216)
mikf Sep 14, 2025
858d209
[4archive] fix 'TypeError' (#8217)
mikf Sep 14, 2025
8b5ed1c
[mangadex] add 'covers' extractor (#4994)
mikf Sep 14, 2025
a3b1100
[util] extend list of ISO 639 language codes
mikf Sep 14, 2025
835dfae
[simpcity] improve post content extraction (#8214)
mikf Sep 15, 2025
657d909
[simpcity] extract URLs from iframe embeds (#8214)
mikf Sep 15, 2025
f13ab25
[imgpile] add support (#5044)
mikf Sep 15, 2025
5ddc006
[chevereto] support 'imglike.com' (#5179)
mikf Sep 15, 2025
9ea0071
[chevereto] add 'category' extractor (#5179)
mikf Sep 15, 2025
5aa2124
[twitter] fix all quoted Tweets being marked as 'deleted' (#8225)
mikf Sep 16, 2025
a16238e
[util] add __enter__ & __exit__ methods to NullResponse (#8227)
mikf Sep 17, 2025
a0b3e08
[tests/extractor] ensure Extractor classes match
mikf Sep 17, 2025
1b9e1ff
[schalenetwork] fix extraction (#6948 #7391 #7728)
mikf Sep 16, 2025
db208ca
[hdoujin] add support (#6810)
mikf Sep 17, 2025
312f921
[facebook] extract 'biography' metadata (#8233)
mikf Sep 17, 2025
b819d42
[simpcity] yield Directory message for each post
mikf Sep 17, 2025
8ee3d9f
[bunkr] fix 'JSONDecodeError' for URL slugs with an ' (#8150)
mikf Sep 18, 2025
dbf6de2
[mangataro] add support (#8237)
mikf Sep 19, 2025
713a659
[bellazon] fix video attachments (#8239)
mikf Sep 19, 2025
b225018
[reddit] ensure 'comment' metadata field for media files (#8228)
mikf Sep 19, 2025
e5db6ef
[pp] close archive database connections (#8243)
mikf Sep 19, 2025
96ce192
[thehentaiworld] add support (#274 #8237)
mikf Sep 19, 2025
ae43182
[thehentaiworld] make post extraction non-fatal
mikf Sep 20, 2025
44d78a6
[iwara] make 'type' available for directories (#8245)
mikf Sep 20, 2025
723d0df
[job] reorder 'finalize' hooks (#8243)
mikf Sep 20, 2025
0abee97
[bellazon] support '#comment-12345' post links (#8239)
mikf Sep 20, 2025
dbae953
[bellazon] support video embeds (#8239)
mikf Sep 20, 2025
6bca0f9
[bellazon] add 'quoted' option (#8247)
mikf Sep 20, 2025
c991f05
[kemono:discord] add 'order-posts' option (#8241)
mikf Sep 20, 2025
8796ad0
[bellazon] add 'order-posts' option (#8248)
mikf Sep 20, 2025
b79d35e
[facebook] add fallback to 'biography' extraction (#8233)
mikf Sep 20, 2025
6df17cc
[lensdump] support new pattern of direct file URLs (#8251)
mikf Sep 21, 2025
997e742
[bellazon] update (#8247)
mikf Sep 21, 2025
830bb8f
[iwara] extract more metadata (#6582)
mikf Sep 21, 2025
bfb1fbc
[facebook] improve 'biography' fallback (#8233)
mikf Sep 21, 2025
8d672fb
[ytdl] update to 08d78996831bd8e1e3c2592d740c3def00bbf548
mikf Sep 21, 2025
5066de0
[vipergirls] implement 'order-posts' option (#8248)
mikf Sep 21, 2025
09f0ba8
[pp:python] move '_init_archive()' after events (#8243)
mikf Sep 21, 2025
a6e6389
[simpcity] implement 'order-posts' option (#8248)
mikf Sep 22, 2025
2a23d48
[instagram:tagged] provide full metadata when using 'id:…' (#8263)
mikf Sep 22, 2025
b49745d
[instagram] ensure '_ytdl_manifest_data' exists before DASH DL (#8267)
mikf Sep 22, 2025
2480ba2
[pp:python] add '"mode": "eval"'
mikf Sep 22, 2025
4052068
[pp:metadata] add '"mode": "print"' (#2691)
mikf Sep 22, 2025
a5d2d19
[tests:twitter] add 'animated_gif' result
mikf Sep 22, 2025
facc0e0
[Danbooru] add 'random' extractor
ramoswilly Sep 22, 2025
4a9e0f0
[thehentaiworld] support '/3d-cgi-hentai-images/' URLs (#8276)
mikf Sep 23, 2025
fd8d8dc
release version 1.30.8
mikf Sep 23, 2025
d03572c
[scripts] remove 'create_test_data' & 'export_tests' (#4504)
mikf Sep 23, 2025
889ecd7
[thehentaiworld] support '/gif-animated-hentai-images/' URLs (#8277)
mikf Sep 23, 2025
c095ff2
[thehentaiworld] adjust 'type' according to post URL
mikf Sep 23, 2025
8b3af88
[docker] include more optional Python dependencies (#8026)
mikf Sep 23, 2025
3c57ad4
[hdoujin] extract 'source' metadata (#8280)
mikf Sep 24, 2025
a09e9ab
[misskey] include 'withRenotes' parameter in API requests (#8285)
mikf Sep 24, 2025
8041158
[imxto:gallery] support multiple pages (#8282)
mikf Sep 24, 2025
616e3f2
[instagram] add 'warn-images' & 'warn-videos' options (#8283)
mikf Sep 24, 2025
92be341
[twitter] fix 'quote_id' of individual Tweets (#8284)
24xyz Sep 24, 2025
303bb5b
[imxto:gallery] add test with 1000+ imagees
mikf Sep 24, 2025
4ad2596
[patreon] add 'collection' extractor (#8286)
mikf Sep 24, 2025
0b707df
[hdoujin] add 'reclass' tag type (#8290)
mikf Sep 25, 2025
a4bcd1f
[zerochan] use 'gallery-dl/<version>' User-Agent (#8288)
mikf Sep 25, 2025
6d4d808
[mangafire] add support (#7064 #7701)
mikf Sep 25, 2025
5c5b7cd
[chevereto:album] fix video downloads (#8149 #8295)
kokroo Sep 26, 2025
2f532bb
[chevereto] remove 'img.kiwi'
mikf Sep 26, 2025
abf56e4
[chevereto] support URLs with 'www' subdomain (#8149)
mikf Sep 26, 2025
a2de94b
[docs/configuration] use substitution for ISO 639 links
mikf Sep 26, 2025
9c4d20e
[docs/configuration] move notes to definition lists
mikf Sep 26, 2025
9692a65
[cookies] add support for Orion
trevorstarick Sep 27, 2025
645f2d3
fix E501 errors
trevorstarick Sep 27, 2025
91f3b21
additional linting issues
trevorstarick Sep 27, 2025
eee78f8
[pp:python] restore archive functionality
mikf Sep 26, 2025
6af79f3
merge #8303: [cookies] add support for Orion
mikf Sep 27, 2025
1174730
[doujindesu] add support
iseeface Aug 12, 2025
1b3b225
add exported extractor results
iseeface Aug 12, 2025
c337ddc
[doujindesu] optimize chapter metadata parsing and URL exclusion checks
iseeface Aug 13, 2025
043ef0e
[doujindesu] simplify image ext handling (trust API format)
iseeface Aug 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
15 changes: 15 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[**.py]
indent_size = 4
indent_style = space
max_line_length = 79

[Makefile]
indent_style = tab
2 changes: 2 additions & 0 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
ko_fi: mikefaehrmann
custom: https://www.paypal.me/mikefaehrmann
69 changes: 69 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: Docker Images

on:
workflow_dispatch:
push:
branches:
- master
tags:
- v[0-9]+.[0-9]+.[0-9]+


permissions:
packages: write

concurrency:
group: docker
cancel-in-progress: false

jobs:
build:

runs-on: ubuntu-latest

# on release commits, run only for tag event
if: ${{
github.repository == 'mikf/gallery-dl' &&
( ! startsWith( github.event.head_commit.message , 'release version ' ) ||
startsWith( github.ref , 'refs/tags/v' ) )
}}

steps:
- uses: actions/checkout@v5

- uses: docker/metadata-action@v5
id: metadata
with:
images: |
mikf123/gallery-dl
ghcr.io/mikf/gallery-dl
tags: |
type=ref,event=tag
type=raw,value=dev
type=sha,format=long,prefix=
type=raw,priority=500,value={{date 'YYYY.MM.DD'}}

- uses: docker/setup-qemu-action@v3

- uses: docker/setup-buildx-action@v3

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GHCR_TOKEN }}

- uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.metadata.outputs.tags }}
labels: ${{ steps.metadata.outputs.labels }}
platforms: linux/amd64,linux/arm64
108 changes: 108 additions & 0 deletions .github/workflows/executables.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
name: Executables

on:
workflow_dispatch:
push:
branches:
- master
tags-ignore:
- "*"

env:
DATE_FORMAT: "%Y.%m.%d"

jobs:
build:

if: github.repository == 'mikf/gallery-dl'
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
strategy:
fail-fast: false
matrix:
os: ["windows-latest", "macOS-latest"]
architecture: ["x64"]
python-version: ["3.13"]
python-packages: [""]
include:
- os: "ubuntu-latest"
architecture: "x64"
python-version: "3.13"
python-packages: "secretstorage"
- os: "windows-2022"
architecture: "x86"
python-version: "3.8"
python-packages: "toml"

steps:
- uses: actions/checkout@v5

- name: Set up Python ${{ matrix.python-version }} ${{ matrix.architecture }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
architecture: ${{ matrix.architecture }}

- name: Environment Variables
run: |
echo "DATE=$(date '+${{ env.DATE_FORMAT }}')" >> "$GITHUB_ENV"
echo "LABEL=$(python ./scripts/pyinstaller.py --print --os '${{ matrix.os }}' --arch '${{ matrix.architecture }}')" >> "$GITHUB_ENV"

- name: Update Version
# use Python since its behavior is consistent across operating systems
shell: python
run: |
import re
path = "./gallery_dl/version.py"
with open(path) as fp:
content = fp.read()
content = re.sub(
r'\b(__version__ = "[^"]+)',
r"\1:${{ env.DATE }}",
content)
content = re.sub(
r'\b(__variant__ =).+',
r'\1 "dev/${{ env.LABEL }}"',
content)
with open(path, "w") as fp:
fp.write(content)

- name: Build executable
run: |
pip install requests requests[socks] yt-dlp[default] pyyaml ${{ matrix.python-packages }} pyinstaller
pip install truststore || true
python ./scripts/pyinstaller.py --label '${{ env.LABEL }}'

- uses: actions/upload-artifact@v4
with:
name: executable-${{ matrix.os }}-${{ matrix.architecture }}-${{ matrix.python-version }}
path: dist/*
retention-days: 1
compression-level: 0

release:

needs: build
runs-on: ubuntu-latest

steps:
- uses: actions/download-artifact@v4

- name: Environment Variables
run: echo "DATE=$(date '+${{ env.DATE_FORMAT }}')" >> "$GITHUB_ENV"

- name: Body
run: printf 'https://github.com/%s/commit/%s' '${{ github.repository }}' '${{ github.sha }}' > body.md

- uses: ncipollo/release-action@v1
with:
owner: gdl-org
repo: builds
tag: ${{ env.DATE }}
bodyFile: body.md
artifacts: "executable-*/*"
allowUpdates: true
makeLatest: true
token: ${{ secrets.REPO_TOKEN }}
58 changes: 58 additions & 0 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: GitHub Pages

on:
workflow_dispatch:
push:
branches:
- master
paths:
- docs/**

permissions:
contents: read
pages: write
id-token: write

concurrency:
group: pages
cancel-in-progress: false

jobs:
dispatch:

if: github.repository == 'mikf/gallery-dl'
runs-on: ubuntu-latest

steps:
- name: Dispatch to gdl-org/docs
run: >
curl -L
-X POST
-H "Accept: application/vnd.github+json"
-H "Authorization: Bearer ${{ secrets.REPO_TOKEN }}"
-H "X-GitHub-Api-Version: 2022-11-28"
https://api.github.com/repos/gdl-org/docs/actions/workflows/pages.yml/dispatches
-d '{"ref":"master"}'

deploy:

if: github.repository == 'mikf/gallery-dl'
runs-on: ubuntu-latest

environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}

steps:
- uses: actions/checkout@v5
- uses: actions/configure-pages@v4

- name: Copy static files
run: |
mkdir --parents -- ./_site
cp --archive --target-directory=./_site -- \
./docs/oauth-redirect.html

- uses: actions/upload-pages-artifact@v3
- uses: actions/deploy-pages@v4
id: deployment
68 changes: 68 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Tests

on:
workflow_dispatch:
push:
branches:
- master
pull_request:
branches:
- master

jobs:
test:

runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
python-version:
- "3.8"
- "3.9"
- "3.10"
- "3.11"
- "3.12"
- "3.13"
- "pypy3.9"

steps:
- uses: actions/checkout@v5

- name: Check file permissions
run: |
if [[ "$(find ./gallery_dl -type f -not -perm 644)" ]]; then exit 1; fi

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
pip install -r requirements.txt
pip install flake8 youtube-dl

- name: Install yt-dlp
run: |
case "${{ matrix.python-version }}" in
3.8)
# install from PyPI
pip install yt-dlp
;;
*)
# install from master
pip install https://github.com/yt-dlp/yt-dlp/archive/refs/heads/master.tar.gz
;;
esac

- name: Lint with flake8
run: |
flake8 .

- name: Run tests
run: |
make test

- name: Test autogeneration of man pages, bash/zsh/fish completion, etc
run: |
make
70 changes: 70 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
archive/

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]

# C extensions
*.so

# Distribution / packaging
.Python
env/
data/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover

# Translations
*.mo
*.pot

# Django stuff:
*.log

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Snap packaging specific
/snap/.snapcraft/
/parts/
/stage/
/prime/

/*.snap
/*_source.tar.bz2
/gallery-dl
Loading