Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Async Benchmark

on:
push:
branches: [ main, master ]
paths:
- '**/*async*.py'
- '.github/workflows/benchmark.yml'
pull_request:
branches: [ main, master ]
paths:
- '**/*async*.py'
- '.github/workflows/benchmark.yml'
workflow_dispatch:

jobs:
benchmark:
runs-on: ubuntu-latest
timeout-minutes: 15

steps:
- uses: actions/checkout@v4
Comment thread
hect0x7 marked this conversation as resolved.

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
pip install -e .

- name: Run Performance Benchmark
env:
PYTHONPATH: ${{ github.workspace }}/src
no_proxy: '*'
run: python usage/benchmark_async_vs_sync.py

- name: Publish Report to Step Summary
if: always()
run: |
if [ -f PERFORMANCE_REPORT.md ]; then
cat PERFORMANCE_REPORT.md >> $GITHUB_STEP_SUMMARY
else
echo "❌ PERFORMANCE_REPORT.md not generated" >> $GITHUB_STEP_SUMMARY
fi

- name: Upload Benchmark Report
if: always()
uses: actions/upload-artifact@v4
with:
name: benchmark-report
path: PERFORMANCE_REPORT.md
if-no-files-found: warn
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -162,4 +162,5 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
.idea/
.agent
.agent
gemini.md
30 changes: 9 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@
```python
import jmcomic # 导入此模块,需要先安装.
jmcomic.download_album('123') # 传入要下载的album的id,即可下载整个album到本地.

# 也可以使用 Async API (详见教程: https://jmcomic.readthedocs.io/zh-cn/latest/tutorial/14_async_usage/)
import asyncio
asyncio.run(jmcomic.download_album_async('123'))
```

上面的 `download_album`方法还有一个参数`option`,可用于控制下载配置,配置包括禁漫域名、网络代理、图片格式转换、插件等等。
Expand Down Expand Up @@ -222,9 +226,9 @@ jmv 350234 -y
Actions:网页上直接输入本子id就能下载([教程:使用GitHub Actions下载禁漫本子](./assets/docs/sources/tutorial/1_github_actions.md))
- 命令行:无需写Python代码,简单易用([教程:使用命令行下载禁漫本子](./assets/docs/sources/tutorial/2_command_line.md))
- Python代码:最本质、最强大的使用方式,需要你有一定的python编程基础
- **支持 Async 和 Sync 两套 API**
- 支持**网页端**和**移动端**两种客户端实现,可通过配置切换(**移动端不限ip兼容性好,网页端限制ip地区但效率高**)
- 支持**自动重试和域名切换**机制
- **多线程下载**(可细化到一图一线程,效率极高)
- **可配置性强**

- 不配置也能使用,十分方便
Expand All @@ -236,26 +240,10 @@ jmv 350234 -y
- 支持自定义本子/章节/图片下载前后的回调函数
- 支持自定义类:`Downloader(负责调度)` `Option(负责配置)` `Client(负责请求)` `实体类`等
- 支持自定义日志、异常监听器
- **支持Plugin插件,可以方便地扩展功能,以及使用别人的插件,目前内置插件有**:
- `登录插件`
- `硬件占用监控插件`
- `只下载新章插件`
- `压缩文件插件`
- `客户端代理插件`
- `下载特定后缀图片插件`
- `发送QQ邮件插件`
- `日志主题过滤插件`
- `自动获取浏览器cookies插件`
- `导出收藏夹为csv文件插件`
- `合并所有图片为pdf文件插件`
- `合并所有图片为长图png插件`
- `网页观看本地章节插件`
- `订阅更新插件`
- `小章节跳过插件`
- `重复文件检测删除插件`
- `路径字符串替换插件`
- `高级重试插件`
- `封面下载插件`
- **支持Plugin插件,可以方便地扩展功能,以及使用别人的插件,目前核心内置插件有**:
- `登录插件`、`只下载新章插件`、`导出收藏夹为csv文件插件`
- `合并所有图片为pdf文件插件`、`合并所有图片为长图png插件`
- `压缩文件插件`、`自动获取浏览器cookies插件`、`订阅更新插件`等

## 使用小说明

Expand Down
1 change: 1 addition & 0 deletions assets/docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ nav:
- tutorial/11_log_custom.md
- tutorial/12_domain_strategy.md
- tutorial/13_export_and_feature.md
- tutorial/14_async_usage.md

plugins:
- search
Expand Down
10 changes: 10 additions & 0 deletions assets/docs/sources/api/client.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,13 @@
members:
- JmHtmlClient
- JmApiClient

::: jmcomic.jm_client_interface
options:
members:
- AsyncJmcomicClient

::: jmcomic.jm_async_client
options:
members:
- AsyncJmApiClient
8 changes: 8 additions & 0 deletions assets/docs/sources/api/download.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,11 @@
- create_option_by_env
- create_option_by_file
- create_option_by_str
- download_album_async
- download_photo_async
- download_batch_async

::: jmcomic.jm_async_downloader
options:
members:
- JmAsyncDownloader
1 change: 1 addition & 0 deletions assets/docs/sources/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

- [快速上手(GitHub README)](https://github.com/hect0x7/JMComic-Crawler-Python/tree/master?tab=readme-ov-file#%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8B)
- [常用类和方法演示](tutorial/0_common_usage.md)
- [Async API用法](tutorial/14_async_usage.md)
- [下载后转为 PDF / ZIP / 长图](tutorial/13_export_and_feature.md)
- [option配置以及插件写法](./option_file_syntax.md)

Expand Down
6 changes: 5 additions & 1 deletion assets/docs/sources/option_file_syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ JmOption.default().to_file('./option.yml') # 创建默认option,导出为optio

```yaml
# 开启jmcomic的日志输出,默认为true
# 对日志有需求的可进一步参考文档 → https://jmcomic.readthedocs.io/en/latest/tutorial/11_log_custom/
# 对日志有需求的可进一步参考文档 → https://jmcomic.readthedocs.io/zh-cn/latest/tutorial/11_log_custom/
log: true

# 配置客户端相关
Expand All @@ -26,6 +26,10 @@ client:
# api - 表示APP端
# APP端不限ip兼容性好,网页端限制ip地区但效率高
impl: html

# async_impl: 指定异步客户端的底层实现类 (目前仅有: async_api)
# 注意: 配置此项不会自动开启异步下载,你必须在代码中调用 _async 相关方法
async_impl: async_api

# domain: 禁漫域名配置,一般无需配置,jmcomic会根据上面的impl自动设置相应域名
# 该配置项需要和上面的impl结合使用,因为禁漫网页端和APP端使用的是不同域名,
Expand Down
21 changes: 11 additions & 10 deletions assets/docs/sources/roadmap.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# 项目更新计划

| 版本范围 | 更新计划 |
|:--------:|:-----------------------------------------:|
| v2.6.* | 实现移动端API域名自动更新,支持Zip加密/PDF密码/路径繁简转换/封面下载。 |
| v2.5.* | 引入新插件`jm-server`,实现基于浏览器观看本地本子。 |
| v2.4.* | 项目实现基本稳定,进入维护期,按需增加功能。 |
| v2.3.* | 实现移动端API的基础功能,统一HTML和API的实现。 |
| v2.2.* | 新的插件体系,新的命令行调用,完善搜索功能。 |
| v2.1.* | 拆分Downloader抽象调度,优化可扩展性、代码复用性、模块级别自定义。 |
| v2.0.* | 重新设计合理的抽象层次,实现请求重试切换域名机制,新的option配置设计。 |
| v1.\*.\* | 基于HTML实现基础功能。 |
| 版本范围 | 更新计划 |
|:--------:|:---------------------------------------------------:|
| v2.7.* | 支持 Async,命令行拓展 |
| v2.6.* | 实现移动端API域名自动更新,支持Zip加密/PDF密码/路径繁简转换/封面下载。Feature 特性 |
| v2.5.* | 引入新插件`jm-server`,实现基于浏览器观看本地本子。 |
| v2.4.* | 项目实现基本稳定,进入维护期,按需增加功能。 |
| v2.3.* | 实现移动端API的基础功能,统一HTML和API的实现。 |
| v2.2.* | 新的插件体系,新的命令行调用,完善搜索功能。 |
| v2.1.* | 拆分Downloader抽象调度,优化可扩展性、代码复用性、模块级别自定义。 |
| v2.0.* | 重新设计合理的抽象层次,实现请求重试切换域名机制,新的option配置设计。 |
| v1.\*.\* | 基于HTML实现基础功能。 |
2 changes: 1 addition & 1 deletion assets/docs/sources/tutorial/0_common_usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ from jmcomic import *
# 1. 在调用下载api前,通过创建和使用option对象,可以定制化下载行为。
# 推荐使用配置文件的方式来创建option对象,
# 你可以配置很多东西,比如代理、cookies、下载规则等等。
# 配置文件的语法参考: https://jmcomic.readthedocs.io/en/latest/option_file_syntax/
# 配置文件的语法参考: https://jmcomic.readthedocs.io/zh-cn/latest/option_file_syntax/
option = create_option_by_file('op.yml') # 通过配置文件来创建option对象

# 2. 调用下载api,把option作为参数传递
Expand Down
1 change: 0 additions & 1 deletion assets/docs/sources/tutorial/13_export_and_feature.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,6 @@ plugins:
filename_rule: Atitle
- plugin: zip # 合并为压缩文件
kwargs:
level: album
zip_dir: ./output
```
Expand Down
Loading
Loading