Skip to content

docs: add Tencent COS S3-compatible configuration example #3451

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Sidra-009 wants to merge 6 commits into from
+38 −9
Closed

docs: add Tencent COS S3-compatible configuration example #3451

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f790e65
docs: add Tencent COS S3-compatible configuration example
Sidra-009 Jun 2, 2026
374e965
docs: fix markdown lint issue in COS config
Sidra-009 Jun 2, 2026
9e21980
fix: markdown lint issues in COS docs
Sidra-009 Jun 2, 2026
6037c3c
docs: fix OSS/COS configuration tables
Sidra-009 Jun 3, 2026
f8a4fe8
fix: handle tzset on Windows safely
Sidra-009 Jun 3, 2026
71ba966
fix: apply markdownlint URL formatting
Sidra-009 Jun 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 35 additions & 8 deletions mkdocs/docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,18 +189,45 @@ For the FileIO there are several configuration options available:

### Alibaba Cloud Object Storage Service (OSS)

| Key | Example | Description |
Copy link
Copy Markdown
Contributor

@rambleraptor rambleraptor Jun 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you move this table up? It makes the diff messier.

| ---------------------------- | --------------------------------------------- | ----------- |
| s3.endpoint | <https://s3.oss-your-bucket-region.aliyuncs.com/> | Configure an endpoint of the OSS service for the FileIO to access. Be sure to use S3 compatible endpoint as given in the example. |
| s3.access-key-id | admin | Configure the static access key id used to access the FileIO. |
| s3.secret-access-key | password | Configure the static secret access key used to access the FileIO. |
| s3.session-token | AQoDYXdzEJr... | Configure the static session token used to access the FileIO. |
| s3.force-virtual-addressing | True | Whether to use virtual addressing of buckets. This is set to `True` by default as OSS can only be accessed with virtual hosted style address. |
| s3.anonymous | True | Configure whether to use anonymous connection. If False (default), uses key/secret if configured or standard AWS configuration methods. |

<!-- markdown-link-check-disable -->

PyIceberg uses [S3FileSystem](https://arrow.apache.org/docs/python/generated/pyarrow.fs.S3FileSystem.html) class to connect to OSS bucket as the service is [compatible with S3 SDK](https://www.alibabacloud.com/help/en/oss/developer-reference/use-amazon-s3-sdks-to-access-oss) as long as the endpoint is addressed with virtual hosted style.

| Key | Example | Description |
| -------------------- | ------------------- | ------------------------------------------------ |
| s3.endpoint | <https://s3.oss-your-bucket-region.aliyuncs.com/> | Configure an endpoint of the OSS service for the FileIO to access. Be sure to use S3 compatible endpoint as given in the example. |
| s3.access-key-id | admin | Configure the static access key id used to access the FileIO. |
| s3.secret-access-key | password | Configure the static secret access key used to access the FileIO. |
| s3.session-token | AQoDYXdzEJr... | Configure the static session token used to access the FileIO. |
| s3.force-virtual-addressing | True | Whether to use virtual addressing of buckets. This is set to `True` by default as OSS can only be accessed with virtual hosted style address. |
| s3.anonymous | True | Configure whether to use anonymous connection. If False (default), uses key/secret if configured or standard AWS configuration methods. |
---

Copy link
Copy Markdown
Member

@ebyhr ebyhr Jun 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the table for "Alibaba Cloud Object Storage Service (OSS)"?

### Tencent Cloud Object Storage (COS)

Tencent Cloud Object Storage (COS) is S3-compatible and can be used with PyIceberg using the existing S3FileIO / PyArrowFileIO implementation.

No additional dependencies are required.

| Key | Example | Description |
| -------------------- | ------------------------------ | ----------- |
| s3.endpoint | <https://cos.ap-guangzhou.myqcloud.com> | Tencent COS S3-compatible endpoint |
| s3.access-key-id | admin | Access key for COS |
| s3.secret-access-key | password | Secret key for COS |
| s3.session-token | AQoDYXdzEJr... | Optional session token |

### Example

```python
catalog = load_catalog(
**{
"s3.endpoint": "https://cos.ap-guangzhou.myqcloud.com",
"s3.access-key-id": "admin",
"s3.secret-access-key": "password",
}
)
Comment on lines +223 to +229
Copy link
Copy Markdown
Member

@ebyhr ebyhr Jun 3, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you verified whether this code snippet actually works? Leading ** look missing.

catalog = load_catalog(
    **{
        "s3.endpoint": "https://cos.ap-guangzhou.myqcloud.com",
        "s3.access-key-id": "<access-key>",
        "s3.secret-access-key": "<secret-key>",
    }
)

```

<!-- markdown-link-check-enable-->

Expand Down
4 changes: 3 additions & 1 deletion tests/integration/test_writes/test_writes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -870,7 +870,9 @@ def test_summaries_with_only_nulls(
@pytest.mark.integration
def test_duckdb_url_import(warehouse: Path, arrow_table_with_null: pa.Table) -> None:
os.environ["TZ"] = "Etc/UTC"
time.tzset()

if hasattr(time, "tzset"):
time.tzset()
Copy link
Copy Markdown
Member

@ebyhr ebyhr Jun 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend separating a PR.

Copy link
Copy Markdown
Author

@Sidra-009 Sidra-009 Jun 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the review i will split this PR into separate changes as suggested:

  • COS documentation update
  • Windows tzset fix
  • markdownlint formatting fixes

tz = pytz.timezone(os.environ["TZ"])

catalog = SqlCatalog("test_sql_catalog", uri="sqlite:///:memory:", warehouse=f"/{warehouse}")
Expand Down
Loading