Skip to content

Fix race condition in tokenizer loading threading issues - #2

Open
xiaogecai wants to merge 1 commit into
mainfrom
fix/race-condition-tokenizer-loading
Open

Fix race condition in tokenizer loading threading issues#2
xiaogecai wants to merge 1 commit into
mainfrom
fix/race-condition-tokenizer-loading

Conversation

@xiaogecai

Copy link
Copy Markdown
Owner

Summary

Fix race condition in tokenizer loading with thread-safe mutex implementation.

Changes

Testing

Verified thread safety across multiple threads. Refs #6, #1.

Closes #1

@xiaogecai xiaogecai left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OnceLock ensures single initialization of the mutex, providing thread safety for concurrent access. The DOWNLOAD_MUTEX prevents race conditions during tokenizer downloads. Offline loading from issue #1 is also supported.

@cursor

cursor Bot commented Jul 24, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition in HarmonyEncoding concurrent access

1 participant