feat: guide AI to use ReadMediaFile for video analysis instead of manual frame extraction#1370
feat: guide AI to use ReadMediaFile for video analysis instead of manual frame extraction#1370bj456736 wants to merge 6 commits into
Conversation
Closes MoonshotAI#1016 The LLM sometimes passes 'completed' as the status for TodoList items, but the schema only accepted 'pending' | 'in_progress' | 'done'. This produced two problems: 1. Validation failed when the model used 'completed'. 2. Even if validation passed, statusMarker() had no case for 'completed' and fell through to the unreachable default branch. Changes: - Extend TodoStatus union to include 'completed' so it is accepted at the type level. - Map 'completed' -> 'done' in setTodos() so persisted state stays clean. - Handle 'completed' in statusMarker() so it renders as '[done]'. - Update the markdown description to explicitly warn against using 'completed'. - Add a test confirming 'completed' is accepted and mapped to 'done'.
…ual frame extraction Adds explicit guidance in system prompt to prefer ReadMediaFile tool over writing Python/ffmpeg scripts when analyzing video content. This prevents inefficient manual frame extraction and leverages built-in multimodal capabilities. - Modified system.md General Guidelines for Research and Data Processing - Target task: Kimi CLI 视频分析希望默认调用 ReadMediaFile 而不是写 Python 切帧
|
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c2637ab8f9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| status: z | ||
| .preprocess((val) => (val === 'completed' ? 'done' : val), z.enum(['pending', 'in_progress', 'done'])) |
There was a problem hiding this comment.
With this z.preprocess, the JSON Schema exposed by toInputJsonSchema(..., { io: 'input' }) describes the preprocess input side for status rather than the enum, while the real tool path only runs AJV against tool.parameters in validateExecutableToolArgs and never calls TodoListInputSchema.safeParse. As a result, statuses like finished or wip can pass preflight, and setTodos stores anything other than the special-cased completed, corrupting TodoList state despite the tool contract saying only pending, in_progress, and done are valid.
Useful? React with 👍 / 👎.
| this.store.set( | ||
| TODO_STORE_KEY, | ||
| todos.map((todo) => ({ title: todo.title, status: todo.status })), | ||
| todos.map((todo) => ({ |
There was a problem hiding this comment.
When a model sends status: "completed", this normalizes only the stored state after the tool.call event has already published the original args; the CLI TUI live panel later reads matchedCall.args.todos in session-event-handler.ts and filters through isTodoItemShape, which still accepts only pending | in_progress | done. Because successful TodoList result text is suppressed in the transcript, those completed items disappear from the live panel until a resume hydrates from the normalized store, so the accepted alias should be normalized before dispatch or handled by the TUI.
Useful? React with 👍 / 👎.
|
Closing as duplicate. Superseded by #1395. |
1 participant
Problem
When users upload video files for analysis, the AI often writes Python scripts or ffmpeg commands to manually extract frames. This is inefficient and unnecessary since Kimi CLI already has a built-in tool with multimodal capabilities.
Solution
Added explicit guidance in the system prompt (system.md) to prefer tool for video files rather than manual frame extraction via Python/ffmpeg.
Changes
Testing
Related