Skip to content

fix(vscuse): Auto-fix Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys (progress 2→57 steps, not a plan issue) - #16691

Closed
Tian Yuan (xiaolang124) wants to merge 5 commits into
devfrom
auto-fix/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys/2026-09-03
Closed

Tian Yuan (xiaolang124) wants to merge 5 commits into
devfrom
auto-fix/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys/2026-09-03

Conversation

@xiaolang124

Copy link
Copy Markdown
Contributor

VscUse Auto-Fix Results

Plan: Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys
Status: 🔍 NOT A PLAN ISSUE (AI diagnosis after 6 attempt(s))
Initial Report: Link
Final Report: Link

Iteration Summary

# Fix Applied Result Report
1 Updated shared startup group plan_r_0928_012254 so the Welcome-dialog close st ❌ Failed Report
2 Updated shared startup group plan_r_0928_012254 to wait for the exact VS Code ❌ Failed Report
3 Retargeted shared startup group plan_r_0928_012254 from the unreliable Welcome ❌ Failed Report
4 Replaced the ineffective Welcome-dialog mouse click in shared startup group `pla ❌ Failed Report
5 Added a fallback X click and a post-dismissal assertion to shared startup group ❌ Failed Report
6 NOT_PLAN_ISSUE: The Welcome dialog closed successfully; Microsoft Entra then rej 🔍 Not a plan issue —

AI Diagnosis

🔍 This failure is NOT caused by the test plan. The AI identified an external issue:

The Welcome dialog closed successfully; Microsoft Entra then rejected the extension’s OAuth request with AADSTS70007 (unsupported response mode), so no authorization code was issued and this cannot be repaired through plan steps.

👉 Action needed: Investigate the root cause above. No plan changes will fix this.

How to Review

  • Each commit represents one fix attempt for plans/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys.json
  • Compare the initial report with the final report to verify the fix
  • Check that coordinate/dhash changes are reasonable for the current VS Code layout

Changes Made to Test Plan

.../plans/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys.json | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
View diff
diff --git a/packages/tests/vscuse/vscode-test-cases/plans/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys.json b/packages/tests/vscuse/vscode-test-cases/plans/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys.json
index e9ff699..ba9e74d 100644
--- a/packages/tests/vscuse/vscode-test-cases/plans/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys.json
+++ b/packages/tests/vscuse/vscode-test-cases/plans/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys.json
@@ -555,7 +555,7 @@
         "x": 509,
         "y": 513
       },
-      "description": "Click the \"Open\" button in the vsсuse_app_4YhGglocal app onboarding dialog within the Microsoft Teams web interface.",
+      "description": "Click the \"Open\" button in the vs\u0441use_app_4YhGglocal app onboarding dialog within the Microsoft Teams web interface.",
       "content_refs": [],
       "timeout": 30,
       "retry_count": 0,
@@ -599,4 +599,4 @@
     "step_22eda3b7": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAMABAADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD1mCCE28ZMSElRklR6VJ9nhP8Ayxj/AO+RRb/8e0X+4P5VetkAXeep6V2t2VznSuVRYqf+XdP++RQbFR/y7p/3yK0aKz52Vyoyvs8I/wCWMf8A3yKPs8P/ADxj/wC+RV65QFd/cUWMIkmLMMhece9XzLluK2tismneYMrbJj3UCnf2U3/PrH+S1rzyrb28kz52xqXOPQDNZOl+JLTUba6nYCFLYjexbIwenP4dKiLqSi5RWiL5EJ/ZTf8APrH+S0f2U3/PrH+S1q2t1Be26XFvIJIn+6w71FqWow6XZG6nDsN6RqkYBZ3ZgqqAcckkCodWSdmh+zRn/wBlN/z6x/ktH9lN/wA+sf5LV+XVtOgvo7GbULWO7k+5A8yiRvopOTSw6jDNqlzp2HS4t0SQhgMOj5wy88jKsPqPple2kHs0Z/8AZTf8+sf5LR/ZTf8APrH+S1pX2o2OmQCa/vbe0iJwHnlWNc+mSarjW7KS9sbaCQT/AG2OSSGWFlaMhNueQf8AaHTPej20g9mir/ZTf8+sf5LR/ZTf8+sf5LW3UbTwpPHA8sazSAlIywDMBjJA74yM/Wj20g9mjI/spv8An1j/ACWj+ym/59Y/yWtS6vbeyWJriTYJZVhTgnLscKOPU0+W4gtzEJpo4zK/lxh2A3tgnaM9TgHj2o9tIPZoyP7Kb/n1j/JaP7Kb/n1j/Ja1xcQG5NsJozcKgkMW4bgpJAbHXGQRn2NR395Hp2nXV9MrNFbQvM4QZYhQScZ78Ue2kHs0Zn9lN/z6x/ktH9lN/wA+sf5LWvBMtxbxzICFkUOAeuCM1JR7aQezRif2U3/PrH+S0f2U3/PrH+S1t0Ue2kHs0Yn9lN/z6x/ktH9lN/z6x/ktbdRz3EFrCZriaOGIEAvIwUAkgAZPqSB+NHtpB7NGR/ZTf8+sf5LR/ZTf8+sf5LWu9xBHcR27zRrNKGaOMsAzhcZIHU4yM/UVJR7aQezRif2U3/PrH+S0f2U3/PrH+S1t0Ue2kHs0Yn9lN/z6x/ktH9lN/wA+sf5LW0rBhkHIpaPbSD2aMT+ym/59Y/yWmvp3ljLWyY9lBrdoo9sw9mjnfs8P/PGP/vkUfZ4f+eMf/fIq9fRCOYFRgMM496q10RaauZNWdiP7PD/zxj/75FH2eH/njH/3yKdJIkUbSSOqIgLMzHAAHUk0RyJ
... (truncated, see commit diff for full changes)

Generated by vscuse-auto-fix.yml | model: gpt-5.6-sol

…2254` so the Welcome-dialog close step waits up to 60 seconds for its preconditions instead of force-clicking while VS Code is still loading.
…2254` to wait for the exact VS Code Welcome sign-in overlay before force-running its close click, preventing the startup race where the click ran before the dialog appeared.
…_012254` from the unreliable Welcome-dialog X to the captured “Continue without Signing In” action at (850, 634), removing stale target preconditions.
…e click in shared startup group `plan_r_0928_012254` with an Esc key press so the modal closes before the M365 sidebar step.
…l assertion to shared startup group `plan_r_0928_012254`, ensuring the VS Code Welcome dialog is closed before opening the M365 panel.
@github-actions

github-actions Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

❌ VscUse Test Plan — Tests failure

Why these tests: Selected 61 impacted test plan(s) changed in this PR (no product code modified).

Branch diff: auto-fix/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys/2026-09-03 → dev

Results: ✅ 38 passed · ❌ 23 failed (of 61 plans)

Plans run:

  • DA_No_Action_Web_Search
  • Feature_Arm_Deoploy_Support_Json_Format_And_Multiple_Templates
  • Feature_Bot_Collaboration_Remote_Debug
  • Feature_DA_Add_MCP_Server
  • Feature_DA_Manifest_File_Function_Env_Support
  • Feature_DA_Manifest_File_Function_Manifest_Plugin_Support
  • Feature_DA_Manifest_File_Function_Path_Support
  • Feature_DA_No_Action_Add_PKCE_OAuth_Auth_Configurations
  • Feature_Del_Resource_Group_And_Reprovision_For_Bot
  • Feature_Del_Resource_Group_And_Reprovision_For_Tab
  • Feature_Deploy_Without_Provision
  • Feature_LocalDebug_AI_Search_without_AzureOpenAI_Keys
  • Feature_LocalDebug_AI_Search_without_OpenAI_Keys
  • Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys
  • Feature_LocalDebug_Custom_API_without_OpenAI_Keys
  • Feature_LocalDebug_Port_in_Use
  • Feature_LocalDebug_Second_Press_F5_for_Bot
  • Feature_LocalDebug_without_M365Login
  • Feature_Modify_Playground_MockData
  • Feature_Open_DeveloperPortal_Publish
  • Feature_Prompt_Use_Run_From_Package
  • Feature_Sign_In_No_Subscription
  • Feature_Sign_Out
  • Feature_Simple_Bot_ts_Local_Debug_With_Different_Account
  • Feature_Validate_Plugin_Manifest
  • Feature_Validate_copilot_manifest
  • Feature__DA_No_Action_EK_Fail_With_Oversize_And_Incorrect_Format
  • Feature__Share_Basic_DA
  • Feature__Test_Bot_AAD
  • Regular_AppYmlFile_Invalid_Syntax
  • Regular_AppYmlFile_Remove_DeployOrPublish_Section
  • Regular_AppYmlFile_Unknown_Actions_Report_Error
  • Regular_Basic_Tab_Instant_Tab_Local
  • Regular_Basic_Tab_Instant_Tab_Remote
  • Regular_Check_DA_New_Env_AgentScope
  • Regular_Command_Palette_Commands_Display
  • Regular_DA_Add_Action_And_Capability_Remote_Debug
  • Regular_Deploy_Triggered_Codelens_Support
  • Regular_LocalDebug_Remove_App_then_Rerun
  • Regular_LocalDebug_bot_with_existing_AAD
  • Regular_Message_Extension_With_Action_Command_ts_Playground_Debug
  • Regular_Newproject_In_Exist_Project
  • Regular_Provision_Triggered_Codelens_Support
  • Regular_Publish_to_Teams_Build_Package_If_Not_Provision
  • Regular_Publish_to_Teams_from_Codelens
  • Regular_Remove_App_From_Devportal_And_Reprovision
  • Regular_Report_Issue
  • Regular_Reusing_Simple_Bot
  • Regular_Set_Sub_Id_And_Rg_Name
  • Regular_Simple_Bot_Remote_Template_Verification
  • Regular_Simple_Bot_Single_Progress_Bar_ts_Remote_Debug
  • Regular_Simple_Bot_With_Symbol_Link_Target_js_playground
  • Regular_Tab_Collaboration_Remote_Debug
  • Regular_Update_App
  • Regular_Validate_Manifest
  • Regular_Verify_NLS_Cache_Cleared_After_Metadata_Fetch
  • Regular_Verify_Scaffold_Works_For_Dynamically_Added_Template
  • Regular_Verify_Wizard_UI_Loads_All_Project_Types_From_Wizard_Node
  • Regular__Debug_All_Telemetry_Check_And_Terminated_Previous_Tasks
  • Regular_intelli_sense_invalid_yaml_use_DA_None
  • Template_And_Sample_Readme_Verification

Step Status
1️⃣ Build VSIX (CD) ✅ Done
2️⃣ Build Docker image ✅ Done
3️⃣ Run UI tests ❌ Tests failure (38/61 passed)

🎯 Actual UI test run
🔗 Full pipeline results
📊 Detailed test report

ℹ️ How were these tests selected?

GitHub Copilot (GPT-5.6-sol, high reasoning) analysed the PR title, description, and the diff between
auto-fix/Feature_LocalDebug_Custom_API_without_AzureOpenAI_Keys/2026-09-03 and dev
to pick the most relevant test plans from packages/tests/vscuse/Index.md.

@github-actions

github-actions Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Selection — AI Selected

Why these tests: All changed files are under packages/tests/vscuse/**, which are handled by the separate vscuse pipeline and require no e2e tests.

Cases selected (0):


View pipeline run

Need to run more tests?

Comment on this PR:

  • /e2e-run ./path/to/test.tests.ts — add specific cases to AI selection
  • /e2e-run-all — run all e2e cases

Then re-run the workflow.

@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.26%. Comparing base (9bfa94a) to head (1d5af86).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##              dev   #16691   +/-   ##
=======================================
  Coverage   88.26%   88.26%           
=======================================
  Files         659      659           
  Lines       35786    35786           
  Branches     8618     8618           
=======================================
  Hits        31587    31587           
  Misses       2316     2316           
  Partials     1883     1883           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch was previously deployed

1 inactive deployment
engineering — 1d5af86b Deployed Sep 3, 2026 by teamstoolkitworkflowapp[bot] via run / report #1315
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant