Skip to content

Snow-3859269 migrate regress test - #4304

Merged
sfc-gh-jzeng merged 2 commits into
mainfrom
SNOW-3859269-migrate-regress-test
Aug 6, 2026
Merged

Snow-3859269 migrate regress test#4304
sfc-gh-jzeng merged 2 commits into
mainfrom
SNOW-3859269-migrate-regress-test

Conversation

@sfc-gh-jzeng

Copy link
Copy Markdown
Collaborator

  1. Which Jira issue is this PR addressing?

Fixes SNOW-3859269

  1. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe.
    • If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support.

This PR only touches .github/workflows/, so it adds no tests, logging, telemetry, credentials, or dependencies.

  1. Please describe how your code solves the related issue.

Step one of SNOW-3859269: this only adds the GitHub Actions workflow. The Jenkins job SnowparkPythonSnowflakePythonClientRegressRunner is untouched and still running. Retiring it, and removing it from the release-validation fan-out in jenkins_utils, is a separate PR once this has run for a few nights.

That job runs the snowpark suite against the connector's unreleased main branch, so a connector change that breaks snowpark shows up before the connector ships. On Jenkins it connects to a preprod account whose parameters can differ from prod, so a red build was often an environment difference rather than a real regression, and its definition lived in a different repo from the code it tests.

The line to review closely: daily_connector_regress.yml runs tox -e notdoctest-pandascap-pyarrowcap, identical to what scripts/jenkins_regress.sh runs today. jenkins_regress.sh has no test list of its own and delegates everything to tox.ini, so that string is the whole guarantee that coverage matches the Jenkins job. Editing it changes coverage silently.

The workflow checks out the connector at main, builds a wheel, and sets snowflake_path to it, which is all scripts/tox_install_cmd.sh needs. Nothing under src/ or tests/ changes, and no new secrets are needed: it reuses decrypt_parameters.sh and PARAMETER_PASSWORD, which is also what lands the run on the same prod account as daily_precommit.yml. That account change is the actual fix for the drift.

It runs nightly at 06:00 UTC and on workflow_dispatch. trigger_daily_tests_on_branch.yml gains a fifth entry so release validation dispatches it alongside the four existing daily workflows.

The three assert steps exist because the bad outcome here is a green run that tested the released connector instead of main: the wheel has to carry a cp310 tag, contain the nanoarrow .so, and be the one the tests actually imported. Two things deliberately differ from the other daily workflows, both commented in the file: the env resolves to -n logical rather than -n 6, because a daily* env would change the env name and would also drop --reruns 5 (precommit.yml already runs this env on the same runner size), and no junit artifact is uploaded, matching the other workflows here which upload coverage instead.


@sfc-gh-jzeng
sfc-gh-jzeng requested a review from a team as a code owner August 4, 2026 23:05
@sfc-gh-jzeng sfc-gh-jzeng added the NO-CHANGELOG-UPDATES This pull request does not need to update CHANGELOG.md label Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@sfc-gh-jzeng sfc-gh-jzeng changed the title Snow 3859269 migrate regress test Snow-3859269 migrate regress test Aug 4, 2026
@sfc-gh-jzeng

Copy link
Copy Markdown
Collaborator Author

I have read the CLA Document and I hereby sign the CLA

@codecov-commenter

codecov-commenter commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.26%. Comparing base (e7d77e7) to head (03ed427).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4304   +/-   ##
=======================================
  Coverage   95.26%   95.26%           
=======================================
  Files         171      171           
  Lines       44720    44720           
  Branches     7676     7676           
=======================================
  Hits        42604    42604           
  Misses       1299     1299           
  Partials      817      817           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sfc-gh-yuwang

Copy link
Copy Markdown
Collaborator

I have two questions:

  1. is there a way to test the added action before merge?
  2. how long does this job usually take?

@sfc-gh-jzeng

Copy link
Copy Markdown
Collaborator Author

I have two questions:

  1. is there a way to test the added action before merge?
  2. how long does this job usually take?
  1. The action/workflow can only be triggered after the definition .yml is merged. However, after merged, we can manually trigger it to see how it goes, and then evaluate how it performs at its expected triggering time.
  2. Observed from the Jenkins job history, a typical one would take 1-2 hours. But in some cases, it would take over 10 hours with several false-positive failures. This is also the motivation for migrating from Jenkins to here.

@sfc-gh-jzeng
sfc-gh-jzeng force-pushed the SNOW-3859269-migrate-regress-test branch from 1a230ab to d88c4ee Compare August 5, 2026 21:35
@sfc-gh-jzeng
sfc-gh-jzeng marked this pull request as draft August 5, 2026 23:17
@sfc-gh-jzeng
sfc-gh-jzeng marked this pull request as ready for review August 5, 2026 23:18
@sfc-gh-jzeng
sfc-gh-jzeng marked this pull request as draft August 5, 2026 23:47
@sfc-gh-jzeng
sfc-gh-jzeng marked this pull request as ready for review August 5, 2026 23:47
@sfc-gh-jzeng
sfc-gh-jzeng force-pushed the SNOW-3859269-migrate-regress-test branch from d88c4ee to 03ed427 Compare August 6, 2026 07:07
@sfc-gh-jzeng
sfc-gh-jzeng marked this pull request as draft August 6, 2026 15:24
@sfc-gh-jzeng
sfc-gh-jzeng marked this pull request as ready for review August 6, 2026 15:25
@sfc-gh-jzeng
sfc-gh-jzeng marked this pull request as draft August 6, 2026 17:45
@sfc-gh-jzeng
sfc-gh-jzeng marked this pull request as ready for review August 6, 2026 17:45
@sfc-gh-jzeng
sfc-gh-jzeng enabled auto-merge (squash) August 6, 2026 18:19
@sfc-gh-jzeng
sfc-gh-jzeng merged commit 1b972ba into main Aug 6, 2026
27 of 30 checks passed
@sfc-gh-jzeng
sfc-gh-jzeng deleted the SNOW-3859269-migrate-regress-test branch August 6, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

NO-CHANGELOG-UPDATES This pull request does not need to update CHANGELOG.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants