Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1892390: Move pandas version tests to jenkins #3158

Merged
merged 5 commits into from
Mar 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 0 additions & 65 deletions .github/workflows/daily_modin_precommit_py39_py310.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,71 +203,6 @@ jobs:
.tox/.coverage
.tox/coverage.xml

test-pandas-patch-versions:
name: Test Snowpark pandas with pandas ${{ matrix.pandas-version }}
needs: build
runs-on: ${{ matrix.os.image_name }}
strategy:
fail-fast: false
matrix:
os:
- image_name: ubuntu-latest-64-cores
download_name: linux
pandas-version: ["2.2.1", "2.2.2"]
python-version: ["3.9"]
cloud-provider: [aws]
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: Decrypt parameters.py
shell: bash
run: .github/scripts/decrypt_parameters.sh
env:
PARAMETER_PASSWORD: ${{ secrets.PARAMETER_PASSWORD }}
CLOUD_PROVIDER: ${{ matrix.cloud-provider }}
- name: Install protoc
shell: bash
run: .github/scripts/install_protoc.sh
- name: Download wheel(s)
uses: actions/download-artifact@v4
with:
name: wheel
path: dist
- name: Show wheels downloaded
run: ls -lh dist
shell: bash
- name: Upgrade setuptools, pip and wheel
run: python -m pip install -U setuptools pip wheel
- name: Install tox
run: python -m pip install tox
- if: ${{ contains('macos', matrix.os.download_name) }}
name: Run Snowpark pandas API doctests
run: python -m tox -e "modin_pandas_version-py${PYTHON_VERSION}-doctest-snowparkpandasdoctest-modin-ci"
env:
MODIN_PANDAS_PATCH_VERSION: ${{ matrix.pandas-version }}
PYTHON_VERSION: ${{ matrix.python-version }}
cloud_provider: ${{ matrix.cloud-provider }}
PYTEST_ADDOPTS: --color=yes --tb=short
TOX_PARALLEL_NO_SPINNER: 1
# Specify SNOWFLAKE_IS_PYTHON_RUNTIME_TEST: 1 when adding >= python3.11 with no server-side support
# For example, see https://github.com/snowflakedb/snowpark-python/pull/681
shell: bash
- name: Run Snowpark pandas API tests (excluding doctests)
run: python -m tox -e "modin_pandas_version-py${PYTHON_VERSION/\./}-snowparkpandasnotdoctest-modin-ci"
env:
MODIN_PANDAS_PATCH_VERSION: ${{ matrix.pandas-version }}
PYTHON_VERSION: ${{ matrix.python-version }}
cloud_provider: ${{ matrix.cloud-provider }}
PYTEST_ADDOPTS: --color=yes --tb=short
TOX_PARALLEL_NO_SPINNER: 1
shell: bash

test-disable-sql-simplifier: # Will be removed after sql simplifier is stable and no option to opt out.
name: Test Disable SQL Simplifier modin-${{ matrix.os.download_name }}-${{ matrix.python-version }}-${{ matrix.cloud-provider }}
needs: build
Expand Down
13 changes: 9 additions & 4 deletions scripts/jenkins_regress_snowpandas.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
#
# this script is called in daily jenkins job SnowparkPythonSnowPandasDailyRegressRunner, it runs all snowpandas
# tests under tests/integ/modin and tests/unit/modin
# it is also called in SnowparkPandasPandasVersionsRunner, which runs the tests with the specified version
# of native pandas installed

set -euxo pipefail

Expand All @@ -14,8 +16,11 @@ gpg --quiet --batch --yes --decrypt --passphrase="$GPG_KEY" --output "tests/para
# Install tox, which is by default not present in the environment.
python -m pip install tox

# Install protoc
pip install protoc-wheel-0==21.1 mypy-protobuf

# Run snowpandas tests
python -m tox -c $WORKING_DIR -e snowparkpandasjenkins-modin
if [[ -z "${MODIN_PANDAS_PATCH_VERSION+x}" ]]; then
TOX_ENV="snowparkpandasjenkins-modin"
else
# If $MODIN_PANDAS_VERSION env variable is set, tell tox to install it
TOX_ENV="modin_pandas_version-snowparkpandasjenkins-modin"
fi
python -m tox -c $WORKING_DIR -e $TOX_ENV
Loading