Skip to content

Conversation

@mollyxu
Copy link
Contributor

@mollyxu mollyxu commented Nov 18, 2025

Remove skipped tests from collection in internal environment

Prevent internal CI failures from skipped tests by excluding skipped tests from collection in internal environment.

TorchCodec tests were flaky / broken internal because:

  • Tests were using runtime pytest.skip() calls, so they were collected but then skipped during execution

Changes Made

1. Added @needs_ffmpeg_cli marker to tests that need ffmpeg-cli

  • test_ops.py::TestVideoDecoderOps::test_seek_mode_custom_frame_mappings
  • test_ops.py::TestVideoDecoderOps:: test_seek_mode_custom_frame_mappings_fails
  • test_encoders.py::TestAudioEncoder::test_against_cli
  • test_encoders.py::TestAudioEncoder::test_video_encoder_against_ffmpeg_cli
  • test_encoders.py::TestAudioEncoder::test_codec_parameter_utilized
  • test_encoders.py::TestAudioEncoder::test_codec_spec_vs_impl_equivalence
  • test_encoders.py::TestAudioEncoder::test_extra_options_utilized
  • test_decoders.py::TestVideoDecoder::test_custom_frame_mappings_json_and_bytes
  • test_decoders.py::TestVideoDecoder::test_custom_frame_mappings_init_fails

2. Converted runtime skips to collection-time skips

Replaced runtime pytest.skip() calls with pytest.param(..., marks=pytest.mark.skipif(...)) for:

  • FFmpeg version compatibility checks (e.g., FFmpeg 4 incompatibility with wav files)
  • Platform-specific codec availability (e.g., webm codec unavailable on Windows with certain FFmpeg versions)

3. Updated conftest.py to exclude tests internally

Modified pytest_collection_modifyitems hook to exclude tests from collection when running internally if they have:

  • @needs_ffmpeg_cli marker
  • @pytest.mark.skip() decorator
  • @pytest.mark.skipif() decorator

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 18, 2025
@meta-codesync
Copy link

meta-codesync bot commented Nov 20, 2025

@mollyxu has imported this pull request. If you are a Meta employee, you can view this in D87512192.

@mollyxu mollyxu marked this pull request as ready for review November 20, 2025 16:17
@mollyxu mollyxu marked this pull request as draft November 20, 2025 16:17
@mollyxu mollyxu closed this Nov 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant