-
Notifications
You must be signed in to change notification settings - Fork 222
pytestplugin: introduce @pytest.mark.lg_xfail_feature
#1751
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
pytestplugin: introduce @pytest.mark.lg_xfail_feature
#1751
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1751 +/- ##
========================================
- Coverage 45.4% 45.3% -0.1%
========================================
Files 172 172
Lines 13503 13516 +13
========================================
Hits 6131 6131
- Misses 7372 7385 +13
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good, some smaller suggestions.
…d error messages Explain the purpose of the marker, turn the bare Exception into a pytest.UsageError, improve and simplify the error message and skip reason. Signed-off-by: Bastian Krause <[email protected]>
Every test creates a bare env config with or without feature. A future commit will introduce even more tests that will do this. So move the env config creation to a factory fixture that takes the desired features as an argument. Signed-off-by: Bastian Krause <[email protected]>
Tests if the appropriate pytest.UsageErrors are raised for calls to @pytest.mark.lg_feature() with unexpected arguments. Signed-off-by: Bastian Krause <[email protected]>
Adding more headlines makes it easier to scan over these sections when searching for something. It also helps pointing people to these sections due to the HTML anchors available for headlines in the HTML version of the docs. Signed-off-by: Bastian Krause <[email protected]>
Currently, the pytest plugin section lacks some structure: - Command-Line Options - Environment Variables - LG_ENV - LG_PROXY - Simple Example - Custom Fixture Example - Strategy Fixture Example - Feature Flags - Test Reports It's a mixture of tutorial-like examples, basic explanations and features. Move the examples into their own section and make their headlines more catchy. Signed-off-by: Bastian Krause <[email protected]>
… docs The feature flags are not a pytest plugin feature. Instead they should be described in the configuration chapter in the environment config section. So move them there. The pytest plugin docs should, however, document @pytest.mark.lg_feature, so name the section accordingly and simplify the section, so the relevant parts are described there in a pointed fashion. Signed-off-by: Bastian Krause <[email protected]>
`@pytest.mark.lg_xfail_feature()` is meant to mark a test as "xfail" if the target or global env contains the specified feature flag. Signed-off-by: Bastian Krause <[email protected]>
cbc9aa3
to
edee111
Compare
Changes:
|
Description
@pytest.mark.lg_xfail_feature()
is meant to mark a test as "xfail" if the target or global environment contains the specified feature flag. There is no other sane way of testing for features in pytest's collection phase. Marking a test as xfail later is unsupported in pytest. So apytest.mark
decorator is necessary.Imagine two targets contain the
camera
feature flag. One of them has the additionalspecial-camera-2000
feature flag. The other has the additionalspecial-camera-3000
feature flag. Due to a known bug onspecial-camera-3000
, the test is expected to fail. The test can be marked asxfail
for that feature:While at it, improve error messages, skip reason and documentation of
@pytest.mark.lg_feature()
and feature flags. Especially the documentation changes make it easier to add documentation for the new@pytest.mark.lg_xfail_feature()
.Checklist