Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a minimal environment with core dependencies #830

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

s3alfisc
Copy link
Member

@s3alfisc s3alfisc commented Mar 8, 2025

Hi @elchorro , here is a first attempt at creating a minimal environment.

It allows users to access all estimation function and methods (feols, fepois, feglm etc) and a range of post-processing methods (tidy, summary, plotting via plt).

It drops joblib, great_tables, tqdm and tabulate from the main env and moves it into a secondary environment full which will be installable by typing pip install pyfixest[full].

In the new "core" env

  • tqdm is replaced by a dummy function
  • joblib only executed when installed
  • calling GT produces an ImportError
  • calling tabulate produces an ImportError

What still would have to be done:

  • We should add logic to the CI workflow to run tests in the "core environment" vs the base environment, to make sure things work well without the "optional" dependencies.
  • We have to update the readme.
  • We should decide if the "minimal" environment should be the "default" environment that gets installed by running pip install pyfixest. I assume that it would be more user friendly to install etable etc by default and require users with other specs to specifically install the "core" env? I would be curious about @dsliwka's and @juanitorduz's thoughts on this?
  • Should we make seaborn an optional dependency? If no, we could replace the seaborn dependency by a matplotlib dependecy (the only piece of code that relies on seaborn is the plotting method for randomization inference which should be an easy fix.
  • We could even make narwhals optional, in which case we'd have to thrown an error wherever it converts non pd.DataFrame's for us (happens in maybe 3 scripts).
  • Clean up some code duplicates (i.e. the try from tqdm import tqdm parts).

If you have some time to polish this PR, it would be much appreciated @elchorro ! =)

@s3alfisc s3alfisc linked an issue Mar 8, 2025 that may be closed by this pull request
@s3alfisc s3alfisc added the dependencies Pull requests that update a dependency file label Mar 8, 2025
Copy link

codecov bot commented Mar 8, 2025

Codecov Report

Attention: Patch coverage is 34.78261% with 30 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pyfixest/estimation/ritest.py 25.00% 15 Missing ⚠️
pyfixest/report/summarize.py 0.00% 8 Missing ⚠️
pyfixest/estimation/decomposition.py 53.84% 6 Missing ⚠️
pyfixest/estimation/feols_compressed_.py 80.00% 1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (8476d50) and HEAD (c086b45). Click for more details.

HEAD has 4 uploads less than BASE
Flag BASE (8476d50) HEAD (c086b45)
core-tests 2 0
tests-vs-r 2 1
tests-extended 1 0
Flag Coverage Δ
core-tests ?
tests-extended ?
tests-vs-r 48.00% <34.78%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pyfixest/estimation/feols_compressed_.py 76.57% <80.00%> (-7.06%) ⬇️
pyfixest/estimation/decomposition.py 24.11% <53.84%> (-74.01%) ⬇️
pyfixest/report/summarize.py 4.41% <0.00%> (-83.33%) ⬇️
pyfixest/estimation/ritest.py 20.66% <25.00%> (-45.30%) ⬇️

... and 29 files with indirect coverage changes

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@s3alfisc
Copy link
Member Author

s3alfisc commented Mar 9, 2025

I adjusted the pyproject toml so that the default env has a richer dependency than minimal. Users who run pip install pyfixest install gt, tabulate, etc by default; users who run pip install pyfixest[minimal] will get an env with only the core dependency set.

(pyfixest) PS C:\Users\alexa\Documents\pyfixest> pixi list -x                                               
Package       Version  Build               Size        Kind   Source                                        
formulaic     1.1.1                        371.3 KiB   pypi   formulaic-1.1.1-py3-none-any.whl              
great_tables  0.16.1                       6.4 MiB     pypi   great_tables-0.16.1-py3-none-any.whl
joblib        1.4.2                        886.2 KiB   pypi   joblib-1.4.2-py3-none-any.whl
narwhals      1.29.1                       1.5 MiB     pypi   narwhals-1.29.1-py3-none-any.whl
numba         0.61.0                       11.7 MiB    pypi   numba-0.61.0-cp312-cp312-win_amd64.whl        
numpy         2.1.3                        40.2 MiB    pypi   numpy-2.1.3-cp312-cp312-win_amd64.whl
pandas        2.2.3                        37.7 MiB    pypi   pandas-2.2.3-cp312-cp312-win_amd64.whl        
pyfixest      0.28.0                                   pypi    (editable)
python        3.12.8   h3f84c4b_1_cpython  15.1 MiB    conda  python
scipy         1.15.2                       115.1 MiB   pypi   scipy-1.15.2-cp312-cp312-win_amd64.whl        
seaborn       0.13.2                       1018.2 KiB  pypi   seaborn-0.13.2-py3-none-any.whl
tabulate      0.9.0                        128.4 KiB   pypi   tabulate-0.9.0-py3-none-any.whl
tqdm          4.67.1                       220.5 KiB   pypi   tqdm-4.67.1-py3-none-any.whl
(pyfixest) PS C:\Users\alexa\Documents\pyfixest> pixi shell --environment minimal
(pyfixest:minimal) PS C:\Users\alexa\Documents\pyfixest> pixi list -x
Environment: minimal
Package    Version  Build               Size        Kind   Source
formulaic  1.1.1                        371.3 KiB   pypi   formulaic-1.1.1-py3-none-any.whl
narwhals   1.29.1                       1.5 MiB     pypi   narwhals-1.29.1-py3-none-any.whl
numba      0.61.0                       11.7 MiB    pypi   numba-0.61.0-cp312-cp312-win_amd64.whl
numpy      2.1.3                        40.2 MiB    pypi   numpy-2.1.3-cp312-cp312-win_amd64.whl
pandas     2.2.3                        37.7 MiB    pypi   pandas-2.2.3-cp312-cp312-win_amd64.whl
pyfixest   0.28.0                                   pypi    (editable)
python     3.12.8   h3f84c4b_1_cpython  15.1 MiB    conda  python
scipy      1.15.2                       115.1 MiB   pypi   scipy-1.15.2-cp312-cp312-win_amd64.whl
seaborn    0.13.2                       1018.2 KiB  pypi   seaborn-0.13.2-py3-none-any.whl

I guess we should still check with the pixi team if this is "supported" behavior or a non-supported hack?

Comment on lines 9 to 16

try:
from tqdm import tqdm
except ImportError:

def tqdm(iterable, *args, **kwargs):
"Define a dummy tqdm function."
return iterable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is being repeated a lot ... why don't we let tqdm be a core dep? It seems very lightweight

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with joblib, actually - it has no other dependencies except for python . More or less the motivation of this PR is to make the package so lightweight that it can easily be installed in closed environments (with no access to pypi / conda). My prior would be that both joblib and tqdm should actually be installed on every system that runs Python, though I'm not 100% sure.

I am more or less gravitating towards having a default / core env that includes joblib, tqdm, GT etc that users install with pip install pyfixest and then a minimal but still functional env for those who can really only work with numpy, scipy, etc. Would be your suggestion to keep tqdm in the "minimal" dependency set @juanitorduz?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add minimal environment option
2 participants