Skip to content

pyproject.toml install, created pytest suite, lint consolidation #408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
name: CI

on:
push:
branches:
- main
- dev
pull_request:
workflow_dispatch:

env:
UV_CACHE_DIR: /tmp/.uv-cache
PROJECT_PATH: "src/memory_profiler"

jobs:
code-quality:
name: Check code quality
runs-on: ubuntu-latest

steps:
# https://github.com/actions/checkout
- name: ⤵️ Checkout repository
uses: actions/checkout@v4

# https://github.com/astral-sh/setup-uv
- name: 🏗 Install uv and Python
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
cache-local-path: ${{ env.UV_CACHE_DIR }}
python-version: "3.13"

- name: 🏗 Install the project
run: uv sync --locked --dev

- name: Run mypy
run: uv run --frozen mypy ${{ env.PROJECT_PATH }}/
- name: Pylint review
run: uv run --frozen pylint ${{ env.PROJECT_PATH }}/
- name: Ruff check
run: uv run --frozen ruff check ${{ env.PROJECT_PATH }}/

tests:
name: Run tests
runs-on: ubuntu-latest

strategy:
matrix:
python-version:
- "3.10"
- "3.11"
- "3.12"
- "3.13"

steps:
# https://github.com/actions/checkout
- name: ⤵️ Checkout repository
uses: actions/checkout@v4

# https://github.com/astral-sh/setup-uv
- name: 🏗 Install uv and Python ${{ matrix.python-version }}
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
cache-local-path: ${{ env.UV_CACHE_DIR }}
python-version: ${{ matrix.python-version }}

- name: 🏗 Install the project
run: uv sync --locked --dev

- name: Run pytest
run: uv run --frozen pytest tests/ --cov=./ --cov-report=xml --junitxml=pytest-report.xml

# https://github.com/actions/upload-artifact
- name: Upload test report
uses: actions/upload-artifact@v4
with:
name: pytest-report-${{ matrix.python-version }}
path: pytest-report.xml

# https://github.com/actions/upload-artifact
- name: Upload coverage results
uses: actions/upload-artifact@v4
with:
name: coverage-results-${{ matrix.python-version }}
path: coverage.xml

# # https://github.com/codecov/codecov-action
# - name: Upload coverage to Codecov
# uses: codecov/codecov-action@v5
# with:
# token: ${{ secrets.CODECOV_TOKEN }}
# fail_ci_if_error: true
88 changes: 0 additions & 88 deletions .github/workflows/lint_python.yml

This file was deleted.

31 changes: 31 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
# Pre-commit configuration
# For details, visit: https://pre-commit.com/hooks.html

ci:
autofix_prs: false
skip:
# These steps run in the CI workflow. Keep in sync.
- mypy
- ruff

repos:

# Local hooks for mypy and pylint
- repo: local
hooks:
- id: mypy
name: Run mypy
entry: scripts/run-mypy.sh
language: script
types: [python]
- id: ruff
name: Run ruff
entry: scripts/run-ruff.sh
language: script
types: [ python ]
# - id: lint
# name: Run pylint
# entry: scripts/run-pylint.sh
# language: script
# types: [ python ]
25 changes: 0 additions & 25 deletions Makefile

This file was deleted.

12 changes: 6 additions & 6 deletions examples/exxample_psutil_memory_full_info.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def subprocess(i):
return a[i,i]

results = joblib.Parallel(n_jobs=n_jobs)(
joblib.delayed(subprocess)(i)
joblib.delayed(subprocess)(i)
for i in range(n_jobs))

return results
Expand All @@ -60,7 +60,7 @@ def subprocess(i):
return aa[i,i]

results = joblib.Parallel(n_jobs=n_jobs)(
joblib.delayed(subprocess)(i)
joblib.delayed(subprocess)(i)
for i in range(n_jobs))

return results
Expand Down Expand Up @@ -91,7 +91,7 @@ def func():
# Creating data: 10000x10000 ... done (0.75 Gb). Starting processing: n_jobs=64 ... done (0:00:14.701243). RSS: 111362.79
# Creating data: 10000x10000 ... done (0.75 Gb). Starting processing: n_jobs=64 ... done (0:00:15.020202). USS: 56108.69
# Creating data: 10000x10000 ... done (0.75 Gb). Starting processing: n_jobs=64 ... done (0:00:15.072918). PSS: 54826.61

# Conclusion:
# * RSS is overestimating like crazy (I checked the actual memory usage using htop)

Expand All @@ -108,17 +108,17 @@ def subprocess(i):
aa = a.copy()
time.sleep(10)
return r

# r = a[1,1]
# # time.sleep(10)
# return r

pass

start = datetime.datetime.now()
print("Starting processing: n_jobs={n_jobs} ... ".format(n_jobs=n_jobs), end="")
results = joblib.Parallel(n_jobs=n_jobs)(
joblib.delayed(subprocess)(i)
joblib.delayed(subprocess)(i)
for i in range(n_jobs))
print("done ({}). ".format(datetime.datetime.now() - start), end="")

Expand Down
2 changes: 1 addition & 1 deletion examples/reporting_file.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ def my_func1():

if __name__ == '__main__':
my_func()
my_func1()
my_func1()
Loading
Loading