List venvs #308

juancarlospaco · 2018-04-07T02:59:49Z

Add fades.helpers.list_venvs(index_path, logger=None) function.
Add --list-venvs to CLI args.
FIX Show venv info #276
FIX Help does not indicate how to obtain a UUID to remove it #306
FIX Have a --list-envs #216
All Tests pass OK.

Example:

juan@x ~/code/fades list-venvs|✓ $ bin/fades --list-venvs
*** fades ***  2018-04-06 23:38:49,233  INFO     
VENV_UUID:      9dbbe58b-94ef-40a7-8a35-098a04cf4dfd
DATE_TIME:      2017-02-21T00:54:00-03:00
FULL_PATH:      /home/juan/.local/share/fades/9dbbe58b-94ef-40a7-8a35-098a04cf4dfd
PACKAGES:       {'pypi': {'microraptor': '1.9.5'}}
INTERPRETER:    /usr/bin/python3.6
OPTIONS:        {'pyvenv_options': [], 'virtualenv_options': []}

*** fades ***  2018-04-06 23:38:49,233  INFO     
VENV_UUID:      9fc04480-b2f2-43da-a7ed-6afc32321522
DATE_TIME:      2017-04-27T14:48:40-03:00
FULL_PATH:      /home/juan/.local/share/fades/9fc04480-b2f2-43da-a7ed-6afc32321522
PACKAGES:       {'pypi': {'requests': '2.13.0'}}
INTERPRETER:    /usr/bin/python3.6
OPTIONS:        {'pyvenv_options': [], 'virtualenv_options': []}

*** fades ***  2018-04-06 23:38:49,234  INFO
VENV_UUID:      9affefcf-1a7b-4545-8bf5-52739ee90299
DATE_TIME:      2018-04-06T21:55:55-03:00
FULL_PATH:      /home/juan/.local/share/fades/9affefcf-1a7b-4545-8bf5-52739ee90299
PACKAGES:       {'pypi': {'ipython': '6.3.1', 'fades': '7.0'}}
INTERPRETER:    /usr/bin/python3.6
OPTIONS:        {'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']}

juan@x ~/code/fades list-venvs|✓ $

(The logger=None allows more clean output without the line *** fades *** 2018-04-06 23:38:49,233 INFO)
🐍💻🐱👍

…, FIX #306 , FIX ##216

coveralls · 2018-04-07T03:04:54Z

Coverage decreased (-1.04%) to 73.899% when pulling c2c548d on juancarlospaco:list-venvs into 56e6f8e on PyAr:master.

facundobatista · 2018-04-12T23:01:53Z

Hello! Thanks for this work!!

I'll do a general analysis on the example you shown above, and after you consider this feedback, a proper review.

So, some items (numbered, in case you want to refer them, feel free to discuss this here or in the Telegram group):

This work addresses Have a --list-envs #216; certainly not Show venv info #276 which is for a specific venv according to the dependencies, and surely not Help does not indicate how to obtain a UUID to remove it #306 which may be a duplicate or very closely related to Show venv info #276.
This new option (--list-venvs) needs an entry and example in the README, and proper documentation in the man page
Tests needed (no, it's not enough to not break older tests for a new parameter)
Don't use logging for this; the user is requiring some information, so is ok to use stdout in this case (see for example what we do for -V option)
I would do a better layout of the results; specifically:

separate each block using an empty line and the virtualenv ID, with the rest of the information indented
the timestamp should be more readable; it's ok to keep the timezone (maybe improved) but let's just get rid of the intermediate "T"
reorder the rest of the options according to it's "importance" (?)
use more human friendly names and case for the options

here is a sample layout that I think is better, using the same content you provided above:

juan@x ~/code/fades list-venvs|✓ $ bin/fades --list-venvs
Virtualenv id: 9dbbe58b-94ef-40a7-8a35-098a04cf4dfd
    path:          /home/juan/.local/share/fades/9dbbe58b-94ef-40a7-8a35-098a04cf4dfd
    dependencies:  {'pypi': {'microraptor': '1.9.5'}}
    timestamp:     2017-02-21T00:54:00-03:00
    interpreter:   /usr/bin/python3.6
    options:       {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv id: 9fc04480-b2f2-43da-a7ed-6afc32321522
    path:          /home/juan/.local/share/fades/9fc04480-b2f2-43da-a7ed-6afc32321522
    dependencies:  {'pypi': {'requests': '2.13.0'}}
    timestamp:     2017-04-27T14:48:40-03:00
    interpreter:   /usr/bin/python3.6
    options:       {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv id: 9affefcf-1a7b-4545-8bf5-52739ee90299
    path:          /home/juan/.local/share/fades/9affefcf-1a7b-4545-8bf5-52739ee90299
    dependencies:  {'pypi': {'ipython': '6.3.1', 'fades': '7.0'}}
    timestamp:     2018-04-06T21:55:55-03:00
    interpreter:   /usr/bin/python3.6
    options:       {'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']}

juan@x ~/code/fades list-venvs|✓ $

juancarlospaco · 2018-04-14T02:36:05Z

Some tests failing because of #309 (comment)

juancarlospaco · 2018-04-22T15:53:17Z

Now #309 seems Fixed. Is this Ok?. 🤔

gilgamezh · 2018-04-22T15:57:19Z

fades/helpers.py

+                    pac=v_dct_get("installed"),
+                    pyv=v_dct_get("interpreter"),
+                    opt=v_dct_get("options"),
+                    dat=datetime.fromtimestamp(v_dct_get("timestamp")).replace(


You can use {:%Y-%m-%d %H:%m} (or similar) in tmplt and here you only need to use fromtimestamp

gilgamezh · 2018-04-22T16:00:17Z

as fades is a CLI maybe we have to provide a way to "grep" the output.

fades --list-venvs | grep my_dep <--- all venvs with my_dep installed ?

Or maybe is better to add a filter ?

juancarlospaco · 2018-04-22T16:12:30Z

Not sure I follow, output IS grep-able. 🤔

You want an Option for it to spit out the pretty printed JSON, then you can use any JSON specific tool to parse/highlight/filter/whatever ?.

facundobatista · 2018-04-22T16:40:36Z

Being able to grep is a nice thing... we could show them one per line, like...

id='9dbbe58b-94ef-40a7-8a35-098a04cf4dfd', path='/home/juan/.local/share/fades/9dbbe58b-94ef-40a7-8a35-098a04cf4dfd', dependencies={'pypi': {'microraptor': '1.9.5'}}, timestamp='2017-02-21T00:54:00-03:00', interpreter='/usr/bin/python3.6', options={'pyvenv_options': [], 'virtualenv_options': []},
id='9fc04480-b2f2-43da-a7ed-6afc32321522', path='/home/juan/.local/share/fades/9fc04480-b2f2-43da-a7ed-6afc32321522', dependencies={'pypi': {'requests': '2.13.0'}}, timestamp='2017-04-27T14:48:40-03:00', interpreter='/usr/bin/python3.6', options={'pyvenv_options': [], 'virtualenv_options': []},
id='9affefcf-1a7b-4545-8bf5-52739ee90299', path='/home/juan/.local/share/fades/9affefcf-1a7b-4545-8bf5-52739ee90299', dependencies={'pypi': {'ipython': '6.3.1', 'fades': '7.0'}}, timestamp='2018-04-06T21:55:55-03:00', interpreter='/usr/bin/python3.6', options={'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']},

However, that escalates really quickly to "horrible".

We could output a real json with all the info... but how would we consume it? if we're making that into a Python code anyway, probable it's easier to do ... ?

>>> import fades
>>> fades.list_venvs()
<some json>

However, having to process a full json only to see those venvs that have some dependency installed is not really nice. Is this a use case? I think it's a nice to have.

Note that we should have a different --show-venv or something to dump all the info "of the venv that matches the dependencies I'm passing" (which is what #276) is about.

juancarlospaco · 2018-04-22T16:48:45Z

Right now fades.list_venvs() returns a str, the same printed to std out.

juancarlospaco · 2018-04-24T20:25:56Z

This adds optional Filtering of the list of venvs!.

Filter by UUID:

juan@x ~/code/fades list-venvs|✓ $ bin/fades --list-venvs 9affefcf-1a7b-4545-8bf5-52739ee90299

Virtualenv UUID:  9affefcf-1a7b-4545-8bf5-52739ee90299
    timestamp:    2018-04-06 21:55:55
    full path:    /home/juan/.local/share/fades/9affefcf-1a7b-4545-8bf5-52739ee90299
    dependencies: {'pypi': {'ipython': '6.3.1', 'fades': '7.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']}

juan@x ~/code/fades list-venvs|✓ $

Filter by Dependency:

juan@x ~/code/fades list-venvs|✓ $ bin/fades --list-venvs ipython

Virtualenv UUID:  158c5d41-10b1-412b-8522-f5ad3caba62c
    timestamp:    2017-12-16 04:32:32
    full path:    /home/juan/.local/share/fades/158c5d41-10b1-412b-8522-f5ad3caba62c
    dependencies: {'pypi': {'ipython': '6.2.1', 'scrapy': '1.4.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']}

Virtualenv UUID:  9affefcf-1a7b-4545-8bf5-52739ee90299
    timestamp:    2018-04-06 21:55:55
    full path:    /home/juan/.local/share/fades/9affefcf-1a7b-4545-8bf5-52739ee90299
    dependencies: {'pypi': {'ipython': '6.3.1', 'fades': '7.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']}

juan@x ~/code/fades list-venvs|✓ $

Filter by Dependencies:

juan@x ~/code/fades list-venvs|✓ $ bin/fades --list-venvs ipython,pep257

Virtualenv UUID:  b07962da-5719-47a9-a8d0-e5ae29232e6e
    timestamp:    2017-06-22 22:55:51
    full path:    /home/juan/.local/share/fades/b07962da-5719-47a9-a8d0-e5ae29232e6e
    dependencies: {'pypi': {'flake8': '2.5.2', 'logassert': '2', 'mccabe': '0.4.0', 'nose': '1.3.7', 'pep8': '1.7.0', 'pyflakes': '1.0.0', 'pyxdg': '0.25', 'wheel': '0.26.0', 'rst2html5': '1.7.5', 'setuptools': '28.8.0', 'nosexcover': '1.0.10', 'pep257': '0.7.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv UUID:  158c5d41-10b1-412b-8522-f5ad3caba62c
    timestamp:    2017-12-16 04:32:32
    full path:    /home/juan/.local/share/fades/158c5d41-10b1-412b-8522-f5ad3caba62c
    dependencies: {'pypi': {'ipython': '6.2.1', 'scrapy': '1.4.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']}

Virtualenv UUID:  9affefcf-1a7b-4545-8bf5-52739ee90299
    timestamp:    2018-04-06 21:55:55
    full path:    /home/juan/.local/share/fades/9affefcf-1a7b-4545-8bf5-52739ee90299
    dependencies: {'pypi': {'ipython': '6.3.1', 'fades': '7.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']}

Virtualenv UUID:  5f11ebe8-46a7-45d1-ba01-6bf5011818a8
    timestamp:    2018-04-06 23:53:57
    full path:    /home/juan/.local/share/fades/5f11ebe8-46a7-45d1-ba01-6bf5011818a8
    dependencies: {'pypi': {'pycodestyle': '2.3.1', 'nose': '1.3.7', 'wheel': '0.26.0', 'rst2html5': '1.9.3', 'flake8': '3.5.0', 'pyflakes': '1.6.0', 'pyxdg': '0.25', 'pyuca': '1.1.2', 'nosexcover': '1.0.10', 'logassert': '2', 'pep257': '0.7.0', 'mccabe': '0.6.1', 'pep8': '1.7.0', 'setuptools': '28.8.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

juan@x ~/code/fades list-venvs|✓ $

Filter by non-existent:

juan@x ~/code/fades list-venvs|✓ $ bin/fades --list-venvs non_existent

juan@x ~/code/fades list-venvs|✓ $

Dont Filter:

juan@x ~/code/fades list-venvs|✓ $ bin/fades --list-venvs 

Virtualenv UUID:  9dbbe58b-94ef-40a7-8a35-098a04cf4dfd
    timestamp:    2017-02-21 00:54:00
    full path:    /home/juan/.local/share/fades/9dbbe58b-94ef-40a7-8a35-098a04cf4dfd
    dependencies: {'pypi': {'microraptor': '1.9.5'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv UUID:  6852052a-da4a-4b48-8542-d1c183f82cdd
    timestamp:    2017-04-26 20:02:30
    full path:    /home/juan/.local/share/fades/6852052a-da4a-4b48-8542-d1c183f82cdd
    dependencies: {'pypi': {'unicodemoticon': '3.0.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv UUID:  9fc04480-b2f2-43da-a7ed-6afc32321522
    timestamp:    2017-04-27 14:48:40
    full path:    /home/juan/.local/share/fades/9fc04480-b2f2-43da-a7ed-6afc32321522
    dependencies: {'pypi': {'requests': '2.13.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv UUID:  ce84c8de-ea4a-4b56-b992-0f284cb8bafd
    timestamp:    2017-06-22 22:09:21
    full path:    /home/juan/.local/share/fades/ce84c8de-ea4a-4b56-b992-0f284cb8bafd
    dependencies: {'pypi': {}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv UUID:  b07962da-5719-47a9-a8d0-e5ae29232e6e
    timestamp:    2017-06-22 22:55:51
    full path:    /home/juan/.local/share/fades/b07962da-5719-47a9-a8d0-e5ae29232e6e
    dependencies: {'pypi': {'flake8': '2.5.2', 'logassert': '2', 'mccabe': '0.4.0', 'nose': '1.3.7', 'pep8': '1.7.0', 'pyflakes': '1.0.0', 'pyxdg': '0.25', 'wheel': '0.26.0', 'rst2html5': '1.7.5', 'setuptools': '28.8.0', 'nosexcover': '1.0.10', 'pep257': '0.7.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv UUID:  95bf380a-b164-4c3c-8c50-b9edc3e18667
    timestamp:    2017-06-28 12:20:44
    full path:    /home/juan/.local/share/fades/95bf380a-b164-4c3c-8c50-b9edc3e18667
    dependencies: {'pypi': {'anglerfish': '2.5.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv UUID:  158c5d41-10b1-412b-8522-f5ad3caba62c
    timestamp:    2017-12-16 04:32:32
    full path:    /home/juan/.local/share/fades/158c5d41-10b1-412b-8522-f5ad3caba62c
    dependencies: {'pypi': {'ipython': '6.2.1', 'scrapy': '1.4.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']}

Virtualenv UUID:  9affefcf-1a7b-4545-8bf5-52739ee90299
    timestamp:    2018-04-06 21:55:55
    full path:    /home/juan/.local/share/fades/9affefcf-1a7b-4545-8bf5-52739ee90299
    dependencies: {'pypi': {'ipython': '6.3.1', 'fades': '7.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': ['--system-site-packages'], 'virtualenv_options': ['--system-site-packages']}

Virtualenv UUID:  5f11ebe8-46a7-45d1-ba01-6bf5011818a8
    timestamp:    2018-04-06 23:53:57
    full path:    /home/juan/.local/share/fades/5f11ebe8-46a7-45d1-ba01-6bf5011818a8
    dependencies: {'pypi': {'pycodestyle': '2.3.1', 'nose': '1.3.7', 'wheel': '0.26.0', 'rst2html5': '1.9.3', 'flake8': '3.5.0', 'pyflakes': '1.6.0', 'pyxdg': '0.25', 'pyuca': '1.1.2', 'nosexcover': '1.0.10', 'logassert': '2', 'pep257': '0.7.0', 'mccabe': '0.6.1', 'pep8': '1.7.0', 'setuptools': '28.8.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv UUID:  158988c8-abbe-4a35-ad71-d82221b0c065
    timestamp:    2018-04-24 15:24:45
    full path:    /home/juan/.local/share/fades/158988c8-abbe-4a35-ad71-d82221b0c065
    dependencies: {}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

juan@x ~/code/fades list-venvs|✓ $

🐈

facundobatista · 2018-04-24T20:34:28Z

I really don't like having to filter ourselves.

You would be adding a lot of code, and it will never be enough, like... hey, I now want to filter by interpreter!

The best filtering tool is grep, people should use grep, we need to make (at least some of) our output consumable by grep

juancarlospaco · 2018-04-24T20:36:43Z

Filter by Interpreter:

juan@x ~/code/fades list-venvs|✓ $ bin/fades --list-venvs /usr/bin/python3.6

Virtualenv UUID:  9dbbe58b-94ef-40a7-8a35-098a04cf4dfd
    timestamp:    2017-02-21 00:54:00
    full path:    /home/juan/.local/share/fades/9dbbe58b-94ef-40a7-8a35-098a04cf4dfd
    dependencies: {'pypi': {'microraptor': '1.9.5'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

Virtualenv UUID:  6852052a-da4a-4b48-8542-d1c183f82cdd
    timestamp:    2017-04-26 20:02:30
    full path:    /home/juan/.local/share/fades/6852052a-da4a-4b48-8542-d1c183f82cdd
    dependencies: {'pypi': {'unicodemoticon': '3.0.0'}}
    interpreter:  /usr/bin/python3.6
    options:      {'pyvenv_options': [], 'virtualenv_options': []}

juan@x ~/code/fades list-venvs|✓ $

juancarlospaco · 2018-04-24T20:40:46Z

The Template is now an arg of the function so anyone can customize helpers.LIST_VENVS_TEMPLATE or pass it as argument and it works, for grep or whatever tool.

I think the point of --list-venvs is to make it human friendly and smooth on the eye,
else you can just cat venvs.idx | grep foo... 🤣

facundobatista · 2018-05-04T22:25:36Z

Ok, we discussed with @gilgamezh about this PR. We have chosen the desired output.

There will not be any template, and no internal heuristic for filtering.

The user will have two options: --list-venvs and --list-venvs-raw. The former will output what I specified in this comment, the later will just show one record per line, json-formated (to be explicit: it's not a json of the whole DB, each line is a json of it's own, each record of the DB encoded to json and shown).

I think that with this info you can continue and prepare the PR for the next review, right? Please let me know if you have any doubt (or we can talk in fades' IRC channel or Telegram group.

Thanks!!!!

juancarlospaco added 3 commits April 6, 2018 23:34

Add list_venvs helper function, add --list-venvs to CLI args, FIX #276 …

f15a320

…, FIX #306 , FIX ##216

Add list_venvs helper function, add --list-venvs to CLI args, FIX #276 …

fcff4df

…, FIX #306 , FIX ##216

Add DocString to list_venvs

417d0c8

Juan Carlos added 12 commits April 13, 2018 08:28

Remove logger, use print [SKIP CI]

967f0d0

Add new option to readme [SKIP CI]

bca5e57

Add changes from feedback to list_venvs function

c84d745

Add list_venvs to man.

1c43235

Add basic test list_venvs, WIP.

9bc6f97

Improve basic test for list_venvs.

e5d6517

Improve basic test for list_venvs.

4857e85

Improve basic test for list_venvs.

f2f92af

Improve basic test for list_venvs.

0ebee49

Improve basic test for list_venvs.

f2cab00

Improve basic test for list_venvs.

37e1f0e

Improve basic test for list_venvs.

c0ad32d

gilgamezh reviewed Apr 22, 2018

View reviewed changes

juancarlospaco added 6 commits April 24, 2018 17:12

Add optional filter to list of venvs.

f93dce9

Add optional filter to list of venvs.

0c4377c

Add optional filter to list of venvs.

b27c10c

Add optional filter to list of venvs.

8026cf5

Add optional filter to list of venvs.

9a0a91c

Remove a print on main.

c2c548d

juancarlospaco closed this Nov 17, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

List venvs #308

List venvs #308

juancarlospaco commented Apr 7, 2018

coveralls commented Apr 7, 2018 •

edited

Loading

facundobatista commented Apr 12, 2018

juancarlospaco commented Apr 14, 2018

juancarlospaco commented Apr 22, 2018

gilgamezh Apr 22, 2018

gilgamezh commented Apr 22, 2018

juancarlospaco commented Apr 22, 2018 •

edited

Loading

facundobatista commented Apr 22, 2018

juancarlospaco commented Apr 22, 2018

juancarlospaco commented Apr 24, 2018 •

edited

Loading

facundobatista commented Apr 24, 2018

juancarlospaco commented Apr 24, 2018

juancarlospaco commented Apr 24, 2018 •

edited

Loading

facundobatista commented May 4, 2018

List venvs #308

List venvs #308

Conversation

juancarlospaco commented Apr 7, 2018

coveralls commented Apr 7, 2018 • edited Loading

facundobatista commented Apr 12, 2018

juancarlospaco commented Apr 14, 2018

juancarlospaco commented Apr 22, 2018

gilgamezh Apr 22, 2018

Choose a reason for hiding this comment

gilgamezh commented Apr 22, 2018

juancarlospaco commented Apr 22, 2018 • edited Loading

facundobatista commented Apr 22, 2018

juancarlospaco commented Apr 22, 2018

juancarlospaco commented Apr 24, 2018 • edited Loading

facundobatista commented Apr 24, 2018

juancarlospaco commented Apr 24, 2018

juancarlospaco commented Apr 24, 2018 • edited Loading

facundobatista commented May 4, 2018

coveralls commented Apr 7, 2018 •

edited

Loading

juancarlospaco commented Apr 22, 2018 •

edited

Loading

juancarlospaco commented Apr 24, 2018 •

edited

Loading

juancarlospaco commented Apr 24, 2018 •

edited

Loading