chore: update Python version #753

igalic · 2020-04-05T19:14:08Z

the default is now 3.7 on FreeBSD.

igalic · 2020-04-05T19:19:25Z

Travis error is unrelated:

./libioc/helpers_ioctl.py:44:23: T484 No overload variant of "ioctl" matches argument types "int", "Any", "bytes"
./libioc/helpers_ioctl.py:44:23: T400 note: Possible overload variants:
./libioc/helpers_ioctl.py:44:23: T400 note: def ioctl(__fd, Union[int, HasFileno], int, int = ..., bool = ...) -> int
./libioc/helpers_ioctl.py:44:23: T400 note: def ioctl(__fd, Union[int, HasFileno], int, Union[bytearray, memoryview, array[Any]], Literal[True] = ...) -> int
./libioc/helpers_ioctl.py:44:23: T400 note: <2 more non-matching overloads not shown>

just means newer mypy version is now detecting better… things?

make sure it comes in the correct version

what, even, are packages? and what are dependencies? huh??

igalic · 2020-04-07T14:07:05Z

Test run is failing, but i don't know why

env ZPOOL="ioc-test-`uname -r`" make test
pytest tests --zpool ioc-test-12.1-RELEASE
============================= test session starts ==============================
platform freebsd12 -- Python 3.7.7, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- /usr/local/bin/python3.7
cachedir: .pytest_cache
rootdir: /tmp/cirrus-ci-build, inifile: setup.cfg
plugins: cov-2.8.1, pep8-1.0.6, mock-3.0.0
collecting ... collected 63 items
tests/test_Backup.py::TestBackup::test_nullfs_basejail_can_be_exported_standalone ERROR [  1%]
==================================== ERRORS ====================================
_ ERROR at setup of TestBackup.test_nullfs_basejail_can_be_exported_standalone _
release = <libioc.Release.Release object at 0x803ad7ad0>
root_dataset = <libzfs.ZFSDataset name 'ioc-test-12.1-RELEASE/libioc-test' type 'FILESYSTEM'>
zfs = <libzfs.ZFS object at 0x803a67840>
    @pytest.fixture
    def local_release(
        release: 'libioc.Release.ReleaseGenerator',
        root_dataset: libzfs.ZFSDataset,
        zfs: libzfs.ZFS
    ) -> 'libioc.Release.ReleaseGenerator':
        """Mock a local release."""
        if not release.fetched:
>           release.fetch(fetch_updates=True, update=True)
tests/conftest.py:181: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/libioc/Release.py:1087: in fetch
    event_scope=event_scope
tests/libioc/Release.py:742: in fetch
    self._extract_assets()
tests/libioc/Release.py:969: in _extract_assets
    destination=self.root_dir,
tests/libioc/Release.py:370: in root_dir
    if self.root_dataset.mountpoint:
tests/libioc/LaunchableResource.py:131: in root_dataset
    self._require_dataset_mounted(root_dataset)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <libioc.Release.Release object at 0x803ad7ad0>
dataset = <libzfs.ZFSDataset name 'ioc-test-12.1-RELEASE/libioc-test/releases/12.1-RELEASE/root' type 'FILESYSTEM'>
    def _require_dataset_mounted(self, dataset: libzfs.ZFSDataset) -> None:
        if dataset.mountpoint is None:
            raise libioc.errors.DatasetNotMounted(
                dataset=dataset,
>               logger=self.logger
            )
E           libioc.errors.DatasetNotMounted
tests/libioc/LaunchableResource.py:118: DatasetNotMounted
---------------------------- Captured stdout setup -----------------------------
Log directory '/var/log/iocage' created
Cache miss: .cache/libioc/download.freebsd.org/ftp/releases/amd64/amd64/12.1-RELEASE/base.txz (http://download.FreeBSD.org/ftp/releases/amd64/amd64/12.1-RELEASE/base.txz)
http://download.FreeBSD.org/ftp/releases/amd64/amd64/12.1-RELEASE/base.txz saved to .cache/libioc/download.freebsd.org/ftp/releases/amd64/amd64/12.1-RELEASE/base.txz
Cache miss: .cache/libioc/download.freebsd.org/ftp/releases/amd64/amd64/12.1-RELEASE/lib32.txz (http://download.FreeBSD.org/ftp/releases/amd64/amd64/12.1-RELEASE/lib32.txz)
http://download.FreeBSD.org/ftp/releases/amd64/amd64/12.1-RELEASE/lib32.txz saved to .cache/libioc/download.freebsd.org/ftp/releases/amd64/amd64/12.1-RELEASE/lib32.txz
Cache miss: .cache/libioc/download.freebsd.org/ftp/releases/amd64/amd64/12.1-RELEASE/MANIFEST (http://download.FreeBSD.org/ftp/releases/amd64/amd64/12.1-RELEASE/MANIFEST)
http://download.FreeBSD.org/ftp/releases/amd64/amd64/12.1-RELEASE/MANIFEST saved to .cache/libioc/download.freebsd.org/ftp/releases/amd64/amd64/12.1-RELEASE/MANIFEST
Dataset 'ioc-test-12.1-RELEASE/libioc-test/releases/12.1-RELEASE/root' is not mounted
---------------------------- Captured stderr setup -----------------------------
127.0.0.1 - - [07/Apr/2020 13:46:14] "GET http://download.FreeBSD.org/ftp/releases/amd64/amd64/12.1-RELEASE/base.txz HTTP/1.1" 200 -
127.0.0.1 - - [07/Apr/2020 13:46:20] "GET http://download.FreeBSD.org/ftp/releases/amd64/amd64/12.1-RELEASE/lib32.txz HTTP/1.1" 200 -
127.0.0.1 - - [07/Apr/2020 13:46:21] "GET http://download.FreeBSD.org/ftp/releases/amd64/amd64/12.1-RELEASE/MANIFEST HTTP/1.1" 200 -
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
========================= 1 error in 89.78s (0:01:29) ==========================
/usr/local/lib/python3.7/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
*** Error code 1
Stop.
make: stopped in /tmp/cirrus-ci-build

yonk42 · 2020-04-12T14:52:31Z

A couple of days ago I upgraded my server to FreeBSD 12.1-RELEASE and used this branch to upgrade my years old libiocage to libioc + ioc. I used this branch to install and am now encountering the same issue as you do in your test run, namely a "Dataset xyz is not mounted". It occurs when creating a new jail and also when fetching a new release:

# ioc fetch -r 12.1-RELEASE
[-] [email protected]: ...
  [+] [email protected]: OK [0.112s]
  [+] [email protected]: OK [6.487s]
    [+] [email protected]: OK [2.291s]
  [-] [email protected]: FAILED [0.714s]
Dataset 'zroot/iocage/releases/12.1-RELEASE/root' is not mounted

Altough the dataset is actually mounted:

# mount | grep 'zroot/iocage/releases/12.1-RELEASE/root'
zroot/iocage/releases/12.1-RELEASE/root on /iocage/releases/12.1-RELEASE/root (zfs, local, noatime, nfsv4acls)

Digging through the codebase, I traced the issue back to the dataset.mountpoint call in _require_dataset_mounted (L115 in LaunchableResource.py), but I cannot reproduce libzfs returing None for the dataset's mountpoint.

I added a sleep before the mountpoint is checked and ran a script in a loop to check the mountpoint using both libzfs and libioc.ZFS.get_zfs(), but both reported the mountpoint correctly every time. I have no idea what the issue is here, but since I am interested in a fix I thought I'd shar my findings here. If there is anything I can do (testing, etc.) I'd gladly help out. I can reproduce the issue in @gronke's chore/update-default-python-version branch as well. I also tried using the master branch of libzfs but that did not help either.

Test script for reference:

#!/usr/bin/env python3.7
import libzfs
import libioc

dataset_name = "zroot/iocage/releases/12.1-RELEASE"

stdzfs = libzfs.ZFS()
ioczfs = libioc.ZFS.get_zfs()

def check(zfs):
    normal = zfs.get_dataset(dataset_name)
    root = zfs.get_dataset(dataset_name + "/root")
    print(f"normal mountpoint: {normal.mountpoint}")
    print(f"root mountpoint: {root.mountpoint}")

print("without libioc zfs:")
check(stdzfs)
print("with libioc zfs:")
check(ioczfs)

BTW, shouldn't you also change the python version for the install-travis target to `$pyver$ as well?

igalic · 2020-04-12T19:19:05Z

BTW, shouldn't you also change the python version for the install-travis target to `$pyver$ as well?

well, that's probably a good idea
(done)

urosgruber · 2020-04-13T08:21:51Z

I'm seeing the same problem with

Dataset 'zroot/iocage/releases/12.1-RELEASE/root' is not mounted

urosgruber · 2020-04-13T08:26:00Z

Also same problem creating new jails.

urosgruber · 2020-04-13T09:28:52Z

Not sure if it helps but I reverted py-libzfs from 20191113 to 20181220 and error seems to go away

gronke · 2020-04-13T10:15:20Z

.. on it.. @igalic want to join me in tmux?

igalic · 2020-04-13T15:07:08Z

Not sure if it helps but I reverted py-libzfs from 20191113 to 20181220 and error seems to go away

i should look at the diff between those two

gronke · 2020-04-20T11:03:14Z

The regression was introduced with ZFS parallel mounts in FreeBSD 12.1:
https://svnweb.freebsd.org/base?view=revision&revision=345578

gronke · 2020-04-20T11:07:30Z

Please note, that this PR is a duplicate and will not be merged. The Python-version will be updated in #751. This PR remains open, so that we don't forget to clean dependencies (such as in d074402).

urosgruber · 2020-04-20T11:33:11Z

@gronke so should I open a separate ZFS ticket?

igalic · 2020-04-20T11:55:56Z

Let's start collecting everyone's kernel and py-libzfs versions in #755

gronke · 2020-04-20T12:09:40Z

@gronke so should I open a separate ZFS ticket?

An issue exists, yes. Right now I'm further investigating the problem. Its deterministic across systems, built with or without the commit mentioned above. Before opening a ticket for FreeBSD, I'd like to have reduced my PoC to the minimal steps to reproduce without involving py-libzfs.

A hotfix to address this issue is to re-open ZFS handles for all operations involving mounts. At the same time it requires changes across the entire library and comes with a huge performance penalty. Rather than working around the issue, a custom C module tailored for ZFS operations of libioc seems more appropriate. We do know exactly what operations to perform on ZFS datasets, so this a good opportunity.

Be assured that there will be a hotfix for 12.1 as soon as I have fully understood the impact of the regression. Next step is a debugging session with @fabianfreyer on tmux. After that I should be able to provide a PoC that's worth a ticket on FreeBSD, reason about a short-term solution for libioc and maybe even find motivation to optimize ZFS workflows.

Makefile

gronke · 2020-04-21T16:34:32Z

Commits have been incorporated in #751.

gronke · 2020-04-21T16:35:54Z

The hotfix was to disable mnttab_cache in py-libzfs.ZFS(..., mnttab_cache=False).

chore: update Python version

bf663a7

the default is now 3.7 on FreeBSD.

igalic added 5 commits April 5, 2020 21:31

make sqlite3 version also depend on pyver

1f729ec

update FreeBSD release on cirrus ci as well!

a55e559

we don't need cython to build, but we do need setuptools!

d074402

make sure it comes in the correct version

setuptools dont come with wheel, install that too

51d3e78

what, even, are packages? and what are dependencies? huh??

document what the Make string replacement expression does

d014079

update python version in .travis and setup.cfg

8bafa7b

This was referenced Apr 14, 2020

add debian/ build for libucl vstakhov/libucl#169

Closed

Create release Tags truenas/py-libzfs#98

Open

yonk42 mentioned this pull request Apr 20, 2020

ZFS root dataset is not mounted #755

Open

gronke reviewed Apr 21, 2020

View reviewed changes

Makefile Show resolved Hide resolved

gronke closed this Apr 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: update Python version #753

chore: update Python version #753

igalic commented Apr 5, 2020

igalic commented Apr 5, 2020

igalic commented Apr 7, 2020

yonk42 commented Apr 12, 2020 •

edited

Loading

igalic commented Apr 12, 2020

urosgruber commented Apr 13, 2020

urosgruber commented Apr 13, 2020

urosgruber commented Apr 13, 2020

gronke commented Apr 13, 2020

igalic commented Apr 13, 2020

gronke commented Apr 20, 2020 •

edited

Loading

gronke commented Apr 20, 2020

urosgruber commented Apr 20, 2020

igalic commented Apr 20, 2020

gronke commented Apr 20, 2020

gronke commented Apr 21, 2020

gronke commented Apr 21, 2020

chore: update Python version #753

chore: update Python version #753

Conversation

igalic commented Apr 5, 2020

igalic commented Apr 5, 2020

igalic commented Apr 7, 2020

yonk42 commented Apr 12, 2020 • edited Loading

igalic commented Apr 12, 2020

urosgruber commented Apr 13, 2020

urosgruber commented Apr 13, 2020

urosgruber commented Apr 13, 2020

gronke commented Apr 13, 2020

igalic commented Apr 13, 2020

gronke commented Apr 20, 2020 • edited Loading

gronke commented Apr 20, 2020

urosgruber commented Apr 20, 2020

igalic commented Apr 20, 2020

gronke commented Apr 20, 2020

gronke commented Apr 21, 2020

gronke commented Apr 21, 2020

yonk42 commented Apr 12, 2020 •

edited

Loading

gronke commented Apr 20, 2020 •

edited

Loading