freebsd couldn't convert tap to xml

[sam-github]

python not setup correctly, or something: No module named 'tap13'

Haven't had time to see how consistent it is.

20:01:24 Performing Post build task...
20:01:24 Match found for : : True
20:01:24 Logical operation result is TRUE
20:01:24 Running script  : #/bin/bash -x +e
20:01:24 
20:01:24 mkdir out/junit || true
20:01:24 tap2junit -i test.tap -o out/junit/test.xml
20:01:24 tap2junit -i cctest.tap -o out/junit/cctest.xml || true
20:01:24 [freebsd11-x64] $ /bin/sh -xe /tmp/jenkins28096504523017284.sh
20:01:24 + mkdir out/junit
20:01:24 mkdir: out/junit: File exists
20:01:24 + true
20:01:24 + tap2junit -i test.tap -o out/junit/test.xml
20:01:25 Traceback (most recent call last):
20:01:25   File "/usr/local/bin/tap2junit", line 11, in <module>
20:01:25     load_entry_point('tap2junit==0.1.4', 'console_scripts', 'tap2junit')()
20:01:25   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
20:01:25     return get_distribution(dist).load_entry_point(group, name)
20:01:25   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2843, in load_entry_point
20:01:25     return ep.load()
20:01:25   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2434, in load
20:01:25     return self.resolve()
20:01:25   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2440, in resolve
20:01:25     module = __import__(self.module_name, fromlist=['__name__'], level=0)
20:01:25   File "/usr/local/lib/python3.6/site-packages/tap2junit/__main__.py", line 5, in <module>
20:01:25     from tap13 import TAP13 as tap13
20:01:25 ModuleNotFoundError: No module named 'tap13'

• https://ci.nodejs.org/job/node-test-commit-freebsd/nodes=freebsd11-x64/26739

[sam-github]

@nodejs/platform-freebsd maybe?

[nomadlogic]

I am still trying to get up to speed on how build environments are setup, it looks like this error is a transient issue. Are python modules installed as part of each CI run, or is the host preconfigured to have python modules like this installed ahead of time via Ansible?

I see subsequent builds do not have an issue with executing tap2junit, for example https://ci.nodejs.org/job/node-test-commit-freebsd/nodes=freebsd11-x64/26740/console

(which seems to suffer from another issue).

[refack]

@nomadlogic that's probably because we have two freebsd11 workers - https://ci.nodejs.org/label/freebsd11-x64/
The failing code comes from https://ci.nodejs.org/computer/test-digitalocean-freebsd11-x64-1

The job you refer to was run on https://ci.nodejs.org/computer/test-digitalocean-freebsd11-x64-2/

But the error is strange since tap13.py is there:

refael@refaelux:~$ ssh test-digitalocean-freebsd11-x64-1
$ ls -la /usr/local/lib/python3.6/site-packages/tap2junit/
total 31
drwxr-xr-x   3 root  wheel     6 May 27 03:24 .
drwxr-xr-x  16 root  wheel    22 May 27 03:24 ..
-rw-r--r--   1 root  wheel     0 Oct 24  2016 __init__.py
-rw-r--r--   1 root  wheel  1373 Nov  4  2016 __main__.py
drwxr-xr-x   2 root  wheel     5 May 27 03:24 __pycache__
-rw-r--r--   1 root  wheel  4877 Oct 24  2016 tap13.py

And AFAICT ./ should always be part of PYTHONPATH 😕

[nomadlogic]

thanks for pointing that out @refack - it looks like i can reproduce this in a virtualenv on one of my FreeBSD systems:

(tap13) [pete@bells ~/tmp/tap13]$ bin/tap2junit  --help
Traceback (most recent call last):
  File "bin/tap2junit", line 6, in <module>
    from tap2junit.__main__ import main
  File "/usr/home/pete/tmp/tap13/lib/python3.6/site-packages/tap2junit/__main__.py", line 5, in <module>
    from tap13 import TAP13 as tap13
ModuleNotFoundError: No module named 'tap13'

i created a fresh virtualenv, executed pip install tap2junit then tried to run the tap2junit script. hopefully i'll have some cycles to take a closer look today and see whats going on.

[refack]

Yeah I think I figured the issue. Default python is 3.6, where tap2junit apparently is only compatible with 2.7.

$ cat /usr/local/bin/tap2junit
#!/usr/local/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'tap2junit==0.1.5','console_scripts','tap2junit'
__requires__ = 'tap2junit==0.1.5'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('tap2junit==0.1.5', 'console_scripts', 'tap2junit')()
    )

$ sudo python2 m pip install --upgrade pip pipenv git+https://github.com/refack/tap2junit.git

$ cat /usr/local/bin/tap2junit
#!/usr/local/bin/python2
# EASY-INSTALL-ENTRY-SCRIPT: 'tap2junit==0.1.5','console_scripts','tap2junit'
__requires__ = 'tap2junit==0.1.5'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('tap2junit==0.1.5', 'console_scripts', 'tap2junit')()
    )

[refack]

P.S.

[iojs@test-digitalocean-freebsd11-x64-1 ~/build/workspace/node-test-commit-freebsd/nodes/freebsd11-x64]$ tap2junit -i test.tap -o out/junit/test.xml
[iojs@test-digitalocean-freebsd11-x64-1 ~/build/workspace/node-test-commit-freebsd/nodes/freebsd11-x64]$ echo $?
0

[refack]

P.P.S. @nomadlogic if you still have the spare cycles and are comfortable with python, maybe you could figure out why this happens in 3.6?

[nomadlogic]

heh i just realized this works with py27 on FreeBSD as well. I'll take a stab at seeing how to get this working py3.6.

[rvagg]

see also #1832
why is this happening now on multiple OSs? what's changed or is it just that we haven't been failing on this and haven't been properly parsing tap output on these machines?

[gem-pete]

I am willing to bet the machines that this still works on has python-2.7 installed and it is invoking the python-2.7 interpreter to execute.

I would suggest running it through "2to3-3.6" or something similar to see what will be needed to update to get this working. the code looks like it hasn't been touched since 2016, so it's not surprising it's not compatible:
https://pypi.org/project/tap2junit/

[sam-github]

Related: #1847 (comment)

[Trott]

tap2junit 0.1.5 is compatible with Python 3 and would seem to fix other bugs. Refs: nodejs/node#30955 (comment)

Can we update to tap2junit 0.1.5?