[WIP][don't merge yet] Python 3.4 compatibility #59

blizzz · 2014-12-10T10:51:48Z

There is something left to do that I was not able to track down yet. I think i was close once, but missed the time to crunch on this :( A goal of course is to continue to support Python 2.7. The test against 2.7 run fine for me.

On 3.4 a test run reveals this:

# ./runtests.sh 
....Assertion error caught with data set  ['文件.txt', b'\xe4\xbd\xa0\xe5\xa5\xbd\xe4\xb8\x96\xe7\x95\x8c', '文件夹']
FAssertion error caught with data set  ['test.txt', 'Hello world!', 'subdir']
F..........FF.FFF.F...F....E/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=6, laddr=('10.0.3.87', 56189), raddr=('10.0.3.1', 80)>
  outcome.errors.clear()
/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/pyocclient_test1418208618/pyoctest.dat'>
  outcome.errors.clear()
E/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=6, laddr=('10.0.3.87', 56193), raddr=('10.0.3.1', 80)>
  outcome.errors.clear()
/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/pyocclient_test1418208619/pyoctest.dir//file2.dat'>
  outcome.errors.clear()
E/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=6, laddr=('10.0.3.87', 56196), raddr=('10.0.3.1', 80)>
  outcome.errors.clear()
/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/pyocclient_test1418208620/pyoctest.dat'>
  outcome.errors.clear()
.E/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=6, laddr=('10.0.3.87', 56202), raddr=('10.0.3.1', 80)>
  outcome.errors.clear()
/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/pyocclient_test1418208622/pyoctest.dat'>
  outcome.errors.clear()
......
======================================================================
ERROR: test_upload_big_file (__main__.TestFileAccess)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/unittest_data_provider/__init__.py", line 7, in repl
    fn(self, *i)
  File "owncloud/test/test.py", line 167, in test_upload_big_file
    self.assertTrue(self.client.put_file(self.test_root + file_name, temp_file))
  File "/root/pyocclient/owncloud/owncloud.py", line 326, in put_file
    **kwargs
  File "/root/pyocclient/owncloud/owncloud.py", line 440, in __put_file_chunked
    headers = headers
  File "/root/pyocclient/owncloud/owncloud.py", line 879, in __make_dav_request
    **kwargs
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 362, in send
    timeout=timeout
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 308, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.4/http/client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1123, in _send_request
    self.putheader(hdr, value)
  File "/usr/lib/python3.4/http/client.py", line 1069, in putheader
    value = b'\r\n\t'.join(values)
TypeError: sequence item 0: expected bytes, bytearray, or an object with the buffer interface, float found

======================================================================
ERROR: test_upload_directory (__main__.TestFileAccess)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "owncloud/test/test.py", line 191, in test_upload_directory
    self.assertTrue(self.client.put_directory(self.test_root + 'subdir', temp_dir))
  File "/root/pyocclient/owncloud/owncloud.py", line 377, in put_directory
    **kwargs
  File "/root/pyocclient/owncloud/owncloud.py", line 326, in put_file
    **kwargs
  File "/root/pyocclient/owncloud/owncloud.py", line 440, in __put_file_chunked
    headers = headers
  File "/root/pyocclient/owncloud/owncloud.py", line 879, in __make_dav_request
    **kwargs
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 362, in send
    timeout=timeout
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 308, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.4/http/client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1123, in _send_request
    self.putheader(hdr, value)
  File "/usr/lib/python3.4/http/client.py", line 1069, in putheader
    value = b'\r\n\t'.join(values)
TypeError: sequence item 0: expected bytes, bytearray, or an object with the buffer interface, float found

======================================================================
ERROR: test_upload_small_file (__main__.TestFileAccess)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/unittest_data_provider/__init__.py", line 7, in repl
    fn(self, *i)
  File "owncloud/test/test.py", line 143, in test_upload_small_file
    self.assertTrue(self.client.put_file(self.test_root + file_name, temp_file))
  File "/root/pyocclient/owncloud/owncloud.py", line 326, in put_file
    **kwargs
  File "/root/pyocclient/owncloud/owncloud.py", line 440, in __put_file_chunked
    headers = headers
  File "/root/pyocclient/owncloud/owncloud.py", line 879, in __make_dav_request
    **kwargs
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 362, in send
    timeout=timeout
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 308, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.4/http/client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1123, in _send_request
    self.putheader(hdr, value)
  File "/usr/lib/python3.4/http/client.py", line 1069, in putheader
    value = b'\r\n\t'.join(values)
TypeError: sequence item 0: expected bytes, bytearray, or an object with the buffer interface, float found

======================================================================
ERROR: test_upload_two_chunks (__main__.TestFileAccess)
Test chunked upload with two chunks
----------------------------------------------------------------------
Traceback (most recent call last):
  File "owncloud/test/test.py", line 154, in test_upload_two_chunks
    self.assertTrue(self.client.put_file(self.test_root + 'chunk_test.dat', temp_file))
  File "/root/pyocclient/owncloud/owncloud.py", line 326, in put_file
    **kwargs
  File "/root/pyocclient/owncloud/owncloud.py", line 440, in __put_file_chunked
    headers = headers
  File "/root/pyocclient/owncloud/owncloud.py", line 879, in __make_dav_request
    **kwargs
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 362, in send
    timeout=timeout
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 308, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.4/http/client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1123, in _send_request
    self.putheader(hdr, value)
  File "/usr/lib/python3.4/http/client.py", line 1069, in putheader
    value = b'\r\n\t'.join(values)
TypeError: sequence item 0: expected bytes, bytearray, or an object with the buffer interface, float found

======================================================================
FAIL: test_download_file (__main__.TestFileAccess)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/unittest_data_provider/__init__.py", line 7, in repl
    fn(self, *i)
  File "owncloud/test/test.py", line 213, in test_download_file
    self.assertEqual(s, content)
AssertionError: '你好世界' != b'\xe4\xbd\xa0\xe5\xa5\xbd\xe4\xb8\x96\xe7\x95\x8c'

======================================================================
FAIL: test_get_file_contents (__main__.TestFileAccess)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/unittest_data_provider/__init__.py", line 7, in repl
    fn(self, *i)
  File "owncloud/test/test.py", line 80, in test_get_file_contents
    self.assertEqual(self.client.get_file_contents(self.test_root + subdir + '/' + file_name), content)
AssertionError: b'Hello world!' != 'Hello world!'

======================================================================
FAIL: test_move_and_rename (__main__.TestFileAccess)
Test rename into subdir
----------------------------------------------------------------------
Traceback (most recent call last):
  File "owncloud/test/test.py", line 303, in test_move_and_rename
    'first file'
AssertionError: b'first file' != 'first file'

======================================================================
FAIL: test_move_rename_in_place (__main__.TestFileAccess)
Test rename in place
----------------------------------------------------------------------
Traceback (most recent call last):
  File "owncloud/test/test.py", line 276, in test_move_rename_in_place
    'to rename'
AssertionError: b'to rename' != 'to rename'

======================================================================
FAIL: test_move_to_dir (__main__.TestFileAccess)
Test move into directory
----------------------------------------------------------------------
Traceback (most recent call last):
  File "owncloud/test/test.py", line 331, in test_move_to_dir
    'z file'
AssertionError: b'z file' != 'z file'

======================================================================
FAIL: test_move_to_non_existing_dir (__main__.TestFileAccess)
Test error when moving to non existing dir
----------------------------------------------------------------------
Traceback (most recent call last):
  File "owncloud/test/test.py", line 430, in test_move_to_non_existing_dir
    'x'
AssertionError: b'x' != 'x'

======================================================================
FAIL: test_move_unicode (__main__.TestFileAccess)
Test move unicode to dir
----------------------------------------------------------------------
Traceback (most recent call last):
  File "owncloud/test/test.py", line 407, in test_move_unicode
    '2'
AssertionError: b'2' != '2'

======================================================================
FAIL: test_rename_unicode (__main__.TestFileAccess)
Test rename unicode
----------------------------------------------------------------------
Traceback (most recent call last):
  File "owncloud/test/test.py", line 381, in test_rename_unicode
    '1'
AssertionError: b'1' != '1'

======================================================================
FAIL: test_update_share_password (__main__.TestFileAccess)
Test updating a share parameters - password
----------------------------------------------------------------------
Traceback (most recent call last):
  File "owncloud/test/test.py", line 601, in test_update_share_password
    self.assertTrue('share_with_displayname' in share_info)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 43 tests in 76.586s

FAILED (failures=9, errors=4)

- has_key is not available anymore - urllib and urlparse compatibility - open files as binary - string type compatibility - division compatibility

PVince81 · 2014-12-10T20:21:00Z

The CI results is also strange, it says "ImportError: No module named six"

PVince81 · 2014-12-10T20:25:40Z

@blizzz if possible you could also try and adapt .travis.yml to have both a python2 + python3 target.
It might be possible by changing ./runtests.sh to call "python3" instead of "python", but I'm not sure whether the remote server supports it.

Or if you can make all tests pass on Python 2.7 on Travis we could merge this earlier (Python 3 is probably broken anyway at this stage on master) and then we can all continue with separate PRs ? (I could look into making Travis run Python 3 CI)

As you like. 😄

Edit: Python 2.4 => 2.7

PVince81 · 2014-12-10T20:43:11Z

Looking into making Travis run Python 3 tests here: #60

blizzz · 2014-12-14T19:27:10Z

The CI results is also strange, it says "ImportError: No module named six"

Could it be that we need to add it to install_requires in the setup.py?

blizzz · 2014-12-14T19:29:45Z

With regard to travis: i can have a look into it, but do not have a setup therefore, yet. Do you have a quick start guide?

PVince81 · 2014-12-15T09:59:02Z

@blizzz for Travis I already started, here is the status: #60 (comment)

PVince81 · 2015-03-11T13:49:55Z

Did this break Python 2 compat ?

PVince81 · 2015-03-11T13:50:14Z

I mean, did Python 2 stuff still worked properly with this PR ? (even if Python 3 might still be broken)

blizzz · 2015-03-16T10:48:52Z

IIRC Py2 was unharmed.

PVince81 · 2016-03-17T06:42:25Z

Possibly obsoleted by #146 ?

blizzz · 2016-03-18T12:08:59Z

Possibly obsoleted by #146 ?

Agreed.

blizzz added 3 commits November 27, 2014 21:38

relativ paths need to start with a dot

299065a

self.assertEquals is deprecated, without s is the latest fashion

e71b74f

Python 3 compatibility

b53f6dd

- has_key is not available anymore - urllib and urlparse compatibility - open files as binary - string type compatibility - division compatibility

PVince81 mentioned this pull request Mar 18, 2015

Update owncloud.py to python 3.4.3 #80

Closed

blizzz closed this Mar 18, 2016

blizzz deleted the py3-compat branch July 24, 2017 09:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP][don't merge yet] Python 3.4 compatibility #59

[WIP][don't merge yet] Python 3.4 compatibility #59

Uh oh!

blizzz commented Dec 10, 2014

Uh oh!

PVince81 commented Dec 10, 2014

Uh oh!

PVince81 commented Dec 10, 2014

Uh oh!

PVince81 commented Dec 10, 2014

Uh oh!

blizzz commented Dec 14, 2014

Uh oh!

blizzz commented Dec 14, 2014

Uh oh!

PVince81 commented Dec 15, 2014

Uh oh!

PVince81 commented Mar 11, 2015

Uh oh!

PVince81 commented Mar 11, 2015

Uh oh!

blizzz commented Mar 16, 2015

Uh oh!

PVince81 commented Mar 17, 2016

Uh oh!

blizzz commented Mar 18, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

[WIP][don't merge yet] Python 3.4 compatibility #59

[WIP][don't merge yet] Python 3.4 compatibility #59

Uh oh!

Conversation

blizzz commented Dec 10, 2014

Uh oh!

PVince81 commented Dec 10, 2014

Uh oh!

PVince81 commented Dec 10, 2014

Uh oh!

PVince81 commented Dec 10, 2014

Uh oh!

blizzz commented Dec 14, 2014

Uh oh!

blizzz commented Dec 14, 2014

Uh oh!

PVince81 commented Dec 15, 2014

Uh oh!

PVince81 commented Mar 11, 2015

Uh oh!

PVince81 commented Mar 11, 2015

Uh oh!

blizzz commented Mar 16, 2015

Uh oh!

PVince81 commented Mar 17, 2016

Uh oh!

blizzz commented Mar 18, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants