Merge pull request #296 from Grokzen/feature/redis-py-3.0

WIP: redis-py 3.0.x support

Author: Grokzen

.travis.yml

@@ -1,12 +1,12 @@
 sudo: false
 dist: xenial
 language: python
+cache: pip
 python:
   - "2.7"
   - "3.4"
   - "3.5"
   - "3.6"
-  - "3.7"
   - "nightly"
 services:
   - redis-server
@@ -15,9 +15,12 @@ install:
   - "if [[ $REDIS_VERSION == '3.2' ]]; then REDIS_VERSION=3.2 make redis-install; fi"
   - "if [[ $REDIS_VERSION == '4.0' ]]; then REDIS_VERSION=4.0 make redis-install; fi"
   - "if [[ $REDIS_VERSION == '5.0' ]]; then REDIS_VERSION=5.0 make redis-install; fi"
+  - "if [[ $TEST_PYCODESTYLE == '1' ]]; then pip install pycodestyle; fi"
   - pip install -r dev-requirements.txt
   - pip install -e .
   - "if [[ $HIREDIS == '1' ]]; then pip install hiredis; fi"
+  - "pip freeze | grep redis"
+  - "pip freeze"
 env:
   # Redis 3.0 & HIREDIS
   - HIREDIS=0 REDIS_VERSION=3.0
@@ -31,7 +34,7 @@ env:
   # Redis 5.0 & HIREDIS
   - HIREDIS=0 REDIS_VERSION=5.0
   - HIREDIS=1 REDIS_VERSION=5.0
-script: 
+script:
   - make start
   - coverage erase
   - coverage run --source rediscluster -p -m py.test
@@ -40,6 +43,21 @@ script:
 after_success:
   - coverage combine
   - coveralls
+  - "if [[ $TEST_PYCODESTYLE == '1' ]]; then pycodestyle --repeat --show-source --exclude=.venv,.tox,dist,docs,build,*.egg,redis_install .; fi"
 matrix:
   allow_failures:
     - python: "nightly"
+    - python: 2.7
+      env: TEST_PYCODESTYLE=1
+    - python: 3.6
+      env: TEST_PYCODESTYLE=1
+    # python 3.7 has to be specified manually in the matrix
+    # https://github.com/travis-ci/travis-ci/issues/9815
+    - python: 3.7
+      dist: xenial
+      sudo: true
+      env: TEST_HIREDIS=0
+    - python: 3.7
+      dist: xenial
+      sudo: true
+      env: TEST_HIREDIS=1

LICENSE

@@ -0,0 +1,23 @@
+Copyright (c) 2014-2019 Johan Andersson
+
+ Permission is hereby granted, free of charge, to any person
+ obtaining a copy of this software and associated documentation
+ files (the "Software"), to deal in the Software without
+ restriction, including without limitation the rights to use,
+ copy, modify, merge, publish, distribute, sublicense, and/or sell
+ copies of the Software, and to permit persons to whom the
+ Software is furnished to do so, subject to the following
+ conditions:
+
+ The above copyright notice and this permission notice shall be
+ included in all copies or substantial portions of the Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+ OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+ HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+ WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ OTHER DEALINGS IN THE SOFTWARE.
+ 

Makefile

@@ -216,7 +216,7 @@ ifndef REDIS_TRIB_RB
 endif
 
 ifndef REDIS_VERSION
-	REDIS_VERSION=4.0.10
+	REDIS_VERSION=5.0.5
 endif
 
 export REDIS_CLUSTER_NODE1_CONF
@@ -378,16 +378,16 @@ redis-install:
 
 benchmark:
 	@echo ""
-	@echo " -- Running Simple benchmark with StrictRedis lib and non cluster server --"
+	@echo " -- Running Simple benchmark with Redis lib and non cluster server --"
 	python benchmarks/simple.py --port 7007 --timeit --nocluster
 	@echo ""
-	@echo " -- Running Simple benchmark with StrictRedisCluster lib and cluster server --"
+	@echo " -- Running Simple benchmark with RedisCluster lib and cluster server --"
 	python benchmarks/simple.py --port 7001 --timeit
 	@echo ""
-	@echo " -- Running Simple benchmark with pipelines & StrictRedis lib and non cluster server --"
+	@echo " -- Running Simple benchmark with pipelines & Redis lib and non cluster server --"
 	python benchmarks/simple.py --port 7007 --timeit --pipeline --nocluster
 	@echo ""
-	@echo " -- Running Simple benchmark with StrictRedisCluster lib and cluster server"
+	@echo " -- Running Simple benchmark with RedisCluster lib and cluster server"
 	python benchmarks/simple.py --port 7001 --timeit --pipeline
 
 ptp:

README.md

@@ -4,8 +4,6 @@ This client provides a client for redis cluster that was added in redis 3.0.
 
 This project is a port of `redis-rb-cluster` by antirez, with alot of added functionality. The original source can be found at https://github.com/antirez/redis-rb-cluster
 
-Gitter chat room: [![Gitter](https://badges.gitter.im/Grokzen/redis-py-cluster.svg)](https://gitter.im/Grokzen/redis-py-cluster?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
-
 [![Build Status](https://travis-ci.org/Grokzen/redis-py-cluster.svg?branch=master)](https://travis-ci.org/Grokzen/redis-py-cluster) [![Coverage Status](https://coveralls.io/repos/Grokzen/redis-py-cluster/badge.png)](https://coveralls.io/r/Grokzen/redis-py-cluster) [![PyPI version](https://badge.fury.io/py/redis-py-cluster.svg)](http://badge.fury.io/py/redis-py-cluster)
 
 
@@ -30,19 +28,21 @@ Latest stable release from pypi
 $ pip install redis-py-cluster
 ```
 
+This major version of `redis-py-cluster` supports `redis-py>=3.0.0,<3.1.0`.
+
 
 
 ## Usage example
 
 Small sample script that shows how to get started with RedisCluster. It can also be found in [examples/basic.py](examples/basic.py)
 
 ```python
->>> from rediscluster import StrictRedisCluster
+>>> from rediscluster import RedisCluster
 
 >>> # Requires at least one node for cluster discovery. Multiple nodes is recommended.
 >>> startup_nodes = [{"host": "127.0.0.1", "port": "7000"}]
 
->>> rc = StrictRedisCluster(startup_nodes=startup_nodes, decode_responses=True)
+>>> rc = RedisCluster(startup_nodes=startup_nodes, decode_responses=True)
 
 >>> rc.set("foo", "bar")
 True
@@ -54,7 +54,7 @@ True
 
 ## License & Authors
 
-Copyright (c) 2013-2018 Johan Andersson
+Copyright (c) 2013-2019 Johan Andersson
 
 MIT (See docs/License.txt file)
 

benchmarks/simple.py

@@ -9,7 +9,7 @@
   -p <port>          Port on redis server [default: 7000]
   -n <request>       Request number [default: 100000]
   -c <concurrent>    Concurrent client number [default: 1]
-  --nocluster        If flag is set then StrictRedis will be used instead of cluster lib
+  --nocluster        If flag is set then Redis will be used instead of cluster lib
   --timeit           Run a mini benchmark to test performance
   --pipeline         Only usable with --timeit flag. Runs SET/GET inside pipelines.
   --resetlastkey     Reset __last__ key
@@ -78,12 +78,13 @@ def timeit_pipeline(rc, num):
 if __name__ == "__main__":
     args = docopt(__doc__, version="0.3.1")
     startup_nodes = [{"host": args['--host'], "port": args['-p']}]
+
     if not args["--nocluster"]:
-        from rediscluster import StrictRedisCluster
-        rc = StrictRedisCluster(startup_nodes=startup_nodes, max_connections=32, socket_timeout=0.1, decode_responses=True)
+        from rediscluster import RedisCluster
+        rc = RedisCluster(startup_nodes=startup_nodes, max_connections=32, socket_timeout=0.1, decode_responses=True)
     else:
-        from redis import StrictRedis
-        rc = StrictRedis(host=args["--host"], port=args["-p"], socket_timeout=0.1, decode_responses=True)
+        from redis import Redis
+        rc = Redis(host=args["--host"], port=args["-p"], socket_timeout=0.1, decode_responses=True)
     # create specified number processes
     processes = []
     single_request = int(args["-n"]) // int(args["-c"])
@@ -102,4 +103,4 @@ def timeit_pipeline(rc, num):
     for p in processes:
         p.join()
     t2 = time.time() - t1
-    print("Tested {0}k SET & GET (each 50%) operations took: {1} seconds... {2} operations per second".format(int(args["-n"]) / 1000, t2, int(args["-n"]) / t2 * 2))
+    print("Tested {0}k SET & GET (each 50%) operations took: {1} seconds... {2} operations per second".format(int(args["-n"]) / 1000, t2, int(args["-n"]) / t2 * 2))

dev-requirements.txt

@@ -9,3 +9,4 @@ tox
 python-coveralls
 ptpdb
 ptpython
+pysnooper

docs/License.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2014-2016 Johan Andersson
+Copyright (c) 2014-2019 Johan Andersson
 
  Permission is hereby granted, free of charge, to any person
  obtaining a copy of this software and associated documentation

docs/benchmarks.rst

@@ -8,12 +8,12 @@ These are a few benchmarks that are designed to test specific parts of the code
 Setup benchmarks
 ----------------
 
-Before running any benchmark you should install this lib in editable mode inside a virtualenv so it can import `StrictRedisCluster` lib.
+Before running any benchmark you should install this lib in editable mode inside a virtualenv so it can import `RedisCluster` lib.
 
 Install with
 
 .. code-block:: bash
-    
+
     pip install -e .
 
 You also need a few redis servers to test against. You must have one cluster with at least one node on port `7001` and you must also have a non-clustered server on port `7007`.
@@ -42,18 +42,18 @@ Example output and comparison of different runmodes
 
 .. code-block::
 
-     -- Running Simple benchmark with StrictRedis lib and non cluster server, 50 concurrent processes and total 50000*2 requests --
+     -- Running Simple benchmark with Redis lib and non cluster server, 50 concurrent processes and total 50000*2 requests --
     python benchmarks/simple.py --host 127.0.0.1 --timeit --nocluster -c 50 -n 50000
     50.0k SET/GET operations took: 2.45 seconds... 40799.93 operations per second
 
-     -- Running Simple benchmark with StrictRedisCluster lib and cluster server, 50 concurrent processes and total 50000*2 requests --
+     -- Running Simple benchmark with RedisCluster lib and cluster server, 50 concurrent processes and total 50000*2 requests --
     python benchmarks/simple.py --host 127.0.0.1 --timeit -c 50 -n 50000
     50.0k SET & GET (each 50%) operations took: 9.51 seconds... 31513.71 operations per second
 
-     -- Running Simple benchmark with pipelines & StrictRedis lib and non cluster server --
+     -- Running Simple benchmark with pipelines & Redis lib and non cluster server --
     python benchmarks/simple.py --host 127.0.0.1 --timeit --nocluster -c 50 -n 50000 --pipeline
     50.0k SET & GET (each 50%) operations took: 2.1728243827819824 seconds... 46023.047602201834 operations per second
 
-     -- Running Simple benchmark with StrictRedisCluster lib and cluster server
+     -- Running Simple benchmark with RedisCluster lib and cluster server
     python benchmarks/simple.py --host 127.0.0.1 --timeit -c 50 -n 50000 --pipeline
-    50.0k SET & GET (each 50%) operations took: 1.7181339263916016 seconds... 58202.68051514381 operations per second
+    50.0k SET & GET (each 50%) operations took: 1.7181339263916016 seconds... 58202.68051514381 operations per second

docs/commands.rst

@@ -1,9 +1,9 @@
 Implemented commands
 ====================
 
-This will describe all changes that StrictRedisCluster have done to make a command to work in a cluster environment.
+This will describe all changes that RedisCluster have done to make a command to work in a cluster environment.
 
-If a command is not listed here then the default implementation from `StrictRedis` in the `redis-py` library is used.
+If a command is not listed here then the default implementation from `Redis` in the `redis-py` library is used.
 
 
 
@@ -107,7 +107,7 @@ Either because they do not work, there is no working implementation or it is not
 Overridden methods
 ------------------
 
-The following methods is overridden from StrictRedis with a custom implementation.
+The following methods is overridden from Redis with a custom implementation.
 
 They can operate on keys that exists in different hashslots and require a client side implementation to work.
 

docs/index.rst

@@ -6,10 +6,11 @@
 Welcome to redis-py-cluster's documentation!
 ============================================
 
-This project is a port of `redis-rb-cluster` by antirez, with a lot of added functionality. The original source can be found at https://github.com/antirez/redis-rb-cluster.
+This project is a port of `redis-rb-cluster` by antirez, with a lot of added functionality.
 
+The original source can be found at https://github.com/antirez/redis-rb-cluster.
 
-The source code is `available on github`_.
+The source code for this project is `available on github`_.
 
 .. _available on github: http://github.com/grokzen/redis-py-cluster
 
@@ -32,62 +33,87 @@ or from source code
 
 
 
-Usage example
+Basic usage example
 -------------
 
-Small sample script that shows how to get started with RedisCluster. It can also be found in the file `exmaples/basic.py`
+Small sample script that shows how to get started with RedisCluster. It can also be found in the file `exmaples/basic.py`.
+
+Additional code examples of more advance functionality can be found in the `examples/` folder in the source code git repo.
 
 .. code-block:: python
 
-    >>> from rediscluster import StrictRedisCluster
+    >>> from rediscluster import RedisCluster
 
     >>> # Requires at least one node for cluster discovery. Multiple nodes is recommended.
     >>> startup_nodes = [{"host": "127.0.0.1", "port": "7000"}]
 
     >>> # Note: See note on Python 3 for decode_responses behaviour
-    >>> rc = StrictRedisCluster(startup_nodes=startup_nodes, decode_responses=True)
+    >>> rc = RedisCluster(startup_nodes=startup_nodes, decode_responses=True)
 
     >>> rc.set("foo", "bar")
     True
     >>> print(rc.get("foo"))
     'bar'
 
-
 .. note:: Python 3
 
-    Since Python 3 changed to Unicode strings from Python 2's ASCII, the return type of *most* commands will be binary strings, unless the class is instantiated with the option ``decode_responses=True``. In this case, the responses will be Python 3 strings (Unicode). For the init argument `decode_responses`, when set to False, redis-py-cluster will not attempt to decode the responses it receives. In Python 3, this means the responses will be of type `bytes`. In Python 2, they will be native strings (`str`). If `decode_responses` is set to True, for Python 3 responses will be `str`, for Python 2 they will be `unicode`.
+    Since Python 3 changed to Unicode strings from Python 2's ASCII, the return type of *most* commands will be binary strings,
+    unless the class is instantiated with the option ``decode_responses=True``.
+
+    In this case, the responses will be Python 3 strings (Unicode).
+
+    For the init argument `decode_responses`, when set to False, redis-py-cluster will not attempt to decode the responses it receives.
+
+    In Python 3, this means the responses will be of type `bytes`. In Python 2, they will be native strings (`str`).
+
+    If `decode_responses` is set to True, for Python 3 responses will be `str`, for Python 2 they will be `unicode`.
+
 
-Dependencies & supported python versions
-----------------------------------------
 
-- Python: redis >= `2.10.2`, <= `2.10.5` is required.
-  Older versions in the `2.10.x` series can work but using the latest one is allways recommended.
+Library Dependencies
+--------------------
+
+It is always recommended to use the latest version of the dependencies of this project.
+
+- Redis-py: 'redis>=3.0.0,<3.1.0' is required in this major version of this cluster lib.
 - Optional Python: hiredis >= `0.2.0`. Older versions might work but is not tested.
-- A working Redis cluster based on version >= `3.0.0` is required. Only `3.0.x` releases is supported.
+- A working Redis cluster based on version `>=3.0.0` is required.
 
 
 
 Supported python versions
 -------------------------
 
-- 2.7
+Python versions should follow the same supported python versions as specificed by the upstream package `redis-py`, based on what major version(s) that is specified.
+
+If this library supports more then one major version line of `redis-py`, then the supported python versions must include the set of supported python versions by all major version lines.
+
+- 2.7.x
 - 3.4.1+ (See note)
-- 3.5
-- 3.6
-- 3.7
+- 3.5.x
+- 3.6.x
+- 3.7.x
 
 .. note:: Python 3.4.0
 
-    A segfault was found when running `redis-py` in python `3.4.0` that was introduced into the codebase in python `3.4.0`. Because of this both `redis-py` and `redis-py-cluster` will not work when running with `3.4.0`. This lib has decided to block the lib from execution on `3.4.0` and you will get a exception when trying to import the code. The only solution is to use python `3.4.1` or some other higher minor version in the `3.4` series.
+    A segfault was found when running `redis-py` in python `3.4.0` that was introduced into the codebase in python itself in the version `3.4.0`.
+    
+    Because of this both `redis-py` and `redis-py-cluster` will not work when running with `3.4.0`.
+    
+    This lib has decided to block the lib from execution on `3.4.0` and you will get a exception when trying to import the code.
+    
+    The only solution is to use python `3.4.1` or some other higher minor version in the `3.4` series.
+
+    When python `3.8.0` is released and when it is added to as a supported pythoon version, python 3.4.x will be removed from supported versions and this hard block will be removed from the source code.
 
 
 
-Regarding duplicate pypi and python naming
-------------------------------------------
+Regarding duplicate package name on pypi
+----------------------------------------
 
 It has been found that the python module name that is used in this library (rediscluster) is already shared with a similar but older project.
 
-This lib will not change the naming of the module to something else to prevent collisions between the libs.
+This lib will `NOT` change the naming of the module to something else to prevent collisions between the libs.
 
 My reasoning for this is the following
 
@@ -116,7 +142,6 @@ The Usage Guide
    commands
    limitations-and-differences
    pipelines
-   threads
    pubsub
    readonly-mode
 

docs/license.rst

@@ -1,7 +1,7 @@
 Licensing
 ---------
 
-Copyright (c) 2013-2018 Johan Andersson
+Copyright (c) 2013-2019 Johan Andersson
 
 MIT (See docs/License.txt file)