Many simultaneous requests create unbounded connections

[grant-aterlo]

On the pool maxsize is the limit on kept connections, however unlimited connections will be spawned as needed if there are no connections to grab from the pool.

This can cause too many connections to be created for memcache. Is this behavior on maxsize intended? Does it make sense to have a hard limit parameter?

[fafhrd91]

this is bug. would you like to make PR?

[grant-aterlo]

I have a change that fixes the issue. Could I get permissions to create a pull request

[asvetlov]

Everyone have a permission for creating a Pull Request, isn't it?

After reviewing PR @fafhrd91 or I will merge your PR and publish new aiomcache release.

[argaen]

What behavior do we want here? If there are no connections available should we block until one is release? or add a hardcoded MAX and only start blocking until MAX is reached?

I would go for the first option because is more explicit and we don't hide anything to the developer. How is aioredis managing this? @popravich

[popravich]

aioredis has hard limit — no more then masize connections will be created in a single pool.
In aioredis we free connections, when all free connections are busy we spawn new one,
if maxsize limit is reached — we block.
AFAIR, aiopg and aiomysql act the same.

[argaen]

Then I would go with this approach to keep consistency with the other projects

[argaen]

@grant-aterlo do you have a code that reproduces this bug? There is a test case that is checking this error you report and its passing correctly:

@pytest.mark.run_loop
def test_acquire_limit_maxsize(mcache_params,
                               loop):
    pool = MemcachePool(minsize=1, maxsize=1, loop=loop, **mcache_params)
    assert pool.size() == 0

    # Create up to max connections
    _conn = yield from pool.acquire()
    assert pool.size() == 1
    pool.release(_conn)

    @asyncio.coroutine
    def acquire_wait_release():
        conn = yield from pool.acquire()
        assert conn is _conn
        yield from asyncio.sleep(0.01, loop=loop)
        assert len(pool._in_use) == 1
        assert pool.size() == 1
        assert pool._pool.qsize() == 0
        pool.release(conn)

    yield from asyncio.gather(*([acquire_wait_release()] * 10), loop=loop)
    assert pool.size() == 1
    assert len(pool._in_use) == 0

[argaen]

Lol, this was already fixed in #14.