Pysolr using Docker names instead of localhost for SOLR connection

[an-vb-is]

Structure

• SOLR Cloud v. 9.8.0, 3 Nodes
• Zookeeper v. 3.9.3, 3 Instances
• Pysolr latest
• Pytest v. 8.3.4

The Problem

In my projects RestAPI I have a function called "search" which makes a request to SOLR to get some search results. To make the request I use Pysolr. Normally all my project components run in the same Docker network but when I run my RestAPI tests using Pytest the RestAPI is started separately. Pysolr then tries to connect to my SOLR Nodes using the Docker container names, which will obviously not work.

What I want

I need Pysolr to connect to the SOLR Node using localhost, but I can't figure out how to get this working.

What I tried

I tried to change my docker-compose-override.yml file in a way that my SOLR Nodes are running on localhost and not on their docker names. And I tried to change the various environment variables for the Zookeeper and SOLR servers / hosts to use localhost. I also tried to somehow set in Pysolr directly the SOLR Nodes.

What I expected

I expected that Pysolr would use localhost:Port to connect to the SOLR Nodes.

Code

docker-compose.yml

 search1:
    image: solr:9.8.0
    restart: always
    ports:
      - "8983:8983"
    networks:
      - default
      - search
    environment:
      ZK_HOST: search-zoo1,search-zoo2,search-zoo3
      SOLR_HOST: search1
      SOLR_PORT: 8983
      SOLR_MODULES: jwt-auth
    depends_on:
      - search-zoo1
      - search-zoo2
      - search-zoo3

  search2:
    image: solr:9.8.0
    restart: always
    networks:
      - default
      - search
    environment:
      ZK_HOST: search-zoo1,search-zoo2,search-zoo3
      SOLR_HOST: search2
      SOLR_PORT: 8983
      SOLR_MODULES: jwt-auth
    depends_on:
      - search-zoo1
      - search-zoo2
      - search-zoo3

  search3:
    image: solr:9.8.0
    restart: always
    networks:
      - default
      - search
    environment:
      ZK_HOST: search-zoo1,search-zoo2,search-zoo3
      SOLR_HOST: search3
      SOLR_PORT: 8983
      SOLR_MODULES: jwt-auth
    depends_on:
      - search-zoo1
      - search-zoo2
      - search-zoo3

  search-zoo1:
    image: zookeeper:3.9.3
    networks:
      - search
      - default
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=search-zoo1:2888:3888;2181 server.2=search-zoo2:2888:3888;2181 server.3=search-zoo3:2888:3888;2181
      ZOO_4LW_COMMANDS_WHITELIST: "mntr,conf,ruok"
    expose:
      - "2181"

  search-zoo2:
    image: zookeeper:3.9.3
    networks:
      - search
      - default
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=search-zoo1:2888:3888;2181 server.2=search-zoo2:2888:3888;2181 server.3=search-zoo3:2888:3888;2181
      ZOO_4LW_COMMANDS_WHITELIST: "mntr,conf,ruok"
    expose:
      - "2181"

  search-zoo3:
    image: zookeeper:3.9.3
    networks:
      - search
      - default
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=search-zoo1:2888:3888;2181 server.2=search-zoo2:2888:3888;2181 server.3=search-zoo3:2888:3888;2181
      ZOO_4LW_COMMANDS_WHITELIST: "mntr,conf,ruok"
    expose:
      - "2181"

docker-compose-override.yml

  search1:
    restart: "no"
    ports:
      - "8983:8983"
    environment:
      ZK_HOST: search-zoo1:2181,search-zoo2:2182,search-zoo3:2183
      SOLR_JAVA_OPTS: -Dsolr.auth.jwt.allowOutboundHttp=true
      SOLR_HOST: search1
      SOLR_PORT: 8983

  search2:
    restart: "no"
    ports:
      - "8984:8983"
    environment:
      ZK_HOST: search-zoo1:2181,search-zoo2:2182,search-zoo3:2183
      SOLR_JAVA_OPTS: -Dsolr.auth.jwt.allowOutboundHttp=true
      SOLR_HOST: search2
      SOLR_PORT: 8984

  search3:
    restart: "no"
    ports:
      - "8985:8983"
    environment:
      ZK_HOST: search-zoo1:2181,search-zoo2:2182,search-zoo3:2183
      SOLR_JAVA_OPTS: -Dsolr.auth.jwt.allowOutboundHttp=true
      SOLR_HOST: search3
      SOLR_PORT: 8985

  search-zoo1:
    ports:
      - "2181:2181"
    environment:
      ZOO_SERVERS: server.1=search-zoo1:2888:3888;2181 server.2=search-zoo2:2888:3888;2182 server.3=search-zoo3:2888:3888;2183

  search-zoo2:
    ports:
      - "2182:2181"
    environment:
      ZOO_SERVERS: server.1=search-zoo1:2888:3888;2181 server.2=search-zoo2:2888:3888;2182 server.3=search-zoo3:2888:3888;2183

  search-zoo3:
    ports:
      - "2183:2181"
    environment:
      ZOO_SERVERS: server.1=search-zoo1:2888:3888;2181 server.2=search-zoo2:2888:3888;2182 server.3=search-zoo3:2888:3888;2183

search.py

async def search(
    skip: int = 0,
    limit: int = 100,
    params: SearchQueryParams = Depends(),
) -> Any:
    """
    Search query.
    """

    zookeeper = pysolr.ZooKeeper("localhost:2181,localhost:2182,localhost:2183")

    solr = pysolr.SolrCloud(zookeeper, "tag", auth=JWTAuth(settings.SOLR_TOKEN))

    res = solr.search(q=params.query, start=skip, rows=limit)

    return SearchResults(data=res, count=res.hits)

Trace back

pysolr:pysolr.py:428 Failed to connect to server at http://search2:8984/solr/tag/select/?q=test-tag&start=0&rows=100&wt=json: HTTPConnectionPool(host='search2', port=8984): Max retries exceeded with url: /solr/tag/select/?q=test-tag&start=0&rows=100&wt=json (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x10ba89fd0>: Failed to resolve 'search2' ([Errno 8] nodename nor servname provided, or not known)"))
Traceback (most recent call last):
  File "/restapi/.venv/lib/python3.13/site-packages/urllib3/connection.py", line 199, in _new_conn
    sock = connection.create_connection(
        (self._dns_host, self.port),
    ...<2 lines>...
        socket_options=self.socket_options,
    )
  File "/restapi/.venv/lib/python3.13/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.pyenv/versions/3.13.1/lib/python3.13/socket.py", line 975, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/restapi/.venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
        conn,
    ...<10 lines>...
        **response_kw,
    )
  File "/restapi/.venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 495, in _make_request
    conn.request(
    ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<6 lines>...
        enforce_content_length=enforce_content_length,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/restapi/.venv/lib/python3.13/site-packages/urllib3/connection.py", line 441, in request
    self.endheaders()
    ~~~~~~~~~~~~~~~^^
  File "/.pyenv/versions/3.13.1/lib/python3.13/http/client.py", line 1331, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.pyenv/versions/3.13.1/lib/python3.13/http/client.py", line 1091, in _send_output
    self.send(msg)
    ~~~~~~~~~^^^^^
  File "/.pyenv/versions/3.13.1/lib/python3.13/http/client.py", line 1035, in send
    self.connect()
    ~~~~~~~~~~~~^^
  File "/restapi/.venv/lib/python3.13/site-packages/urllib3/connection.py", line 279, in connect
    self.sock = self._new_conn()
                ~~~~~~~~~~~~~~^^
  File "/restapi/.venv/lib/python3.13/site-packages/urllib3/connection.py", line 206, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPConnection object at 0x10ba89fd0>: Failed to resolve 'search2' ([Errno 8] nodename nor servname provided, or not known)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/restapi/.venv/lib/python3.13/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
        method=request.method,
    ...<9 lines>...
        chunked=chunked,
    )
  File "/restapi/.venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
        method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
    )
  File "/restapi/.venv/lib/python3.13/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='search2', port=8984): Max retries exceeded with url: /solr/tag/select/?q=test-tag&start=0&rows=100&wt=json (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x10ba89fd0>: Failed to resolve 'search2' ([Errno 8] nodename nor servname provided, or not known)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/restapi/.venv/lib/python3.13/site-packages/pysolr.py", line 414, in _send_request
    resp = requests_method(
        url,
    ...<4 lines>...
        auth=self.auth,
    )
  File "/restapi/.venv/lib/python3.13/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/restapi/.venv/lib/python3.13/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/restapi/.venv/lib/python3.13/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/restapi/.venv/lib/python3.13/site-packages/requests/adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='search2', port=8984): Max retries exceeded with url: /solr/tag/select/?q=test-tag&start=0&rows=100&wt=json (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x10ba89fd0>: Failed to resolve 'search2' ([Errno 8] nodename nor servname provided, or not known)"))