OpenDistro - Index with nested columns throwing errors while exporting as Table Visualization

[syamat]

Index with nested columns throwing errors while exporting as Table Visualization. The data is shown correctly in SQL Lab.

:POST http://localhost:9200/_opendistro/_sql/?format=csv [status:N/A request:10.012s]
Traceback (most recent call last):
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
six.raise_from(e, None)
File "", line 2, in raise_from
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.6/http/client.py", line 1346, in getresponse
response.begin()
File "/usr/lib/python3.6/http/client.py", line 307, in begin
version, status, reason =

self._read_status()
File "/usr/lib/python3.6/http/client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 238, in perform_request
method, url, body, retries=Retry(False), headers=request_headers, **kw
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 344, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 389, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 307, in _raise_timeout
raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)
DEBUG:elasticsearch:> {"query":"SELECT name AS name, encounter AS encounter FROM (select * from pt_idx) AS expr_qry LIMIT 1000"}
ERROR:root:Query SELECT name AS name,
encounter AS encounter
FROM
(select *
from pt_idx) AS expr_qry
LIMIT 1000 on schema default failed
Traceback (most recent call last):
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
six.raise_from(e, None)
File "", line 2, in raise_from
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.6/http/client.py", line 1346, in getresponse
response.begin()
File "/usr/lib/python3.6/http/client.py", line 307, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.6/http/client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 238, in perform_request
method, url, body, retries=Retry(False), headers=request_headers, **kw
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 344, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 389, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 307, in _raise_timeout
raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/es/baseapi.py", line 220, in elastic_query
resp = self.es.transport.perform_request("POST", path, body=payload)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/elasticsearch/transport.py", line 358, in perform_request
timeout=timeout,
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 249, in perform_request
raise ConnectionTimeout("TIMEOUT", str(e), e)
elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/superset/connectors/sqla/models.py", line 999, in query
df = self.database.get_df(sql, self.schema, mutator)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/superset/models/core.py", line 1009, in get_df
self.db_engine_spec.execute(cursor, sqls[-1])
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/superset/db_engine_specs/base.py", line 773, in execute
cursor.execute(query)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/es/baseapi.py", line 25, in g
return f(self, *args, **kwargs)
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/es/opendistro/api.py", line 155, in execute
_results = self.elastic_query(query, csv=True).split("\n")
File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/es/baseapi.py", line 223, in elastic_query
f"Error connecting to {self.url}: {e.info}"
es.exceptions.OperationalError: Error connecting to http://localhost:9200: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)
INFO:werkzeug:127.0.0.1 - - [20/Dec/2019 18:36:08] "POST /superset/explore_json/ HTTP/1.1" 400 -
DEBUG:root:[stats_logger] (incr) log

[syamat]

[dpgaspar]

@syamat, sorry but opendistro is not fully supported yet, charts won't work

[dai-chen]

@dpgaspar FYI, ODFE 1.13 release is coming soon with new SQL query engine turned on. Here is the doc: https://github.com/opendistro-for-elasticsearch/sql/blob/develop/docs/dev/NewSQLEngine.md. Feel free to reach out to us if any question. We can also schedule meeting if needed. Thanks!

[syamat]

Hi @dai-chen

Will the new engine support nested fields? If not, will it fall-back to old engine in case a nested field is queried using the new SQL engine.

[syamat]

With this issue be resolved with the next release?

[dai-chen]

With this issue be resolved with the next release?

The new engine only supports object fields in ODFE 1.13. The nested field query will fall back to old engine.Please find more details here: https://github.com/opendistro-for-elasticsearch/sql/blob/develop/docs/user/beyond/partiql.rst#querying-nested-tuple-values. We will add support for nested fields in future release. Thanks!

[syamat]

Is this issue fixed with the new release?