Addresses Issue #3138 - ESO - Enable Table Access Protocol (TAP) queries #3339

juanmcloaiza · 2025-06-05T12:32:44Z

Closes #3138

Changes:

Switch querying interface from WDB to TAP in querying functions:

Deprecate arguments:

open_form
cache

New functionality:

Allow plain ADQL queries via query_tap_service (with authentication as well)
Cone search using cone_ra, cone_dec, cone_radius arguments
Retrieve record count before querying the archive, via count_only argument
Ask query functions to print the underlying ADQL queries without issuing them

codecov · 2025-06-05T12:59:07Z

Codecov Report

❌ Patch coverage is 84.58781% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.31%. Comparing base (c588b25) to head (2272c91).
⚠️ Report is 93 commits behind head on main.

Files with missing lines	Patch %	Lines
astroquery/eso/core.py	77.96%	39 Missing ⚠️
astroquery/eso/utils.py	96.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3339      +/-   ##
==========================================
+ Coverage   70.54%   71.31%   +0.76%     
==========================================
  Files         232      234       +2     
  Lines       20022    20034      +12     
==========================================
+ Hits        14125    14287     +162     
+ Misses       5897     5747     -150

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

juanmcloaiza · 2025-07-11T09:26:01Z

Hi @bsipocz , could you please give us a time estimate of the review process? No pressure at all though, we only need to plan our activities internally. Thanks a lot in advance.

bsipocz · 2025-07-21T18:08:01Z

@juanmcloaiza - sorry for the delay. I'll try to get to the review this week.

bsipocz

I'm sorry to have this many comments and thank you for your patience while waiting for this review.

I acknowledge that this seems to a lot, but most of them are minor and/or helping API consistency with the other modules and/or making the code cleaner and easier to read and follow; which will be helpful for the future maintenance.

astroquery/eso/core.py

astroquery/eso/utils.py

astroquery/eso/tests/test_eso_remote.py

docs/eso/eso.rst

juanmcloaiza · 2025-08-05T13:32:51Z

@bsipocz, thanks a lot for your comments, I have addressed most of them (the easy ones); for the rest, I'd still need some feedback from you. If you can give me feedback today, I can work immediately on it and push and update by tomorrow. Afterwards, I won't be able to work on Astroquery for the next couple of weeks. Just so you know, in case you can squeeze this review in your day 🤓 Thanks again! 👍

bsipocz · 2025-08-05T17:54:38Z

@juanmcloaiza - I'm sorry, it's very unlikely that I can get back to this today, but I'll try to do it soon so you can pick it up again in a few weeks time.

juanmcloaiza · 2025-09-01T09:37:03Z

Hi @bsipocz , just a ping about the second review round. Have you had some time for it? We're looking forward to hearing back from you.

astroquery/eso/core.py

astroquery/eso/utils.py

astroquery/eso/core.py

juanmcloaiza · 2025-09-24T09:23:54Z

There's no field for me to reply to your las two comments (strange...), but here are my replies:

About the maxrec - Sure thing, ~~I'll make the change~~ I've replaced maxrec by ROW_LIMIT.

About the deprecated keys message - The problem is that the astroquery ESO module (like many others) is currently using the keywords of the underlying querying engine directly in its API (like box, coord1, coord2 of WDB). These are not python arguments, but keys of a very normal dictionary that is passed to the python API and transformed internally to a WDB or TAP query. Since these are not python arguments, we need ad-hoc cecking, and we do it only for the most common WDB keywords.

The messages users get are, for example, "box, coord1 and coord2 are deprecated; use cone_ra, cone_dec and cone_radius instead."

Without _raise_if_has_deprecated_keys, the code would explode later on with a message about a bad TAP query, printing the query itself, but without saying explicitly that the problem is in the keywords.

@ashleythomasbarnes has already an extensive documentation with examples ready to be submitted after this PR. For the moment, the changes to the documentation in this PR, include a warning and a how-to regarding the incompatibility between WDB and TAP:

Thanks for the feedback, @keflavich, I'll get everything done this week. Please just let me know about my question regarding the deprecation of arguments; for the rest everything is clear 👍

juanmcloaiza · 2025-09-26T10:09:35Z

@keflavich, your comments have been taken into account, thanks for having taken the time. [cc @bsipocz, @ashleythomasbarnes, @szampier]. -- EDIT: Apologies, I went again one by one through all the conversations and I noticed three comments that had escaped my sight. I have taken them into account now. Please double check and let me know if anything is still missing.

juanmcloaiza · 2025-10-23T06:54:46Z

@keflavich, @bsipocz, just FYI, on making the requested changes I introduced a bug that is now fixed in commit aef5c0f [@ashleythomasbarnes, @szampier].

bsipocz

A batch of, I hope not too difficult to address, comments, most of them are OK to be done in a follow-up (especially when it concerns a non public method or something internal etc.)

I've also unresolved a couple of conversations where we said the item is for a follow-up; ideally we should collect those out into a separate issue.

astroquery/eso/core.py

bsipocz · 2025-11-19T12:02:24Z

astroquery/eso/core.py

+                          "To retrieve all the records set to None the maxrec attribute",
+                          MaxResultsWarning)
+
+    def _try_download_pyvo_table(self,


but what is a pyvo_table, and is this really a download or instead of running a query?
(we usually use "download" to refer to downloading files from an archive as opposed to run a table query.

Having a parser method is quite usual in the other modules, especially if it can be reused for multiple methods; but here I don't see we gain that much readibility with the added complexity.

bsipocz · 2025-11-19T12:03:05Z

astroquery/eso/core.py

+            table_with_an_extra_row = tap.search(query=query_str, maxrec=row_limit_plus_one).to_table()
+            self._maybe_warn_about_table_length(table_with_an_extra_row, row_limit_plus_one)
+        except DALQueryError:
+            log.error(message(query_str))


Why don't we raise this error, and why swallow the original exception type from the log?

astroquery/eso/core.py

bsipocz · 2025-11-19T12:58:33Z

astroquery/eso/core.py

+
+    @deprecated_renamed_argument(('open_form', 'cache'), (None, None),
+                                 since=['0.4.11', '0.4.11'])
+    def query_main(


Keeping this unresolved so it's easier to pick up the comment for a follow-up PR

bsipocz · 2025-11-19T12:59:03Z

astroquery/eso/core.py

+        astropy.conf.max_lines = n_
+        astropy.conf.max_width = m_
+
+    def _query_on_allowed_values(


Unresolve this to make it easier to find for a follow-up

bsipocz · 2025-11-19T13:04:45Z

astroquery/eso/core.py

-        >>> dptbl = Eso.query_instrument('apex', pi_coi='ginsburg')
-        >>> dpids = [row['DP.ID'] for row in dptbl if 'Map' in row['Object']]
-        >>> files = Eso.retrieve_data(dpids)
+        dptbl = Eso.query_instrument('apex', pi_coi='ginsburg')
+        dpids = [row['DP.ID'] for row in dptbl if 'Map' in row['Object']]
+        files = Eso.retrieve_data(dpids)


Why did you remove the doctesting here? IMO if there are examples included they should be runable (meaning that they will also likely need to be fixed up with an import line)

astroquery/eso/core.py

bsipocz

OK, let's go ahead with the merge, I do one more batch commits for the simpler review comments, and will unresolve a bunch more conversations so it will be easier to pick them up in the follow-up.

bsipocz · 2025-11-20T13:14:36Z

astroquery/eso/core.py

        return username, password

-    def _login(self, *, username: str = None, store_password: bool = False,
+    def _login(self, username: str = None, store_password: bool = False,


Why did you remove the mandatoriness of kwargs here? general rule of thumb to not allow function calls with long positional signatures.

bsipocz · 2025-11-20T13:25:38Z

astroquery/eso/core.py

+    def query_surveys(
+            self,
+            surveys: Union[List[str], str] = None, *,
+            cone_ra: float = None, cone_dec: float = None, cone_radius: float = None,


oh, I've just noticed these to be named weirdly. Please in a follow-up make it change to the usual API to rename cone_ra/dec to one argument that is called coordinates and which takes a SkyCoord object (or a string that can be turned to a SkyCoord/etc, this is very similar to all the other modules.

And also rename cone_radius to just radius.

CHANGES.rst

bsipocz · 2025-11-20T13:27:03Z

CHANGES.rst

+
+- Switch querying interface from WDB to TAP in querying functions. [#3339]
+- Allow plain ADQL queries via ``query_tap_service`` (with authentication as well). [#3339]
+- Cone search using ``cone_ra`, ``cone_dec`, ``cone_radius`` arguments. [#3339]


These will need to be renamed/refactored to match other modules.

bsipocz · 2025-11-20T14:10:38Z

RTD failure is unrelated and is due to a recent plugin release which I'll resolved in a separate PR.

bsipocz · 2025-11-20T14:11:46Z

Thanks @juanmcloaiza for the patience here. I'm looking fwd for the follow-up PRs.

juanmcloaiza · 2025-11-20T15:29:18Z

@bsipocz , thanks a lot for your time today and the careful review. Looking forward for a continuous and fruitful collaboration! 🚀

juanmcloaiza force-pushed the feature/ISSUE-3138-TAP branch from 720fbd9 to 22d0c30 Compare June 5, 2025 12:55

juanmcloaiza force-pushed the feature/ISSUE-3138-TAP branch 3 times, most recently from a987745 to 4b775f8 Compare June 5, 2025 13:49

juanmcloaiza mentioned this pull request Jun 5, 2025

Enable TAP for ESO module #3141

Closed

8 tasks

bsipocz added the eso label Jul 31, 2025

bsipocz added this to the v0.4.11 milestone Jul 31, 2025

bsipocz reviewed Jul 31, 2025

View reviewed changes

juanmcloaiza force-pushed the feature/ISSUE-3138-TAP branch from 3742bb9 to d3be2af Compare August 5, 2025 12:15

juanmcloaiza added 10 commits September 11, 2025 15:05

Define tap_url in config; maxrec=1000

0e4167a

Define helper functions in utils.py

27be9bb

Update and expand test suite

ec74e3b

Update data needed by test suite

63cb4af

Update test data paths

55f768f

Write TAP-enabled query functions

7267429

Update documentation to be consistent with code

4833ab4

Address review from maintainers - mostly renamings

ae1d3b4

Remove __doctest_skip__

c1a09b9

Update CHANGES.rst -- resolve conflict

9d713ee

juanmcloaiza force-pushed the feature/ISSUE-3138-TAP branch from f4a18a6 to 9d713ee Compare September 11, 2025 13:12

bsipocz modified the milestones: v0.4.11, 0.4.12 Sep 18, 2025

Merge branch 'main' into feature/ISSUE-3138-TAP

332b36e

keflavich reviewed Sep 23, 2025

View reviewed changes

astroquery/eso/core.py Show resolved Hide resolved

keflavich reviewed Sep 23, 2025

View reviewed changes

astroquery/eso/core.py Outdated Show resolved Hide resolved

keflavich reviewed Sep 23, 2025

View reviewed changes

juanmcloaiza added 4 commits September 26, 2025 10:41

Remove unused arguments

b798354

Rename maxrec to ROW_LIMIT

dca960a

Use context manager in list_column function

2d5584c

Remove env var tap url

b887f73

juanmcloaiza added 4 commits October 2, 2025 15:35

On unexpected exceptions, re-rise from original type

15043f8

Remove unused kwargs from _login function

285b9c5

Raise MaxResultsWarning only when truncating a table

bb15ee7

Bugfix - int overflow on maxrec calculation

aef5c0f

juanmcloaiza force-pushed the feature/ISSUE-3138-TAP branch from 4dbc87c to aef5c0f Compare October 22, 2025 19:27

bsipocz reviewed Nov 19, 2025

View reviewed changes

bsipocz approved these changes Nov 20, 2025

View reviewed changes

Addressing simplest review comments

2272c91

bsipocz force-pushed the feature/ISSUE-3138-TAP branch from ff18444 to 2272c91 Compare November 20, 2025 13:49

bsipocz merged commit a17bf41 into astropy:main Nov 20, 2025
11 of 12 checks passed

vandalt mentioned this pull request Dec 2, 2025

ESO: authenticated not enabled by default after login for queries #3473

Open

Uh oh!

Addresses Issue #3138 - ESO - Enable Table Access Protocol (TAP) queries #3339

Addresses Issue #3138 - ESO - Enable Table Access Protocol (TAP) queries #3339

Uh oh!

Conversation

juanmcloaiza commented Jun 5, 2025 • edited by bsipocz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes:

Uh oh!

codecov bot commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

juanmcloaiza commented Jul 11, 2025

Uh oh!

bsipocz commented Jul 21, 2025

Uh oh!

bsipocz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juanmcloaiza commented Aug 5, 2025

Uh oh!

bsipocz commented Aug 5, 2025

Uh oh!

juanmcloaiza commented Sep 1, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juanmcloaiza commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juanmcloaiza commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juanmcloaiza commented Oct 23, 2025

Uh oh!

bsipocz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bsipocz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juanmcloaiza commented Jun 5, 2025 •

edited by bsipocz

Loading

codecov bot commented Jun 5, 2025 •

edited

Loading

juanmcloaiza commented Sep 24, 2025 •

edited

Loading

juanmcloaiza commented Sep 26, 2025 •

edited

Loading