Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Photometry download gives bad request when too much data available #196

Open
rachel3834 opened this issue Sep 11, 2024 · 3 comments
Open

Photometry download gives bad request when too much data available #196

rachel3834 opened this issue Sep 11, 2024 · 3 comments
Assignees
@rachel3834
Labels
bug Something isn't working

Comments

@rachel3834
Copy link
Contributor

Using the photometry table to download photometry data, I get the following Bad Request error:

mop-559b6fc7d4-gnnt7 nginx 10.100.33.21 - - [11/Sep/2024:04:19:10 +0000] "GET /targets/42858/?tab=observations HTTP/1.1" 200 16204417 "https://mop.lco.global/observations/LCO/create/?target_id=42858" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "184.189.222.100"
mop-559b6fc7d4-whxp8 mop The number of GET/POST parameters exceeded settings.DATA_UPLOAD_MAX_NUMBER_FIELDS.
mop-559b6fc7d4-whxp8 mop Traceback (most recent call last):
mop-559b6fc7d4-whxp8 mop File "/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
mop-559b6fc7d4-whxp8 mop response = get_response(request)
mop-559b6fc7d4-whxp8 mop File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 185, in _get_response
mop-559b6fc7d4-whxp8 mop response = middleware_method(
mop-559b6fc7d4-whxp8 mop File "/usr/local/lib/python3.10/site-packages/django/middleware/csrf.py", line 470, in process_view
mop-559b6fc7d4-whxp8 mop self._check_token(request)
mop-559b6fc7d4-whxp8 mop File "/usr/local/lib/python3.10/site-packages/django/middleware/csrf.py", line 373, in _check_token
mop-559b6fc7d4-whxp8 mop request_csrf_token = request.POST.get("csrfmiddlewaretoken", "")
mop-559b6fc7d4-whxp8 mop File "/usr/local/lib/python3.10/site-packages/django/core/handlers/wsgi.py", line 93, in _get_post
mop-559b6fc7d4-whxp8 mop self._load_post_and_files()
mop-559b6fc7d4-whxp8 mop File "/usr/local/lib/python3.10/site-packages/django/http/request.py", line 373, in _load_post_and_files
mop-559b6fc7d4-whxp8 mop self._post, self._files = self.parse_file_upload(self.META, data)
mop-559b6fc7d4-whxp8 mop File "/usr/local/lib/python3.10/site-packages/django/http/request.py", line 321, in parse_file_upload
mop-559b6fc7d4-whxp8 nginx 10.100.33.21 - - [11/Sep/2024:04:19:24 +0000] "POST /dataproducts/target/42858/share/ HTTP/1.1" 400 143 "https://mop.lco.global/targets/42858/?tab=observations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "184.189.222.100"
mop-559b6fc7d4-whxp8 mop return parser.parse()
mop-559b6fc7d4-whxp8 mop File "/usr/local/lib/python3.10/site-packages/django/http/multipartparser.py", line 123, in parse
mop-559b6fc7d4-whxp8 mop return self._parse()
mop-559b6fc7d4-whxp8 mop File "/usr/local/lib/python3.10/site-packages/django/http/multipartparser.py", line 203, in _parse
mop-559b6fc7d4-whxp8 mop raise TooManyFieldsSent(
mop-559b6fc7d4-whxp8 mop django.core.exceptions.TooManyFieldsSent: The number of GET/POST parameters exceeded settings.DATA_UPLOAD_MAX_NUMBER_FIELDS.
mop-559b6fc7d4-whxp8 mop Bad Request: /dataproducts/target/42858/share/
mop-559b6fc7d4-whxp8 mop 127.0.0.1 - - [11/Sep/2024:04:19:24 +0000] "POST /dataproducts/target/42858/share/ HTTP/1.1" 400 143 "https://mop.lco.global/targets/42858/?tab=observations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36"
mop-559b6fc7d4-whxp8 nginx 10.100.8.93 - - [11/Sep/2024:04:19:26 +0000] "GET /robots.txt HTTP/1.1" 200 26 "-" "Gabanzabot/1.1 (Gabanza Search Engine; https://www.gabanza.com)" "207.231.111.171"
mop-559b6fc7d4-whxp8 mop 127.0.0.1 - - [11/Sep/2024:04:19:26 +0000] "GET /robots.txt HTTP/1.1" 200 26 "-" "Gabanzabot/1.1 (Gabanza Search Engine; https://www.gabanza.com)"
@rachel3834 rachel3834 added the bug Something isn't working label Sep 11, 2024
@rachel3834 rachel3834 self-assigned this Sep 11, 2024
@rachel3834
Copy link
Contributor Author

Clicking on the datafiles to download them separately produces a different issue:

  • mop-runtap-28767120-8959n › mop
    mop-559b6fc7d4-gnnt7 mop Internal Server Error: /dataproducts/data/87995/share/
    mop-559b6fc7d4-gnnt7 mop Traceback (most recent call last):
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    mop-559b6fc7d4-gnnt7 mop response = get_response(request)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 199, in _get_response
    mop-559b6fc7d4-gnnt7 mop response = self.process_exception_by_middleware(e, request)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 362, in process_exception_by_middleware
    mop-559b6fc7d4-gnnt7 mop response = middleware_method(request, exception)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/tom_common/middleware.py", line 29, in process_exception
    mop-559b6fc7d4-gnnt7 mop raise exception
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    mop-559b6fc7d4-gnnt7 mop response = wrapped_callback(request, *callback_args, **callback_kwargs)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/django/views/generic/base.py", line 104, in view
    mop-559b6fc7d4-gnnt7 mop return self.dispatch(request, *args, **kwargs)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/django/views/generic/base.py", line 143, in dispatch
    mop-559b6fc7d4-gnnt7 mop return handler(request, *args, **kwargs)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/tom_dataproducts/views.py", line 427, in post
    mop-559b6fc7d4-gnnt7 mop return download_data(form_data, selected_data=selected_data)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/tom_dataproducts/sharing.py", line 359, in download_data
    mop-559b6fc7d4-gnnt7 mop table.sort(sort_fields)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/astropy/table/table.py", line 3517, in sort
    mop-559b6fc7d4-gnnt7 mop indexes = self.argsort(keys, kind=kind, reverse=reverse)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/astropy/table/table.py", line 3429, in argsort
    mop-559b6fc7d4-gnnt7 mop index = get_index(self, names=keys)
    mop-559b6fc7d4-gnnt7 mop File "/usr/local/lib/python3.10/site-packages/astropy/table/index.py", line 650, in get_index
    mop-559b6fc7d4-gnnt7 mop raise ValueError(f"{names} is not a subset of table columns")
    mop-559b6fc7d4-gnnt7 mop ValueError: {'timestamp'} is not a subset of table columns

@rachel3834
Copy link
Contributor Author

The above error occurs when downloading both pyDANDIA pipeline datafiles as well as ATLAS photometry harvested from the ATLAS forced photometry service.

@rachel3834 rachel3834 mentioned this issue Sep 11, 2024
Open
@jchate6
Copy link
Contributor

jchate6 commented Sep 13, 2024

You can access DataProducts by clicking the file name under manage data.
There is an issue in TOMBase that I believe is related to this: TOMToolkit/tom_base#991

@rachel3834 rachel3834 moved this to Backlog in MOP Development Apr 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rachel3834 rachel3834

Labels
bug Something isn't working
Projects
MOP Development
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rachel3834 @jchate6