Sending e-mails over 7-8MB fail on Google App Engine Standard

[gae123]

Issue Summary

I just switched from Google App Engine e-mail API to SendGrid. I used to have a 20MB limit per e-mail and I thought that SendGrid's 30MiB would be OK.

Now I cannot any messages over 7-8. The code stack trace is the following:

Code:
sg = sendgrid.SendGridAPIClient(apikey=SENDGRID_KEY)
...
response = sg.client.mail.send.post(request_body=data)

Stack Trace:
File "/base/data/home/apps/sAPPNAME.405544986480217849/swplatform/util/sendgrid_mail.py", line 157, in Send
response = sg.client.mail.send.post(request_body=data)
File "/base/data/home/apps/sneatschool2/1-96-85.405544986480217849/python_libs/python_http_client/client.py", line 227, in http_request
return Response(self._make_request(opener, request))
File "/base/data/home/apps/s~neatschool2/1-96-85.405544986480217849/python_libs/python_http_client/client.py", line 157, in _make_request
return opener.open(request)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 433, in open
response = self._open(req, data)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 451, in _open
'_open', req)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 411, in _call_chain
result = func(*args)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 1245, in https_open
context=self._context)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 1205, in do_open
r = h.getresponse(buffering=True)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/gae_override/httplib.py", line 524, in getresponse
raise InvalidURL(str(e))
InvalidURL: Request exceeds 10 MiB limit for URL: https://api.sendgrid.com/v3/mail/send

[thinkingserious]

Hello @gae123,

It looks like the issue is that when using GAE's google.appengine.api.urlfetch (this is probably the underlying card that urllib2 is using), you are hitting their limit. Perhaps you can ask if they can increase that limit for your account.

With Best Regards,

Elmer

[gae123]

Well I opened a case with Google App Engine as well, see this but the reality is that the limit of GAE came before your APIs and SDK and IMHO you need to find a way to respect it and provide an SDK that works with that limitation.

[thinkingserious]

Hi @gae123,

Thanks for the additional feedback @gae123. I will leave this issue open so that we may provide a helper in this SDK to assist getting around the GAE limitation.

For this issue to gain priority in our backlog, we need additional +1's or a PR. When we receive a PR, that provides the biggest jump in priority.

With Best Regards,

Elmer

[thinkingserious]

Since there has been no activity on this issue since March 1, 2020, we are closing this issue. Please feel free to reopen or create a new issue if you still require assistance. Thank you!

[gae123]

This remains a problem for our customers that use our service and would be one of the top reasons to move away from sendgrid to a competing offering.