What's Wrong ? #103
Comments
|
The binder link in the individual notebook pages points to dask-examples. See for example the Binder button at the top of this page. https://tutorial.dask.org/07_dataframe_storage.html |
Also, hooray! This is great to see. |
|
Fixed in #104
|
|
xref #105 |
|
Internet users are more likely to jump around the materials than students
within an organized tutorial session. As a result, we might consider
making the notebooks more independent. In particular, I jumped to a
notebook and started to press Shift-Enter a bunch and ran into problems
because a dataset was missing. I then had to go back to notebook 00, and
run the prep.py script. I wonder if either:
1. We might run this as part of the docker start script
2. We might include a `%run prep.py` cell at the top of each notebook
…On Mon, Aug 5, 2019 at 12:06 PM James Bourbeau ***@***.***> wrote:
xref #105 <#105>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#103?email_source=notifications&email_token=AACKZTCK2WRPS3E2FJZE4EDQDB2ZZA5CNFSM4IJN55E2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3SYS2Q#issuecomment-518359402>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AACKZTBLTRZN2I653BNKSZDQDB2ZZANCNFSM4IJN55EQ>
.
|
b = db.read_text('s3://dask-data/nyc-taxi/2015/yellow_tripdata_2015-01.csv')
b.take(1)Traceback---------------------------------------------------------------------------
NoCredentialsError Traceback (most recent call last)
<ipython-input-6-f250b1423dc0> in <module>
2 # each partition is a remote CSV text file
3 b = db.read_text('s3://dask-data/nyc-taxi/2015/yellow_tripdata_2015-01.csv')
----> 4 b.take(1)
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/bag/core.py in take(self, k, npartitions, compute, warn)
1222
1223 if compute:
-> 1224 return tuple(b.compute())
1225 else:
1226 return b
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/base.py in compute(self, **kwargs)
154 dask.base.compute
155 """
--> 156 (result,) = compute(self, traverse=False, **kwargs)
157 return result
158
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/base.py in compute(*args, **kwargs)
396 keys = [x.__dask_keys__() for x in collections]
397 postcomputes = [x.__dask_postcompute__() for x in collections]
--> 398 results = schedule(dsk, keys, **kwargs)
399 return repack([f(r, *a) for r, (f, a) in zip(results, postcomputes)])
400
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/multiprocessing.py in get(dsk, keys, num_workers, func_loads, func_dumps, optimize_graph, pool, **kwargs)
190 get_id=_process_get_id, dumps=dumps, loads=loads,
191 pack_exception=pack_exception,
--> 192 raise_exception=reraise, **kwargs)
193 finally:
194 if cleanup:
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/local.py in get_async(apply_async, num_workers, dsk, result, cache, get_id, rerun_exceptions_locally, pack_exception, raise_exception, callbacks, dumps, loads, **kwargs)
460 _execute_task(task, data) # Re-execute locally
461 else:
--> 462 raise_exception(exc, tb)
463 res, worker_id = loads(res_info)
464 state['cache'][key] = res
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/compatibility.py in reraise(exc, tb)
109 def reraise(exc, tb=None):
110 if exc.__traceback__ is not tb:
--> 111 raise exc.with_traceback(tb)
112 raise exc
113
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/local.py in execute_task()
228 try:
229 task, data = loads(task_info)
--> 230 result = _execute_task(task, data)
231 id = get_id()
232 result = dumps((result, id))
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/core.py in _execute_task()
117 func, args = arg[0], arg[1:]
118 args2 = [_execute_task(a, cache) for a in args]
--> 119 return func(*args2)
120 elif not ishashable(arg):
121 return arg
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/bag/core.py in safe_take()
2159
2160 def safe_take(n, b, warn=True):
-> 2161 r = list(take(n, b))
2162 if len(r) != n and warn:
2163 warnings.warn("Insufficient elements for `take`. {0} elements "
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/bag/text.py in file_to_blocks()
102
103 def file_to_blocks(lazy_file):
--> 104 with lazy_file as f:
105 for line in f:
106 yield line
/srv/conda/envs/notebook/lib/python3.7/site-packages/dask/bytes/core.py in __enter__()
181 mode = self.mode.replace('t', '').replace('b', '') + 'b'
182
--> 183 f = SeekableFile(self.fs.open(self.path, mode=mode))
184
185 fobjects = [f]
/srv/conda/envs/notebook/lib/python3.7/site-packages/s3fs/core.py in open()
348 fdesc = S3File(self, path, mode2, block_size=block_size, acl=acl,
349 version_id=version_id, fill_cache=fill_cache,
--> 350 s3_additional_kwargs=kw)
351 if 'b' in mode:
352 return fdesc
/srv/conda/envs/notebook/lib/python3.7/site-packages/s3fs/core.py in __init__()
1204 else:
1205 try:
-> 1206 info = self.info()
1207 self.size = info['Size']
1208 if self.s3.version_aware:
/srv/conda/envs/notebook/lib/python3.7/site-packages/s3fs/core.py in info()
1222 refresh = self.s3.version_aware
1223 return self.s3.info(self.path, version_id=self.version_id,
-> 1224 refresh=refresh, **kwargs)
1225
1226 def metadata(self, refresh=False, **kwargs):
/srv/conda/envs/notebook/lib/python3.7/site-packages/s3fs/core.py in info()
522 bucket, key = split_path(path)
523 out = self._call_s3(self.s3.head_object, kwargs, Bucket=bucket,
--> 524 Key=key, **self.req_kw)
525 return {
526 'ETag': out['ETag'],
/srv/conda/envs/notebook/lib/python3.7/site-packages/s3fs/core.py in _call_s3()
193 additional_kwargs = self._get_s3_method_kwargs(method, *akwarglist,
194 **kwargs)
--> 195 return method(**additional_kwargs)
196
197 def _get_s3_method_kwargs(self, method, *akwarglist, **kwargs):
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/client.py in _api_call()
355 "%s() only accepts keyword arguments." % py_operation_name)
356 # The "self" in this scope is referring to the BaseClient.
--> 357 return self._make_api_call(operation_name, kwargs)
358
359 _api_call.__name__ = str(py_operation_name)
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/client.py in _make_api_call()
646 else:
647 http, parsed_response = self._make_request(
--> 648 operation_model, request_dict, request_context)
649
650 self.meta.events.emit(
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/client.py in _make_request()
665 def _make_request(self, operation_model, request_dict, request_context):
666 try:
--> 667 return self._endpoint.make_request(operation_model, request_dict)
668 except Exception as e:
669 self.meta.events.emit(
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/endpoint.py in make_request()
100 logger.debug("Making request for %s with params: %s",
101 operation_model, request_dict)
--> 102 return self._send_request(request_dict, operation_model)
103
104 def create_request(self, params, operation_model=None):
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/endpoint.py in _send_request()
130 def _send_request(self, request_dict, operation_model):
131 attempts = 1
--> 132 request = self.create_request(request_dict, operation_model)
133 context = request_dict['context']
134 success_response, exception = self._get_response(
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/endpoint.py in create_request()
114 op_name=operation_model.name)
115 self._event_emitter.emit(event_name, request=request,
--> 116 operation_name=operation_model.name)
117 prepared_request = self.prepare_request(request)
118 return prepared_request
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/hooks.py in emit()
354 def emit(self, event_name, **kwargs):
355 aliased_event_name = self._alias_event_name(event_name)
--> 356 return self._emitter.emit(aliased_event_name, **kwargs)
357
358 def emit_until_response(self, event_name, **kwargs):
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/hooks.py in emit()
226 handlers.
227 """
--> 228 return self._emit(event_name, kwargs)
229
230 def emit_until_response(self, event_name, **kwargs):
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/hooks.py in _emit()
209 for handler in handlers_to_call:
210 logger.debug('Event %s: calling handler %s', event_name, handler)
--> 211 response = handler(**kwargs)
212 responses.append((handler, response))
213 if stop_on_response and response is not None:
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/signers.py in handler()
88 # this method is invoked to sign the request.
89 # Don't call this method directly.
---> 90 return self.sign(operation_name, request)
91
92 def sign(self, operation_name, request, region_name=None,
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/signers.py in sign()
155 raise e
156
--> 157 auth.add_auth(request)
158
159 def _choose_signer(self, operation_name, signing_type, context):
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/auth.py in add_auth()
423 self._region_name = signing_context.get(
424 'region', self._default_region_name)
--> 425 super(S3SigV4Auth, self).add_auth(request)
426
427 def _modify_request_before_signing(self, request):
/srv/conda/envs/notebook/lib/python3.7/site-packages/botocore/auth.py in add_auth()
355 def add_auth(self, request):
356 if self.credentials is None:
--> 357 raise NoCredentialsError
358 datetime_now = datetime.datetime.utcnow()
359 request.context['timestamp'] = datetime_now.strftime(SIGV4_TIMESTAMP)
NoCredentialsError: Unable to locate credentials |
|
From the dask bag tests, looks like we may need to specify In [1]: import dask.bag as db
In [2]: b = db.read_text('s3://dask-data/nyc-taxi/2015/yellow_tripdata_2015-01.csv',
...: storage_options={'anon': True})
...: b.take(1)
Out[2]: ('VendorID,tpep_pickup_datetime,tpep_dropoff_datetime,passenger_count,trip_distance,pickup_longitude,pickup_latitude,RateCodeID,store_and_fwd_flag,dropoff_longitude,dropoff_latitude,payment_type,fare_amount,extra,mta_tax,tip_amount,tolls_amount,improvement_surcharge,total_amount\n',) |
|
I went to the binder deployment, then went to the bag notebook and started
pressing Shift-Enter. It complained that various files weren't present
…On Mon, Aug 5, 2019 at 1:02 PM Benjamin Zaitlen ***@***.***> wrote:
@mrocklin <https://github.com/mrocklin> we do call prep in the docker
setup:
https://github.com/dask/dask-tutorial/blob/da0d0310eb356258d3c52bd92e32be36eb665b15/Dockerfile#L10
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#103?email_source=notifications&email_token=AACKZTAHLOMX4QUT2GGFRTDQDCBPHA5CNFSM4IJN55E2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3S5GEY#issuecomment-518378259>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AACKZTCSC5GFLTO2WLP3TTTQDCBPHANCNFSM4IJN55EQ>
.
|
|
Looking now as well |
agree. I don't think we ever had the ability to guess that we should be anonymous, but maybe this was run at some point on ec2, where you have some default ID. |
|
@mrocklin I tried three times running Not sure why. Looking into it -- I suspect this is what happened when mybinder tried to build the image |
|
Also, if folks are planning on running tutorials with mybinder directly they should note that the default maximum is 100 concurrent sessions: I believe we can ask for more |
|
@mrocklin when you have a moment can you try again ? prep data downloading now occurs just after the session starts |
|
Let's open new issues as needed. |
|
I just went through the deployed version to prep for a tutorial tomorrow (haven't looked at our tutorial in > 1 year). Everything works great, this is really well put together, and the binder setup works slick. Thanks y'all! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
https://tutorial.dask.org/ is now live:
Let's use this issue to list what needs to be fixed/changed before announcing.
cc @mrocklin @TomAugspurger @martindurant
The text was updated successfully, but these errors were encountered: