-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathtest_spectrum_ecoli.log
522 lines (519 loc) · 32.3 KB
/
test_spectrum_ecoli.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
Converting from csfasta/qual to sequenceFile
Deleted hdfs://localhost/user/varuzza/ecoli/input
Color Space Fasta
10/10/16 22:49:02 INFO input.FileInputFormat: Total input paths to process : 2
10/10/16 22:49:03 INFO mapred.JobClient: Running job: job_201010161630_0036
10/10/16 22:49:04 INFO mapred.JobClient: map 0% reduce 0%
10/10/17 01:35:13 INFO mapred.JobClient: map 1% reduce 0%
10/10/17 01:35:34 INFO mapred.JobClient: map 2% reduce 0%
10/10/17 01:36:04 INFO mapred.JobClient: map 3% reduce 0%
10/10/17 01:36:31 INFO mapred.JobClient: map 4% reduce 0%
10/10/17 01:36:52 INFO mapred.JobClient: map 5% reduce 0%
10/10/17 01:37:22 INFO mapred.JobClient: map 6% reduce 0%
10/10/17 01:37:28 INFO mapred.JobClient: map 6% reduce 1%
10/10/17 01:37:55 INFO mapred.JobClient: map 6% reduce 2%
10/10/17 01:37:58 INFO mapred.JobClient: map 7% reduce 2%
10/10/17 01:38:19 INFO mapred.JobClient: map 8% reduce 2%
10/10/17 01:38:46 INFO mapred.JobClient: map 9% reduce 2%
10/10/17 01:39:10 INFO mapred.JobClient: map 10% reduce 3%
10/10/17 01:39:40 INFO mapred.JobClient: map 11% reduce 3%
10/10/17 01:40:10 INFO mapred.JobClient: map 12% reduce 3%
10/10/17 01:40:25 INFO mapred.JobClient: map 12% reduce 4%
10/10/17 01:40:37 INFO mapred.JobClient: map 13% reduce 4%
10/10/17 01:41:01 INFO mapred.JobClient: map 14% reduce 4%
10/10/17 01:41:22 INFO mapred.JobClient: map 15% reduce 4%
10/10/17 01:41:37 INFO mapred.JobClient: map 15% reduce 5%
10/10/17 01:41:53 INFO mapred.JobClient: map 16% reduce 5%
10/10/17 01:42:23 INFO mapred.JobClient: map 17% reduce 5%
10/10/17 01:42:47 INFO mapred.JobClient: map 18% reduce 5%
10/10/17 01:43:11 INFO mapred.JobClient: map 19% reduce 6%
10/10/17 01:43:38 INFO mapred.JobClient: map 20% reduce 6%
10/10/17 01:44:11 INFO mapred.JobClient: map 21% reduce 6%
10/10/17 01:44:29 INFO mapred.JobClient: map 21% reduce 7%
10/10/17 01:44:32 INFO mapred.JobClient: map 22% reduce 7%
10/10/17 01:44:50 INFO mapred.JobClient: map 23% reduce 7%
10/10/17 01:45:05 INFO mapred.JobClient: map 24% reduce 7%
10/10/17 01:45:17 INFO mapred.JobClient: map 24% reduce 8%
10/10/17 01:45:26 INFO mapred.JobClient: map 25% reduce 8%
10/10/17 01:45:47 INFO mapred.JobClient: map 26% reduce 8%
10/10/17 01:46:02 INFO mapred.JobClient: map 27% reduce 8%
10/10/17 01:46:17 INFO mapred.JobClient: map 27% reduce 9%
10/10/17 01:46:20 INFO mapred.JobClient: map 28% reduce 9%
10/10/17 01:46:38 INFO mapred.JobClient: map 29% reduce 9%
10/10/17 01:46:56 INFO mapred.JobClient: map 30% reduce 9%
10/10/17 01:47:20 INFO mapred.JobClient: map 31% reduce 10%
10/10/17 01:47:38 INFO mapred.JobClient: map 32% reduce 10%
10/10/17 01:47:56 INFO mapred.JobClient: map 33% reduce 10%
10/10/17 01:48:05 INFO mapred.JobClient: map 34% reduce 10%
10/10/17 01:48:14 INFO mapred.JobClient: map 34% reduce 11%
10/10/17 01:48:23 INFO mapred.JobClient: map 35% reduce 11%
10/10/17 01:48:41 INFO mapred.JobClient: map 36% reduce 11%
10/10/17 01:48:53 INFO mapred.JobClient: map 36% reduce 12%
10/10/17 01:48:59 INFO mapred.JobClient: map 37% reduce 12%
10/10/17 01:49:14 INFO mapred.JobClient: map 38% reduce 12%
10/10/17 01:49:29 INFO mapred.JobClient: map 39% reduce 12%
10/10/17 01:49:41 INFO mapred.JobClient: map 39% reduce 13%
10/10/17 01:49:47 INFO mapred.JobClient: map 40% reduce 13%
10/10/17 01:50:05 INFO mapred.JobClient: map 41% reduce 13%
10/10/17 01:50:23 INFO mapred.JobClient: map 42% reduce 13%
10/10/17 01:50:35 INFO mapred.JobClient: map 42% reduce 14%
10/10/17 01:50:44 INFO mapred.JobClient: map 43% reduce 14%
10/10/17 01:51:05 INFO mapred.JobClient: map 44% reduce 14%
10/10/17 01:51:23 INFO mapred.JobClient: map 45% reduce 14%
10/10/17 01:51:35 INFO mapred.JobClient: map 45% reduce 15%
10/10/17 01:51:38 INFO mapred.JobClient: map 46% reduce 15%
10/10/17 01:51:56 INFO mapred.JobClient: map 47% reduce 15%
10/10/17 01:52:11 INFO mapred.JobClient: map 48% reduce 15%
10/10/17 01:52:30 INFO mapred.JobClient: map 49% reduce 16%
10/10/17 01:52:48 INFO mapred.JobClient: map 50% reduce 16%
10/10/17 01:53:06 INFO mapred.JobClient: map 51% reduce 16%
10/10/17 01:53:21 INFO mapred.JobClient: map 52% reduce 17%
10/10/17 01:53:39 INFO mapred.JobClient: map 53% reduce 17%
10/10/17 01:53:57 INFO mapred.JobClient: map 54% reduce 17%
10/10/17 01:54:15 INFO mapred.JobClient: map 55% reduce 17%
10/10/17 01:54:21 INFO mapred.JobClient: map 55% reduce 18%
10/10/17 01:54:30 INFO mapred.JobClient: map 56% reduce 18%
10/10/17 01:54:45 INFO mapred.JobClient: map 57% reduce 18%
10/10/17 01:54:57 INFO mapred.JobClient: map 57% reduce 19%
10/10/17 01:55:03 INFO mapred.JobClient: map 58% reduce 19%
10/10/17 01:55:18 INFO mapred.JobClient: map 59% reduce 19%
10/10/17 01:55:36 INFO mapred.JobClient: map 60% reduce 19%
10/10/17 01:55:51 INFO mapred.JobClient: map 60% reduce 20%
10/10/17 01:55:54 INFO mapred.JobClient: map 61% reduce 20%
10/10/17 01:56:09 INFO mapred.JobClient: map 62% reduce 20%
10/10/17 01:56:27 INFO mapred.JobClient: map 63% reduce 20%
10/10/17 01:56:48 INFO mapred.JobClient: map 63% reduce 21%
10/10/17 01:56:54 INFO mapred.JobClient: map 64% reduce 21%
10/10/17 01:57:12 INFO mapred.JobClient: map 65% reduce 21%
10/10/17 01:57:30 INFO mapred.JobClient: map 66% reduce 21%
10/10/17 01:57:48 INFO mapred.JobClient: map 67% reduce 21%
10/10/17 01:57:51 INFO mapred.JobClient: map 67% reduce 22%
10/10/17 01:58:06 INFO mapred.JobClient: map 68% reduce 22%
10/10/17 01:58:24 INFO mapred.JobClient: map 69% reduce 22%
10/10/17 01:58:36 INFO mapred.JobClient: map 69% reduce 23%
10/10/17 01:58:39 INFO mapred.JobClient: map 70% reduce 23%
10/10/17 01:58:57 INFO mapred.JobClient: map 71% reduce 23%
10/10/17 01:59:12 INFO mapred.JobClient: map 72% reduce 23%
10/10/17 01:59:30 INFO mapred.JobClient: map 73% reduce 23%
10/10/17 01:59:33 INFO mapred.JobClient: map 73% reduce 24%
10/10/17 01:59:48 INFO mapred.JobClient: map 74% reduce 24%
10/10/17 02:00:06 INFO mapred.JobClient: map 75% reduce 24%
10/10/17 02:00:24 INFO mapred.JobClient: map 76% reduce 24%
10/10/17 02:00:27 INFO mapred.JobClient: map 76% reduce 25%
10/10/17 02:00:39 INFO mapred.JobClient: map 77% reduce 25%
10/10/17 02:00:51 INFO mapred.JobClient: map 78% reduce 25%
10/10/17 02:01:03 INFO mapred.JobClient: map 78% reduce 26%
10/10/17 02:01:06 INFO mapred.JobClient: map 79% reduce 26%
10/10/17 02:01:25 INFO mapred.JobClient: map 80% reduce 26%
10/10/17 02:01:40 INFO mapred.JobClient: map 81% reduce 26%
10/10/17 02:01:52 INFO mapred.JobClient: map 81% reduce 27%
10/10/17 02:02:07 INFO mapred.JobClient: map 82% reduce 27%
10/10/17 02:02:31 INFO mapred.JobClient: map 83% reduce 27%
10/10/17 02:02:49 INFO mapred.JobClient: map 84% reduce 27%
10/10/17 02:03:04 INFO mapred.JobClient: map 84% reduce 28%
10/10/17 02:03:07 INFO mapred.JobClient: map 85% reduce 28%
10/10/17 02:03:26 INFO mapred.JobClient: map 86% reduce 28%
10/10/17 02:03:44 INFO mapred.JobClient: map 87% reduce 28%
10/10/17 02:03:59 INFO mapred.JobClient: map 87% reduce 29%
10/10/17 02:04:02 INFO mapred.JobClient: map 88% reduce 29%
10/10/17 02:04:17 INFO mapred.JobClient: map 89% reduce 29%
10/10/17 02:04:32 INFO mapred.JobClient: map 90% reduce 29%
10/10/17 02:04:47 INFO mapred.JobClient: map 90% reduce 30%
10/10/17 02:04:50 INFO mapred.JobClient: map 91% reduce 30%
10/10/17 02:05:08 INFO mapred.JobClient: map 92% reduce 30%
10/10/17 02:05:23 INFO mapred.JobClient: map 93% reduce 30%
10/10/17 02:05:41 INFO mapred.JobClient: map 94% reduce 31%
10/10/17 02:05:56 INFO mapred.JobClient: map 95% reduce 31%
10/10/17 02:06:14 INFO mapred.JobClient: map 96% reduce 31%
10/10/17 02:06:32 INFO mapred.JobClient: map 97% reduce 32%
10/10/17 02:06:50 INFO mapred.JobClient: map 98% reduce 32%
10/10/17 02:07:08 INFO mapred.JobClient: map 99% reduce 32%
10/10/17 02:07:14 INFO mapred.JobClient: map 100% reduce 32%
10/10/17 02:07:20 INFO mapred.JobClient: map 100% reduce 33%
10/10/17 02:07:56 INFO mapred.JobClient: map 100% reduce 66%
10/10/17 02:08:02 INFO mapred.JobClient: map 100% reduce 67%
10/10/17 02:08:14 INFO mapred.JobClient: map 100% reduce 68%
10/10/17 02:08:20 INFO mapred.JobClient: map 100% reduce 69%
10/10/17 02:08:26 INFO mapred.JobClient: map 100% reduce 70%
10/10/17 02:08:38 INFO mapred.JobClient: map 100% reduce 71%
10/10/17 02:08:44 INFO mapred.JobClient: map 100% reduce 72%
10/10/17 02:08:53 INFO mapred.JobClient: map 100% reduce 73%
10/10/17 02:09:02 INFO mapred.JobClient: map 100% reduce 74%
10/10/17 02:09:11 INFO mapred.JobClient: map 100% reduce 75%
10/10/17 02:09:17 INFO mapred.JobClient: map 100% reduce 76%
10/10/17 02:09:26 INFO mapred.JobClient: map 100% reduce 77%
10/10/17 02:09:35 INFO mapred.JobClient: map 100% reduce 78%
10/10/17 02:09:41 INFO mapred.JobClient: map 100% reduce 79%
10/10/17 02:09:50 INFO mapred.JobClient: map 100% reduce 80%
10/10/17 02:09:59 INFO mapred.JobClient: map 100% reduce 81%
10/10/17 02:10:05 INFO mapred.JobClient: map 100% reduce 82%
10/10/17 02:10:14 INFO mapred.JobClient: map 100% reduce 83%
10/10/17 02:10:23 INFO mapred.JobClient: map 100% reduce 84%
10/10/17 02:10:29 INFO mapred.JobClient: map 100% reduce 85%
10/10/17 02:10:41 INFO mapred.JobClient: map 100% reduce 86%
10/10/17 02:10:47 INFO mapred.JobClient: map 100% reduce 87%
10/10/17 02:10:56 INFO mapred.JobClient: map 100% reduce 88%
10/10/17 02:11:05 INFO mapred.JobClient: map 100% reduce 89%
10/10/17 02:11:11 INFO mapred.JobClient: map 100% reduce 90%
10/10/17 02:11:20 INFO mapred.JobClient: map 100% reduce 91%
10/10/17 02:11:32 INFO mapred.JobClient: map 100% reduce 92%
10/10/17 02:11:41 INFO mapred.JobClient: map 100% reduce 93%
10/10/17 02:11:50 INFO mapred.JobClient: map 100% reduce 94%
10/10/17 02:11:59 INFO mapred.JobClient: map 100% reduce 95%
10/10/17 02:12:08 INFO mapred.JobClient: map 100% reduce 96%
10/10/17 02:12:17 INFO mapred.JobClient: map 100% reduce 97%
10/10/17 02:12:26 INFO mapred.JobClient: map 100% reduce 98%
10/10/17 02:12:35 INFO mapred.JobClient: map 100% reduce 99%
10/10/17 02:12:44 INFO mapred.JobClient: map 100% reduce 100%
10/10/17 02:12:49 INFO mapred.JobClient: Job complete: job_201010161630_0036
10/10/17 02:12:50 INFO mapred.JobClient: Counters: 16
10/10/17 02:12:50 INFO mapred.JobClient: Job Counters
10/10/17 02:12:50 INFO mapred.JobClient: Launched reduce tasks=1
10/10/17 02:12:50 INFO mapred.JobClient: Rack-local map tasks=191
10/10/17 02:12:50 INFO mapred.JobClient: Launched map tasks=191
10/10/17 02:12:50 INFO mapred.JobClient: FileSystemCounters
10/10/17 02:12:50 INFO mapred.JobClient: FILE_BYTES_READ=19756685799
10/10/17 02:12:50 INFO mapred.JobClient: FILE_BYTES_WRITTEN=18809208732
10/10/17 02:12:50 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=4407049604
10/10/17 02:12:50 INFO mapred.JobClient: Map-Reduce Framework
10/10/17 02:12:50 INFO mapred.JobClient: Reduce input groups=28941110
10/10/17 02:12:50 INFO mapred.JobClient: Combine output records=0
10/10/17 02:12:50 INFO mapred.JobClient: Map input records=57882220
10/10/17 02:12:50 INFO mapred.JobClient: Reduce shuffle bytes=5411645722
10/10/17 02:12:50 INFO mapred.JobClient: Reduce output records=28941110
10/10/17 02:12:50 INFO mapred.JobClient: Spilled Records=200426886
10/10/17 02:12:50 INFO mapred.JobClient: Map output bytes=5316359160
10/10/17 02:12:50 INFO mapred.JobClient: Combine input records=0
10/10/17 02:12:50 INFO mapred.JobClient: Map output records=57882220
10/10/17 02:12:50 INFO mapred.JobClient: Reduce input records=57882220
11.47user 2.60system 2:23:52elapsed 0%CPU (0avgtext+0avgdata 323808maxresident)k
2512inputs+9416outputs (25major+43130minor)pagefaults 0swaps
Converting from csfasta/qual to sequenceFile
Deleted hdfs://localhost/user/varuzza/spectrum/build
10/10/17 02:37:48 INFO input.FileInputFormat: Total input paths to process : 1
10/10/17 02:37:48 INFO mapred.JobClient: Running job: job_201010161630_0037
10/10/17 02:37:49 INFO mapred.JobClient: map 0% reduce 0%
10/10/17 02:38:38 INFO mapred.JobClient: map 1% reduce 0%
10/10/17 02:39:17 INFO mapred.JobClient: map 2% reduce 0%
10/10/17 02:40:05 INFO mapred.JobClient: map 3% reduce 0%
10/10/17 02:42:59 INFO mapred.JobClient: map 4% reduce 0%
10/10/17 02:43:32 INFO mapred.JobClient: map 5% reduce 0%
10/10/17 02:44:08 INFO mapred.JobClient: map 6% reduce 0%
10/10/17 02:47:00 INFO mapred.JobClient: map 6% reduce 1%
10/10/17 02:47:36 INFO mapred.JobClient: map 7% reduce 1%
10/10/17 02:47:48 INFO mapred.JobClient: map 7% reduce 2%
10/10/17 02:48:18 INFO mapred.JobClient: map 8% reduce 2%
10/10/17 02:48:54 INFO mapred.JobClient: map 9% reduce 2%
10/10/17 02:51:21 INFO mapred.JobClient: map 9% reduce 3%
10/10/17 02:51:45 INFO mapred.JobClient: map 10% reduce 3%
10/10/17 02:52:24 INFO mapred.JobClient: map 11% reduce 3%
10/10/17 02:53:03 INFO mapred.JobClient: map 12% reduce 3%
10/10/17 02:56:19 INFO mapred.JobClient: map 12% reduce 4%
10/10/17 02:56:37 INFO mapred.JobClient: map 13% reduce 4%
10/10/17 02:57:10 INFO mapred.JobClient: map 14% reduce 4%
10/10/17 02:57:46 INFO mapred.JobClient: map 15% reduce 4%
10/10/17 03:00:47 INFO mapred.JobClient: map 15% reduce 5%
10/10/17 03:01:02 INFO mapred.JobClient: map 16% reduce 5%
10/10/17 03:01:35 INFO mapred.JobClient: map 17% reduce 5%
10/10/17 03:02:11 INFO mapred.JobClient: map 18% reduce 5%
10/10/17 03:05:17 INFO mapred.JobClient: map 18% reduce 6%
10/10/17 03:05:32 INFO mapred.JobClient: map 19% reduce 6%
10/10/17 03:06:08 INFO mapred.JobClient: map 20% reduce 6%
10/10/17 03:06:41 INFO mapred.JobClient: map 21% reduce 6%
10/10/17 03:09:51 INFO mapred.JobClient: map 21% reduce 7%
10/10/17 03:10:06 INFO mapred.JobClient: map 22% reduce 7%
10/10/17 03:10:39 INFO mapred.JobClient: map 23% reduce 7%
10/10/17 03:11:15 INFO mapred.JobClient: map 24% reduce 7%
10/10/17 03:14:18 INFO mapred.JobClient: map 24% reduce 8%
10/10/17 03:14:33 INFO mapred.JobClient: map 25% reduce 8%
10/10/17 03:15:06 INFO mapred.JobClient: map 26% reduce 8%
10/10/17 03:15:39 INFO mapred.JobClient: map 27% reduce 8%
10/10/17 03:18:40 INFO mapred.JobClient: map 27% reduce 9%
10/10/17 03:18:49 INFO mapred.JobClient: map 28% reduce 9%
10/10/17 03:19:28 INFO mapred.JobClient: map 29% reduce 9%
10/10/17 03:20:10 INFO mapred.JobClient: map 30% reduce 9%
10/10/17 03:23:25 INFO mapred.JobClient: map 30% reduce 10%
10/10/17 03:23:34 INFO mapred.JobClient: map 31% reduce 10%
10/10/17 03:24:31 INFO mapred.JobClient: map 32% reduce 10%
10/10/17 03:25:25 INFO mapred.JobClient: map 33% reduce 10%
10/10/17 03:33:36 INFO mapred.JobClient: map 34% reduce 10%
10/10/17 03:33:45 INFO mapred.JobClient: map 34% reduce 11%
10/10/17 03:34:43 INFO mapred.JobClient: map 35% reduce 11%
10/10/17 03:35:34 INFO mapred.JobClient: map 36% reduce 11%
10/10/17 03:42:20 INFO mapred.JobClient: map 36% reduce 12%
10/10/17 03:42:26 INFO mapred.JobClient: map 37% reduce 12%
10/10/17 03:43:02 INFO mapred.JobClient: map 38% reduce 12%
10/10/17 03:43:35 INFO mapred.JobClient: map 39% reduce 12%
10/10/17 03:46:35 INFO mapred.JobClient: map 39% reduce 13%
10/10/17 03:46:50 INFO mapred.JobClient: map 40% reduce 13%
10/10/17 03:47:23 INFO mapred.JobClient: map 41% reduce 13%
10/10/17 03:47:56 INFO mapred.JobClient: map 42% reduce 13%
10/10/17 03:51:05 INFO mapred.JobClient: map 42% reduce 14%
10/10/17 03:51:23 INFO mapred.JobClient: map 43% reduce 14%
10/10/17 03:52:32 INFO mapred.JobClient: map 44% reduce 14%
10/10/17 03:53:27 INFO mapred.JobClient: map 45% reduce 14%
10/10/17 04:01:44 INFO mapred.JobClient: map 46% reduce 14%
10/10/17 04:01:50 INFO mapred.JobClient: map 46% reduce 15%
10/10/17 04:02:44 INFO mapred.JobClient: map 47% reduce 15%
10/10/17 04:03:47 INFO mapred.JobClient: map 48% reduce 15%
10/10/17 04:10:33 INFO mapred.JobClient: map 48% reduce 16%
10/10/17 04:10:39 INFO mapred.JobClient: map 49% reduce 16%
10/10/17 04:11:15 INFO mapred.JobClient: map 50% reduce 16%
10/10/17 04:11:45 INFO mapred.JobClient: map 51% reduce 16%
10/10/17 04:14:48 INFO mapred.JobClient: map 51% reduce 17%
10/10/17 04:14:54 INFO mapred.JobClient: map 52% reduce 17%
10/10/17 04:15:30 INFO mapred.JobClient: map 53% reduce 17%
10/10/17 04:16:03 INFO mapred.JobClient: map 54% reduce 17%
10/10/17 04:19:06 INFO mapred.JobClient: map 54% reduce 18%
10/10/17 04:19:12 INFO mapred.JobClient: map 55% reduce 18%
10/10/17 04:19:48 INFO mapred.JobClient: map 56% reduce 18%
10/10/17 04:20:24 INFO mapred.JobClient: map 57% reduce 18%
10/10/17 04:23:34 INFO mapred.JobClient: map 58% reduce 18%
10/10/17 04:24:01 INFO mapred.JobClient: map 58% reduce 19%
10/10/17 04:24:31 INFO mapred.JobClient: map 59% reduce 19%
10/10/17 04:25:28 INFO mapred.JobClient: map 60% reduce 19%
10/10/17 04:33:48 INFO mapred.JobClient: map 61% reduce 19%
10/10/17 04:34:27 INFO mapred.JobClient: map 61% reduce 20%
10/10/17 04:34:48 INFO mapred.JobClient: map 62% reduce 20%
10/10/17 04:35:45 INFO mapred.JobClient: map 63% reduce 20%
10/10/17 04:42:57 INFO mapred.JobClient: map 64% reduce 20%
10/10/17 04:43:30 INFO mapred.JobClient: map 64% reduce 21%
10/10/17 04:43:51 INFO mapred.JobClient: map 65% reduce 21%
10/10/17 04:44:25 INFO mapred.JobClient: map 66% reduce 21%
10/10/17 04:47:01 INFO mapred.JobClient: map 67% reduce 21%
10/10/17 04:47:43 INFO mapred.JobClient: map 67% reduce 22%
10/10/17 04:47:58 INFO mapred.JobClient: map 68% reduce 22%
10/10/17 04:48:31 INFO mapred.JobClient: map 69% reduce 22%
10/10/17 04:51:01 INFO mapred.JobClient: map 70% reduce 22%
10/10/17 04:51:52 INFO mapred.JobClient: map 70% reduce 23%
10/10/17 04:52:04 INFO mapred.JobClient: map 71% reduce 23%
10/10/17 04:52:49 INFO mapred.JobClient: map 72% reduce 23%
10/10/17 05:00:12 INFO mapred.JobClient: map 73% reduce 23%
10/10/17 05:01:57 INFO mapred.JobClient: map 74% reduce 23%
10/10/17 05:02:15 INFO mapred.JobClient: map 74% reduce 24%
10/10/17 05:02:54 INFO mapred.JobClient: map 75% reduce 24%
10/10/17 05:10:11 INFO mapred.JobClient: map 76% reduce 24%
10/10/17 05:11:44 INFO mapred.JobClient: map 77% reduce 24%
10/10/17 05:12:02 INFO mapred.JobClient: map 77% reduce 25%
10/10/17 05:12:44 INFO mapred.JobClient: map 78% reduce 25%
10/10/17 05:15:08 INFO mapred.JobClient: map 79% reduce 25%
10/10/17 05:16:05 INFO mapred.JobClient: map 79% reduce 26%
10/10/17 05:16:11 INFO mapred.JobClient: map 80% reduce 26%
10/10/17 05:16:44 INFO mapred.JobClient: map 81% reduce 26%
10/10/17 05:19:12 INFO mapred.JobClient: map 82% reduce 26%
10/10/17 05:19:57 INFO mapred.JobClient: map 82% reduce 27%
10/10/17 05:20:12 INFO mapred.JobClient: map 83% reduce 27%
10/10/17 05:20:45 INFO mapred.JobClient: map 84% reduce 27%
10/10/17 05:23:13 INFO mapred.JobClient: map 85% reduce 27%
10/10/17 05:24:28 INFO mapred.JobClient: map 86% reduce 27%
10/10/17 05:26:16 INFO mapred.JobClient: map 86% reduce 28%
10/10/17 05:26:19 INFO mapred.JobClient: map 87% reduce 28%
10/10/17 05:32:41 INFO mapred.JobClient: map 88% reduce 28%
10/10/17 05:34:30 INFO mapred.JobClient: map 89% reduce 28%
10/10/17 05:35:16 INFO mapred.JobClient: map 89% reduce 29%
10/10/17 05:36:07 INFO mapred.JobClient: map 90% reduce 29%
10/10/17 05:36:52 INFO mapred.JobClient: map 88% reduce 29%
10/10/17 05:36:54 INFO mapred.JobClient: Task Id : attempt_201010161630_0037_m_000058_0, Status : FAILED
FSError: java.io.IOException: No space left on device
10/10/17 05:37:07 INFO mapred.JobClient: map 89% reduce 29%
10/10/17 05:39:34 INFO mapred.JobClient: map 87% reduce 29%
10/10/17 05:39:36 INFO mapred.JobClient: Task Id : attempt_201010161630_0037_m_000059_0, Status : FAILED
Error: java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:260)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:190)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:61)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:86)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1298)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$1800(MapTask.java:686)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$SpillThread.run(MapTask.java:1173)
10/10/17 05:39:58 INFO mapred.JobClient: map 88% reduce 29%
10/10/17 05:40:24 INFO mapred.JobClient: Task Id : attempt_201010161630_0037_m_000059_1, Status : FAILED
java.io.IOException: Spill failed
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:860)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at com.lifetech.hadoop.bioseq.spectrum.Build$KmerBuilder.map(Build.java:37)
at com.lifetech.hadoop.bioseq.spectrum.Build$KmerBuilder.map(Build.java:1)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local directory for taskTracker/jobcache/job_201010161630_0037/attempt_201010161630_0037_m_000059_1/output/spill14.out
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:343)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
at org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:107)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1221)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$1800(MapTask.java:686)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$SpillThread.run(MapTask.java:1173)
10/10/17 05:40:45 INFO mapred.JobClient: Task Id : attempt_201010161630_0037_m_000059_2, Status : FAILED
java.io.IOException: Spill failed
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:860)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at com.lifetech.hadoop.bioseq.spectrum.Build$KmerBuilder.map(Build.java:37)
at com.lifetech.hadoop.bioseq.spectrum.Build$KmerBuilder.map(Build.java:1)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.io.FileNotFoundException: /data/hadoop-0.20/cache/hadoop/tmp/hadoop-hadoop/mapred/local/taskTracker/jobcache/job_201010161630_0037/attempt_201010161630_0037_m_000059_2/output/spill8.out (No space left on device)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:180)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:176)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:234)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:465)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:372)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:364)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1223)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$1800(MapTask.java:686)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$SpillThread.run(MapTask.java:1173)
10/10/17 05:42:19 INFO mapred.JobClient: map 88% reduce 0%
10/10/17 05:42:44 INFO mapred.JobClient: Job complete: job_201010161630_0037
10/10/17 05:42:45 INFO mapred.JobClient: Counters: 13
10/10/17 05:42:45 INFO mapred.JobClient: Job Counters
10/10/17 05:42:45 INFO mapred.JobClient: Launched reduce tasks=1
10/10/17 05:42:45 INFO mapred.JobClient: Launched map tasks=64
10/10/17 05:42:45 INFO mapred.JobClient: Data-local map tasks=64
10/10/17 05:42:45 INFO mapred.JobClient: Failed map tasks=1
10/10/17 05:42:45 INFO mapred.JobClient: FileSystemCounters
10/10/17 05:42:45 INFO mapred.JobClient: FILE_BYTES_READ=72204794903
10/10/17 05:42:45 INFO mapred.JobClient: HDFS_BYTES_READ=3892394212
10/10/17 05:42:45 INFO mapred.JobClient: FILE_BYTES_WRITTEN=106385386266
10/10/17 05:42:45 INFO mapred.JobClient: Map-Reduce Framework
10/10/17 05:42:45 INFO mapred.JobClient: Combine output records=1633014435
10/10/17 05:42:45 INFO mapred.JobClient: Map input records=25562935
10/10/17 05:42:45 INFO mapred.JobClient: Spilled Records=2461880098
10/10/17 05:42:45 INFO mapred.JobClient: Map output bytes=35714487410
10/10/17 05:42:45 INFO mapred.JobClient: Combine input records=1784120706
10/10/17 05:42:45 INFO mapred.JobClient: Map output records=894702725
Command exited with non-zero status 1
15.95user 4.73system 3:04:59elapsed 0%CPU (0avgtext+0avgdata 294880maxresident)k
9384inputs+52320outputs (48major+49217minor)pagefaults 0swaps
Converting from csfasta/qual to sequenceFile
Deleted hdfs://localhost/user/varuzza/spectrum/build
10/10/17 10:05:52 INFO input.FileInputFormat: Total input paths to process : 1
10/10/17 10:05:53 INFO mapred.JobClient: Running job: job_201010161630_0038
10/10/17 10:05:54 INFO mapred.JobClient: map 0% reduce 0%
10/10/17 10:06:48 INFO mapred.JobClient: map 1% reduce 0%
10/10/17 10:07:27 INFO mapred.JobClient: map 2% reduce 0%
10/10/17 10:08:03 INFO mapred.JobClient: map 3% reduce 0%
10/10/17 10:11:18 INFO mapred.JobClient: map 4% reduce 0%
10/10/17 10:11:54 INFO mapred.JobClient: map 5% reduce 0%
10/10/17 10:12:30 INFO mapred.JobClient: map 6% reduce 0%
10/10/17 10:15:46 INFO mapred.JobClient: map 6% reduce 1%
10/10/17 10:16:10 INFO mapred.JobClient: map 7% reduce 1%
10/10/17 10:17:01 INFO mapred.JobClient: map 7% reduce 2%
10/10/17 10:17:16 INFO mapred.JobClient: map 8% reduce 2%
10/10/17 10:17:55 INFO mapred.JobClient: map 9% reduce 2%
10/10/17 10:20:59 INFO mapred.JobClient: map 9% reduce 3%
10/10/17 10:21:11 INFO mapred.JobClient: map 10% reduce 3%
10/10/17 10:21:47 INFO mapred.JobClient: map 11% reduce 3%
10/10/17 10:22:23 INFO mapred.JobClient: map 12% reduce 3%
10/10/17 10:25:50 INFO mapred.JobClient: map 12% reduce 4%
10/10/17 10:26:08 INFO mapred.JobClient: map 13% reduce 4%
10/10/17 10:26:47 INFO mapred.JobClient: map 14% reduce 4%
10/10/17 10:27:23 INFO mapred.JobClient: map 15% reduce 4%
10/10/17 10:30:42 INFO mapred.JobClient: map 15% reduce 5%
10/10/17 10:30:54 INFO mapred.JobClient: map 16% reduce 5%
10/10/17 10:31:30 INFO mapred.JobClient: map 17% reduce 5%
10/10/17 10:32:03 INFO mapred.JobClient: map 18% reduce 5%
10/10/17 10:35:15 INFO mapred.JobClient: map 18% reduce 6%
10/10/17 10:35:27 INFO mapred.JobClient: map 19% reduce 6%
10/10/17 10:36:03 INFO mapred.JobClient: map 20% reduce 6%
10/10/17 10:36:36 INFO mapred.JobClient: map 21% reduce 6%
10/10/17 10:39:52 INFO mapred.JobClient: map 21% reduce 7%
10/10/17 10:40:04 INFO mapred.JobClient: map 22% reduce 7%
10/10/17 10:40:37 INFO mapred.JobClient: map 23% reduce 7%
10/10/17 10:41:10 INFO mapred.JobClient: map 24% reduce 7%
10/10/17 10:44:19 INFO mapred.JobClient: map 24% reduce 8%
10/10/17 10:44:31 INFO mapred.JobClient: map 25% reduce 8%
10/10/17 10:45:07 INFO mapred.JobClient: map 26% reduce 8%
10/10/17 10:45:40 INFO mapred.JobClient: map 27% reduce 8%
10/10/17 10:48:46 INFO mapred.JobClient: map 27% reduce 9%
10/10/17 10:48:56 INFO mapred.JobClient: map 28% reduce 9%
10/10/17 10:49:29 INFO mapred.JobClient: map 29% reduce 9%
10/10/17 10:50:02 INFO mapred.JobClient: map 30% reduce 9%
10/10/17 10:53:15 INFO mapred.JobClient: map 30% reduce 10%
10/10/17 10:53:32 INFO mapred.JobClient: map 31% reduce 10%
10/10/17 10:54:29 INFO mapred.JobClient: map 32% reduce 10%
10/10/17 10:55:23 INFO mapred.JobClient: map 33% reduce 10%
10/10/17 11:03:56 INFO mapred.JobClient: map 33% reduce 11%
10/10/17 11:04:04 INFO mapred.JobClient: map 34% reduce 11%
10/10/17 11:05:04 INFO mapred.JobClient: map 35% reduce 11%
10/10/17 11:05:56 INFO mapred.JobClient: map 36% reduce 11%
10/10/17 11:11:51 INFO mapred.JobClient: map 36% reduce 12%
10/10/17 11:11:57 INFO mapred.JobClient: map 37% reduce 12%
10/10/17 11:12:30 INFO mapred.JobClient: map 38% reduce 12%
10/10/17 11:13:00 INFO mapred.JobClient: map 39% reduce 12%
10/10/17 11:16:12 INFO mapred.JobClient: map 39% reduce 13%
10/10/17 11:16:18 INFO mapred.JobClient: map 40% reduce 13%
10/10/17 11:16:51 INFO mapred.JobClient: map 41% reduce 13%
10/10/17 11:17:28 INFO mapred.JobClient: map 42% reduce 13%
10/10/17 11:20:37 INFO mapred.JobClient: map 42% reduce 14%
10/10/17 11:20:49 INFO mapred.JobClient: map 43% reduce 14%
10/10/17 11:21:37 INFO mapred.JobClient: map 44% reduce 14%
10/10/17 11:22:25 INFO mapred.JobClient: map 45% reduce 14%
10/10/17 11:32:14 INFO mapred.JobClient: map 46% reduce 14%
10/10/17 11:32:29 INFO mapred.JobClient: map 46% reduce 15%
10/10/17 11:33:14 INFO mapred.JobClient: map 47% reduce 15%
10/10/17 11:34:11 INFO mapred.JobClient: map 48% reduce 15%
10/10/17 11:41:45 INFO mapred.JobClient: map 49% reduce 15%
10/10/17 11:41:51 INFO mapred.JobClient: map 49% reduce 16%
10/10/17 11:42:21 INFO mapred.JobClient: map 50% reduce 16%
10/10/17 11:42:54 INFO mapred.JobClient: map 51% reduce 16%
10/10/17 11:45:54 INFO mapred.JobClient: map 52% reduce 16%
10/10/17 11:45:57 INFO mapred.JobClient: map 52% reduce 17%
10/10/17 11:46:30 INFO mapred.JobClient: map 53% reduce 17%
10/10/17 11:47:03 INFO mapred.JobClient: map 54% reduce 17%
10/10/17 11:50:22 INFO mapred.JobClient: map 55% reduce 18%
10/10/17 11:50:55 INFO mapred.JobClient: map 56% reduce 18%
10/10/17 11:51:32 INFO mapred.JobClient: map 57% reduce 18%
10/10/17 11:54:55 INFO mapred.JobClient: map 58% reduce 18%
10/10/17 11:55:22 INFO mapred.JobClient: map 58% reduce 19%
10/10/17 11:56:04 INFO mapred.JobClient: map 59% reduce 19%
10/10/17 11:56:59 INFO mapred.JobClient: map 60% reduce 19%
10/10/17 12:05:32 INFO mapred.JobClient: map 61% reduce 19%
10/10/17 12:05:50 INFO mapred.JobClient: map 61% reduce 20%
10/10/17 12:06:29 INFO mapred.JobClient: map 62% reduce 20%
10/10/17 12:07:26 INFO mapred.JobClient: map 63% reduce 20%
10/10/17 12:14:54 INFO mapred.JobClient: map 64% reduce 20%
10/10/17 12:15:09 INFO mapred.JobClient: map 64% reduce 21%
10/10/17 12:15:39 INFO mapred.JobClient: map 65% reduce 21%
10/10/17 12:16:21 INFO mapred.JobClient: map 66% reduce 21%
10/10/17 12:20:01 INFO mapred.JobClient: map 66% reduce 22%
10/10/17 12:20:07 INFO mapred.JobClient: map 67% reduce 22%
10/10/17 12:20:45 INFO mapred.JobClient: map 68% reduce 22%
10/10/17 12:21:24 INFO mapred.JobClient: map 69% reduce 22%
10/10/17 12:24:50 INFO mapred.JobClient: map 69% reduce 23%
10/10/17 12:24:53 INFO mapred.JobClient: map 70% reduce 23%
10/10/17 12:25:35 INFO mapred.JobClient: map 71% reduce 23%
10/10/17 12:26:23 INFO mapred.JobClient: map 72% reduce 23%
10/10/17 12:32:50 INFO mapred.JobClient: map 73% reduce 23%
10/10/17 12:33:41 INFO mapred.JobClient: map 73% reduce 24%
10/10/17 12:33:53 INFO mapred.JobClient: map 74% reduce 24%
10/10/17 12:34:35 INFO mapred.JobClient: map 75% reduce 24%
10/10/17 12:40:49 INFO mapred.JobClient: map 76% reduce 24%
10/10/17 12:40:58 INFO mapred.JobClient: map 76% reduce 25%
10/10/17 12:41:31 INFO mapred.JobClient: map 77% reduce 25%
10/10/17 12:42:10 INFO mapred.JobClient: map 78% reduce 25%
10/10/17 12:45:22 INFO mapred.JobClient: map 79% reduce 25%
10/10/17 12:45:34 INFO mapred.JobClient: map 79% reduce 26%
10/10/17 12:46:07 INFO mapred.JobClient: map 80% reduce 26%
10/10/17 12:46:40 INFO mapred.JobClient: map 81% reduce 26%
10/10/17 12:49:54 INFO mapred.JobClient: map 82% reduce 26%
10/10/17 12:50:21 INFO mapred.JobClient: map 82% reduce 27%
10/10/17 12:50:48 INFO mapred.JobClient: map 83% reduce 27%
10/10/17 12:51:27 INFO mapred.JobClient: map 84% reduce 27%
10/10/17 12:54:42 INFO mapred.JobClient: map 85% reduce 27%
10/10/17 12:56:15 INFO mapred.JobClient: map 86% reduce 27%
10/10/17 12:56:48 INFO mapred.JobClient: map 86% reduce 28%
10/10/17 12:57:12 INFO mapred.JobClient: map 87% reduce 28%
10/10/17 13:05:28 INFO mapred.JobClient: map 88% reduce 28%
Command exited with non-zero status 130
15.60user 4.85system 3:00:01elapsed 0%CPU (0avgtext+0avgdata 307344maxresident)k
10608inputs+56008outputs (42major+49860minor)pagefaults 0swaps