lm heap-buffer-overflow #33
Description
`./lm -m 33 -e 2
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 2
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint :
max_words_cnt : 33
token_ids_size : 33
Warning: GPU backend is not available in CPU build. Now use cpu instead!!!
Allocating memory
for tensors : 36609236 bytes,
for c_tensors: 3194706336 bytes
for grad_tensors: 1241779004 bytes
==441277==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60c0000006bc at pc 0x561aa658e062 bp 0x7ffd49e7fdd0 sp 0x7ffd49e7fdc0
READ of size 4 at 0x60c0000006bc thread T0
#0 0x561aa658e061 in main /home/cs/project/cpp-transformer/lm.cpp:319
#1 0x7f32e8f3d082 in __libc_start_main ../csu/libc-start.c:308
#2 0x561aa658628d in _start (/home/cs/project/cpp-transformer/lm+0x1b28d)
0x60c0000006bc is located 0 bytes to the right of 124-byte region [0x60c000000640,0x60c0000006bc)
allocated by thread T0 here:
#0 0x7f32e9566587 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
#1 0x561aa6598c6a in __gnu_cxx::new_allocator::allocate(unsigned long, void const*) /usr/include/c++/9/ext/new_allocator.h:114
#2 0x561aa65978ca in std::allocator_traits<std::allocator >::allocate(std::allocator&, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:443
#3 0x561aa6595d73 in std::_Vector_base<unsigned int, std::allocator >::_M_allocate(unsigned long) /usr/include/c++/9/bits/stl_vector.h:343
#4 0x561aa6595836 in std::_Vector_base<unsigned int, std::allocator >::_M_create_storage(unsigned long) /usr/include/c++/9/bits/stl_vector.h:358
#5 0x561aa6592a38 in std::_Vector_base<unsigned int, std::allocator >::_Vector_base(unsigned long, std::allocator const&) /usr/include/c++/9/bits/stl_vector.h:302
#6 0x561aa65909cf in std::vector<unsigned int, std::allocator >::vector(std::vector<unsigned int, std::allocator > const&) /usr/include/c++/9/bits/stl_vector.h:552
#7 0x561aa662e694 in void __gnu_cxx::new_allocator<std::vector<unsigned int, std::allocator > >::construct<std::vector<unsigned int, std::allocator >, std::vector<unsigned int, std::allocator > const&>(std::vector<unsigned int, std::allocator >, std::vector<unsigned int, std::allocator > const&) /usr/include/c++/9/ext/new_allocator.h:146
#8 0x561aa662d0f5 in void std::allocator_traits<std::allocator<std::vector<unsigned int, std::allocator > > >::construct<std::vector<unsigned int, std::allocator >, std::vector<unsigned int, std::allocator > const&>(std::allocator<std::vector<unsigned int, std::allocator > >&, std::vector<unsigned int, std::allocator >, std::vector<unsigned int, std::allocator > const&) /usr/include/c++/9/bits/alloc_traits.h:483
#9 0x561aa662d531 in void std::vector<std::vector<unsigned int, std::allocator >, std::allocator<std::vector<unsigned int, std::allocator > > >::_M_realloc_insert<std::vector<unsigned int, std::allocator > const&>(__gnu_cxx::__normal_iterator<std::vector<unsigned int, std::allocator >*, std::vector<std::vector<unsigned int, std::allocator >, std::allocator<std::vector<unsigned int, std::allocator > > > >, std::vector<unsigned int, std::allocator > const&) /usr/include/c++/9/bits/vector.tcc:449
#10 0x561aa662c5b8 in std::vector<std::vector<unsigned int, std::allocator >, std::allocator<std::vector<unsigned int, std::allocator > > >::push_back(std::vector<unsigned int, std::allocator > const&) /usr/include/c++/9/bits/stl_vector.h:1195
#11 0x561aa663164d in LMDataLoader::get_token_ids(std::vector<std::vector<unsigned int, std::allocator >, std::allocator<std::vector<unsigned int, std::allocator > > >&, std::vector<std::vector<unsigned int, std::allocator >, std::allocator<std::vector<unsigned int, std::allocator > > >&, int) dataloaders/language_model/lm_dataloader.cpp:44
#12 0x561aa658a06a in load_tokens_from_file(LMDataLoader&, std::vector<std::vector<unsigned int, std::allocator >, std::allocator<std::vector<unsigned int, std::allocator > > >&, std::vector<std::vector<unsigned int, std::allocator >, std::allocator<std::vector<unsigned int, std::allocator > > >&, int&, int&, int) /home/cs/project/cpp-transformer/lm.cpp:58
#13 0x561aa658b66e in main /home/cs/project/cpp-transformer/lm.cpp:167
#14 0x7f32e8f3d082 in __libc_start_main ../csu/libc-start.c:308
SUMMARY: AddressSanitizer: heap-buffer-overflow /home/cs/project/cpp-transformer/lm.cpp:319 in main
Shadow bytes around the buggy address:
0x0c187fff8080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c187fff8090: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c187fff80a0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
0x0c187fff80b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c187fff80c0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
=>0x0c187fff80d0: 00 00 00 00 00 00 00[04]fa fa fa fa fa fa fa fa
0x0c187fff80e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04
0x0c187fff80f0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c187fff8100: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
0x0c187fff8110: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c187fff8120: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==441277==ABORTING`