Adam,
I still can't get past this error:
Program received signal SIGBUS, Bus error.
0x00007ffff05973cb in __memcpy_ssse3_back () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-157.el7.x86_64
libgcc-4.8.5-11.el7.x86_64 libgfortran-4.8.5-11.el7.x86_64
libquadmath-4.8.5-11.el7.x86_64 mpich-3.2-3.2-2.el7.x86_64
numactl-libs-2.0.9-6.el7_2.x86_64 openssl-libs-1.0.1e-60.el7.x86_64
zlib-1.2.7-17.el7.x86_64
(gdb) bt
#0 0x00007ffff05973cb in __memcpy_ssse3_back () from /lib64/libc.so.6
#1 0x0000000000409926 in unifycr_logio_chunk_write (fid=1, pos=49152, meta=0x7fffddcf443c, chunk_id=0, chunk_offset=49152, buf=0x66280c0, count=4096)
at unifycr-fixed.c:378
#2 0x000000000040a574 in unifycr_fid_store_fixed_write (fid=1, meta=0x7fffddcf443c, pos=49152, buf=0x66280c0, count=4096) at unifycr-fixed.c:699
#3 0x0000000000405280 in unifycr_fid_write (fid=1, pos=49152, buf=0x66280c0, count=4096) at unifycr.c:746
#4 0x000000000040b1aa in unifycr_fd_write (fd=1, pos=49152, buf=0x66280c0, count=4096) at unifycr-sysio.c:541
#5 0x000000000040d45c in __wrap_pwrite (fd=1, buf=0x66280c0, count=4096, offset=49152) at unifycr-sysio.c:1463
#6 0x000000000040bc01 in __wrap_write (fd=1025, buf=0x66280c0, count=4096) at unifycr-sysio.c:865
#7 0x00000000004043a1 in main (argc=13, argv=0x7fffffffe358) at test_write.c:143
Hence I can only successfully write a file up to 12 4096 byte pages. Anything
larger than this, independent of block/transfer sizes, will fail.
I've checked, and the shared memory for the superblock, chunks, etc. is being
allocated correctly using: shm_open, ftruncate, mmap. It's there, the correct
size, and when the server attaches it agrees. However, I cannot access
(dereference) the memory in the chunk region beyond offset 49152 in the first
chunk of the chunk segment. The remainder of the allocated shared memory,
including addresses past the chunk segment, are accessible.
Apparently, this is a common problem with mmap of shared memory (often caused
by failure to size the segment with ftruncate), and according to a google
search, catching signal 7 should allow the program to continue.
I'm perplexed as to why I'm seeing this. I'm running RHEL 7.3 on Intel
X86 (64 bit, 64GB of memory), which is probably different than your
environment. Any ideas would be welcomed.
Thanks.
Joseph
From: "Moody, Adam T."
Date: Thursday, December 21, 2017 at 3:01 PM
To: "Moore, Joseph G."
Subject: Re: Shared memory issue
That's strange. I'll look through the code to get some ideas.
Can you tell whether the ftruncate is succeeding? How about the other calls setting up the segment?
-Adam
From: "Moore, Joseph G."
Date: Thursday, December 21, 2017 at 9:45 AM
To: "Moody, Adam T."
Subject: Shared memory issue
Adam,
The setup for the segment all runs fine. However, I think it's mostly just
assigning addresses from the segment to in-process data structures. The
ftruncate call is validated, and the calls to mmap and the init_pointers
function are skipped if it fails. I'm sure init_pointers runs, so
ftruncate is succeeding. This is at the bottom of unifycr_superblock_shmget()
in unifycr.c. We're using the code in the else for the
if (fs_type != UNIFYCR_LOG)
at the top of the function. (Our fs_type is UNIFYCR_LOG.)
The bus error is always triggered by accessing the range of the chunk storage
beyond 49,152. This is at an offset of 237,568 to 268,623,872 bytes within the
shared-memory region. (The chunk region is almost all of the shared memory.)
I've tested access to the remainder of the SM segment (at the beginning and
end) and it's always okay.
I'm guessing it's the OS trying to keep me from accessing memory that
may be invalid. I'm not an expert on this stuff, however. The weird thing
is the hard boundary at 49,152 into the chunk store. This never varies.
-Joseph
@jgmoore-or reported a SIGBUS error accessing shared memory on the unifycr mailing list. Opening an issue here for bug tracking purposes. Email thread quoted below.