Got "couldn't init manager error" when launching ecapture in docker container.

[pfmiles]

Trying to run ecapture from cmd line, in a Docker container, as root. It emits error:

root@test-xhunterbase-56b679589-w7nmz:/tmp/ecapture-v0.6.3-linux-x86_64# ./ecapture --libssl="/usr/lib/x86_64-linux-gnu/libssl.so.1.1" --gnutls="/usr/lib/x86_64-linux-gnu/libgnutls.so.30" --nspr="/usr/lib/x86_64-linux-gnu/libnspr4.so" tls
2023/10/13 14:08:29 Your environment is a container. We will not detect the BTF config.
tls_2023/10/13 14:08:29 ECAPTURE :: ecapture Version : linux_x86_64:0.6.3-20230927-f0cfbdf:5.15.0-1046-azure
tls_2023/10/13 14:08:29 ECAPTURE :: Pid Info : 29690
tls_2023/10/13 14:08:29 ECAPTURE :: Kernel Info : 4.19.24
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        module initialization
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        master key keylogger: ecapture_masterkey.log
tls_2023/10/13 14:08:29 ECAPTURE ::     Module.Run()
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        UPROBE MODEL
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        origin version:OpenSSL 1.1.1n, as key:openssl 1.1.1n
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        HOOK type:2, binrayPath:/usr/lib/x86_64-linux-gnu/libssl.so.1.1
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        libPthread:/lib/x86_64-linux-gnu/libpthread.so.0
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        Hook masterKey function:SSL_write
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        Your kernel version is less than 5.2, the following parameters will be ignored:[target_pid, target_uid, target_port]
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        BPF bytecode filename:user/bytecode/openssl_1_1_1j_kern_less52.o
tls_2023/10/13 14:08:29 EBPFProbeOPENSSL        module run failed, [skip it]. error:couldn't init manager error:operation not permitted , couldn't adjust RLIMIT_MEMLOCK
tls_2023/10/13 14:08:29 EBPFProbeGNUTLS module initialization
tls_2023/10/13 14:08:29 ECAPTURE ::     Module.Run()
tls_2023/10/13 14:08:29 EBPFProbeGNUTLS BPF bytecode filename:user/bytecode/gnutls_kern_less52.o
tls_2023/10/13 14:08:29 EBPFProbeGNUTLS HOOK type:2, binrayPath:/usr/lib/x86_64-linux-gnu/libgnutls.so.30
tls_2023/10/13 14:08:29 EBPFProbeGNUTLS module run failed, [skip it]. error:couldn't init manager error:operation not permitted , couldn't adjust RLIMIT_MEMLOCK
tls_2023/10/13 14:08:29 EBPFProbeNSPR   module initialization
tls_2023/10/13 14:08:29 ECAPTURE ::     Module.Run()
tls_2023/10/13 14:08:29 EBPFProbeNSPR   BPF bytecode filename:user/bytecode/nspr_kern_less52.o
tls_2023/10/13 14:08:29 EBPFProbeNSPR   HOOK type:2, binrayPath:/usr/lib/x86_64-linux-gnu/libnspr4.so
tls_2023/10/13 14:08:29 EBPFProbeNSPR   module run failed, [skip it]. error:couldn't init manager error:operation not permitted , couldn't adjust RLIMIT_MEMLOCK 
tls_2023/10/13 14:08:29 ECAPTURE ::     No runnable modules, Exit(1)

It seems all of three possible tls hooking modules failed to run. And all of those point to a same error: couldn't init manager error:operation not permitted , couldn't adjust RLIMIT_MEMLOCK

To Reproduce
Steps to reproduce the behavior:
Just download ecapture binary(amd64 version), then unzip it and run the command in a bash repl.

Expected behavior

Screenshots

Linux Server/Android (please complete the following information):

• Env:
• KUBERNETES_SERVICE_PORT_HTTPS=443
  KUBERNETES_SERVICE_PORT=6443
  HOSTNAME=test-xhunterbase-56b679589-w7nmz
  LANGUAGE=en_US.UTF-8
  TEST_XHUNTERBASE_SVC_PORT_22_TCP=tcp://172.21.10.6:22
  JAVA_HOME=/opt/jdk
  PWD=/tmp/ecapture-v0.6.3-linux-x86_64
  LS_OPTIONS=--color=auto
  TEST_XHUNTERBASE_SVC_PORT_22_TCP_PORT=22
  HOME=/root
  LANG=en_US.UTF-8
  KUBERNETES_PORT_443_TCP=tcp://172.21.0.1:443
  TEST_XHUNTERBASE_SVC_PORT_10000_TCP_PORT=10000
  TEST_XHUNTERBASE_SVC_PORT_10000_TCP=tcp://172.21.10.6:10000
  TEST_XHUNTERBASE_SVC_PORT_10000_TCP_ADDR=172.21.10.6
  TEST_XHUNTERBASE_SVC_PORT_10000_TCP_PROTO=tcp
  TERM=xterm-256color
  TEST_XHUNTERBASE_SVC_PORT_22_TCP_ADDR=172.21.10.6
  aliyun_log_crd_user_defined_id=k8s-group-c3e4daee1064f4158891f67a0e996ef7a
  SHLVL=2
  TEST_XHUNTERBASE_SVC_PORT=tcp://172.21.10.6:22
  KUBERNETES_PORT_443_TCP_PROTO=tcp
  KUBERNETES_PORT_443_TCP_ADDR=172.21.0.1
  TEST_XHUNTERBASE_SVC_PORT_22_TCP_PROTO=tcp
  TEST_XHUNTERBASE_SVC_SERVICE_PORT_TEST_XHUNTERBASE_SVC_22_22=22
  TEST_XHUNTERBASE_SVC_SERVICE_PORT_TEST_XHUNTERBASE_SVC_10000_10000=10000
  TEST_XHUNTERBASE_SVC_SERVICE_PORT=22
  KUBERNETES_SERVICE_HOST=172.26.19.176
  LC_ALL=en_US.UTF-8
  KUBERNETES_PORT=tcp://172.21.0.1:443
  KUBERNETES_PORT_443_TCP_PORT=443
  PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/jdk/bin
  TEST_XHUNTERBASE_SVC_SERVICE_HOST=172.21.10.6
  OLDPWD=/tmp
  _=/usr/bin/env
• OS: Debian 11 "bullseye"
• Arch: amd64
• Kernel Version: 4.19.24-7.34.cbp.al7.x86_64
• Version: v0.6.3-linux-x86_64

Additional context
I'm running a debian 11 linux image in a docker container.

[cfc4n]

what's your docker launched command?

[cfc4n]

Have you granted SYS_ADMIN permissions to Docker?

e.g. : docker run --cap-add=SYS_ADMIN

If you encounter any other issues while using eCapture in a container, you can refer to #65.

[pfmiles]

Have you granted SYS_ADMIN permissions to Docker?

e.g. : docker run --cap-add=SYS_ADMIN

If you encounter any other issues while using eCapture in a container, you can refer to #65.

Thanks for the suggestion. I did not add the --cap-add=SYS_ADMIN option when executing docker run. I will try adding this option as soon as possible to see if it works. However, it is a bit troublesome because I am not running docker in the local command line, but on the Alibaba Cloud. Therefore, I may need to wrestle with its web console for a while. I will try it out and provide the results as soon as possible.

[pfmiles]

Sorry for the late reply.
I couldn't find a place in the Aliyun web console where I can add the --cap-add=SYS_ADMIN or --privileged options.
So I tried using a Kubernetes client SDK programmatically to add these options, but I received an error message saying that the managed serverless service does not support adding these options:

Then I ran the same x64-based image on my own Macbook Pro (M1) laptop with Rosetta2 for automatic instruction translation, with the --privileged option:

docker run --hostname=ebd37a374017 --env=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin -p 6022:22 --label='version=0.4.0' --runtime=runc -d -it --privileged xhunterbase:0.4.0

The output was as follows:

admin@ebd37a374017:/tmp/ecapture-v0.6.4-linux-x86_64$ sudo ./ecapture --libssl="/usr/lib/x86_64-linux-gnu/libssl.so.1.1" --gnutls="/usr/lib/x86_64-linux-gnu/libgnutls.so.30" --nspr="/usr/lib/x86_64-linux-gnu/libnspr4.so" tls
2023/10/19 18:05:44 Your environment is a container. We will not detect the BTF config.
tls_2023/10/19 18:05:44 ECAPTURE :: ecapture Version : linux_x86_64:0.6.4-20231015-f50129f:5.15.0-1047-azure
tls_2023/10/19 18:05:44 ECAPTURE :: Pid Info : 223
tls_2023/10/19 18:05:44 ECAPTURE :: Kernel Info : 5.15.49
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	module initialization
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	master key keylogger: ecapture_masterkey.log
tls_2023/10/19 18:05:44 ECAPTURE ::	Module.Run()
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	UPROBE MODEL
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	origin version:OpenSSL 1.1.1w, as key:openssl 1.1.1w
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	OpenSSL/BoringSSL version not found from shared library file, used default version:linux_default_1_1_1
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	HOOK type:2, binrayPath:/usr/lib/x86_64-linux-gnu/libssl.so.1.1
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	libPthread:/lib/x86_64-linux-gnu/libpthread.so.0
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	Hook masterKey function:SSL_write
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	target all process. 
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	target all users. 
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	BPF bytecode filename:user/bytecode/openssl_1_1_1j_kern.o
tls_2023/10/19 18:05:44 EBPFProbeOPENSSL	module run failed, [skip it]. error:couldn't init manager error:map connect_events: map create: function not implemented (without BTF k/v) , couldn't load eBPF programs, cs:&{map[.rodata:Array(keySize=4, valueSize=36, maxEntries=1, flags=128) active_ssl_read_args_map:Hash(keySize=8, valueSize=16, maxEntries=1024, flags=0) active_ssl_write_args_map:Hash(keySize=8, valueSize=16, maxEntries=1024, flags=0) bpf_context:LRUHash(keySize=8, valueSize=408, maxEntries=2048, flags=0) bpf_context_gen:Array(keySize=4, valueSize=408, maxEntries=1, flags=0) connect_events:PerfEventArray(keySize=4, valueSize=4, maxEntries=5, flags=0) data_buffer_heap:PerCPUArray(keySize=4, valueSize=4152, maxEntries=1, flags=0) mastersecret_events:PerfEventArray(keySize=4, valueSize=4, maxEntries=5, flags=0) network_map:LRUHash(keySize=20, valueSize=20, maxEntries=10240, flags=0) skb_data_buffer_heap:PerCPUArray(keySize=4, valueSize=40, maxEntries=1, flags=0) skb_events:PerfEventArray(keySize=4, valueSize=4, maxEntries=5, flags=0) ssl_st_fd:Hash(keySize=8, valueSize=8, maxEntries=10240, flags=0) tls_events:PerfEventArray(keySize=4, valueSize=4, maxEntries=5, flags=0)] map[egress_cls_func:0xc0001e6c60 ingress_cls_func:0xc0001e6bd0 probe_SSL_set_fd:0xc0001e6ab0 probe_connect:0xc0001e6e10 probe_entry_SSL_read:0xc0001e6d80 probe_entry_SSL_write:0xc0001e6900 probe_ret_SSL_read:0xc0001e6a20 probe_ret_SSL_write:0xc0001e6990 probe_ssl_master_key:0xc0001e6b40 tcp_sendmsg:0xc0001e6cf0] 0xc000264180 LittleEndian}
tls_2023/10/19 18:05:44 EBPFProbeGNUTLS	module initialization
tls_2023/10/19 18:05:44 ECAPTURE ::	Module.Run()
tls_2023/10/19 18:05:44 EBPFProbeGNUTLS	BPF bytecode filename:user/bytecode/gnutls_kern.o
tls_2023/10/19 18:05:44 EBPFProbeGNUTLS	HOOK type:2, binrayPath:/usr/lib/x86_64-linux-gnu/libgnutls.so.30
tls_2023/10/19 18:05:44 EBPFProbeGNUTLS	target all process. 
tls_2023/10/19 18:05:44 EBPFProbeGNUTLS	module run failed, [skip it]. error:couldn't init manager error:map active_ssl_write_args_map: load BTF: detect support for Map BTF (Var/Datasec): detect support for BTF: function not implemented , couldn't load eBPF programs, cs:&{map[.rodata:Array(keySize=4, valueSize=32, maxEntries=1, flags=128) active_ssl_read_args_map:Hash(keySize=8, valueSize=8, maxEntries=1024, flags=0) active_ssl_write_args_map:Hash(keySize=8, valueSize=8, maxEntries=1024, flags=0) data_buffer_heap:PerCPUArray(keySize=4, valueSize=4144, maxEntries=1, flags=0) gnutls_events:PerfEventArray(keySize=4, valueSize=4, maxEntries=5, flags=0)] map[probe_entry_SSL_read:0xc0001e7440 probe_entry_SSL_write:0xc0001e7320 probe_ret_SSL_read:0xc0001e74d0 probe_ret_SSL_write:0xc0001e73b0] 0xc0000ca880 LittleEndian}
tls_2023/10/19 18:05:44 EBPFProbeNSPR	module initialization
tls_2023/10/19 18:05:44 ECAPTURE ::	Module.Run()
tls_2023/10/19 18:05:44 EBPFProbeNSPR	BPF bytecode filename:user/bytecode/nspr_kern.o
tls_2023/10/19 18:05:44 EBPFProbeNSPR	HOOK type:2, binrayPath:/usr/lib/x86_64-linux-gnu/libnspr4.so
tls_2023/10/19 18:05:44 EBPFProbeNSPR	target all process. 
tls_2023/10/19 18:05:44 EBPFProbeNSPR	module run failed, [skip it]. error:couldn't init manager error:map active_ssl_write_args_map: load BTF: detect support for Map BTF (Var/Datasec): detect support for BTF: function not implemented , couldn't load eBPF programs, cs:&{map[.rodata:Array(keySize=4, valueSize=32, maxEntries=1, flags=128) active_ssl_read_args_map:Hash(keySize=8, valueSize=8, maxEntries=1024, flags=0) active_ssl_write_args_map:Hash(keySize=8, valueSize=8, maxEntries=1024, flags=0) data_buffer_heap:PerCPUArray(keySize=4, valueSize=4144, maxEntries=1, flags=0) nspr_events:PerfEventArray(keySize=4, valueSize=4, maxEntries=5, flags=0)] map[probe_entry_SSL_read:0xc0001e7cb0 probe_entry_SSL_write:0xc0001e7b90 probe_ret_SSL_read:0xc0001e7b00 probe_ret_SSL_write:0xc0001e7c20] 0xc0000cac80 LittleEndian} 
tls_2023/10/19 18:05:44 ECAPTURE :: 	No runnable modules, Exit(1)

I can notice that the error message has changed, but it still failed to start, indicating "...function not implemented...". I suspect that the eBPF feature is not fully supported by the macOS kernel on my Macbook Pro.

Since the serverless service I mainly use on the cloud does not allow me to add admin privileges to the Docker container instances, I may not use ecapture in this scenario for now.
I also do not currently have any Linux servers with a kernel version greater than 4.18, so I haven't done any further testing.

If I ever need to do something similar on a VPS machine that I can manage myself, I will follow the instructions in issue:#65 and try using ecapture, which should work I guess.

Thanks anyway!

[cfc4n]

The privileged parameter will introduce significant security issues, endangering the host system. Some public cloud vendors disable this.

The host kernel in macOS differs from Linux and does not support eBPF features.

For security issues arising from granting SYS_ADMIN permissions to containers, you can refer to this video: eBPF容器逃逸与隐藏账号rootkit