fix: Command Getting Killed While Updating CVE Database or Scanning with cve-bin-tool

[pooja0805]

Description

Hello Team,

I encountered an issue when attempting to update the CVE database or scan a file/directory using the cve-bin-tool. The process terminates with a "Killed" error, as shown in the output below:

[test@f961efb782bb ~]$ cve-bin-tool --disable-data-source EPSS -u now  
[03:45:39] INFO     cve_bin_tool - CVE Binary Tool v3.4                      cli.py:624  
           INFO     cve_bin_tool - This product uses the NVD API but is not  cli.py:625  
                    endorsed or certified by the NVD.  
           INFO     cve_bin_tool - For potentially faster NVD downloads,     cli.py:628  
                    mirrors are available using -n json-mirror  
           INFO     cve_bin_tool - Disabling data source EPSS                cli.py:771  
[03:45:49] WARNING  cve_bin_tool.CVEDB - Updating cachedir                 cvedb.py:652  
                    /home/test/.cache/cve-bin-tool  
[03:45:52] INFO     cve_bin_tool - Getting NVD CVE data...            nvd_source.py:389  
           INFO     cve_bin_tool - Getting GitLab Advisory Database    gad_source.py:86  
                    CVEs...  
           INFO     cve_bin_tool - Getting RedHat CVEs...           redhat_source.py:69  
           INFO     cve_bin_tool - Getting PURL2CPE data...       purl2cpe_source.py:36  
Downloading CVEs... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:07  
[03:49:15] INFO     cve_bin_tool - Adding 35170 RedHat CVE entries redhat_source.py:136  
[03:51:30] INFO     cve_bin_tool - Getting Open Source Vulnerability  osv_source.py:161  
                    Database CVEs...  
Killed  

The command terminates unexpectedly after downloading and processing some of the CVE data.

To reproduce

Steps to reproduce the behaviour:
Run the following command to update the CVE database:

cve-bin-tool --disable-data-source EPSS -u now 

Expected behaviour: The command should complete the update process or scan successfully without terminating unexpectedly.
Actual behaviour: The command terminates unexpectedly after downloading and processing some of the CVE data.

Version/platform info

Version of CVE-bin-tool( e.g. output of cve-bin-tool --version): 3.4
Installed from pypi or github? pypi
Operating system: Linux/x86-64, linux/ppc64le
Python version (e.g. python3 --version): Python 3.11
Running in any particular CI environment we should know about? NA

Anything else?

Is there a known solution for this issue? Could this be related to resource limitations or another underlying problem? Any guidance or suggestions would be appreciated.

Thank you!

[terriko]

"killed" usually means it's been done by your system (cve-bin-tool itself terminates differently), so you've probably exceeded some system limit.

Likely culprits:

• timeout (this happens a lot in CI and cloud systems with limits imposed)
• disk space full

Running -u now will take about 10-30 minutes (depending on network/disk speed) and will need more than 2.5G of disk space.

BTW, even once you get things sorted I think cve-bin-tool will fail in your command because there's no file to scan, but it should do so after updating the database.

[pooja0805]

@terriko Thanks for your quick response. Let me check my system configurations and try out again.

[shivamv300]

hi @terriko
I'm also running into the same issue, even though I have 50+GB available disk space.
I tried running the scan in batches (by disabling some data sources: --disable-data-source NVD,OSV,PURL2CPE,REDHAT,RSD in the first pass), but the first pass itself has been running for more than 10 hours.
(My system has 4 cores, and 16 GB memory - out of which 14+ GB is free).

I did have an observation with this first pass scan:htop shows resource utilisation for one of the CPU cores close to 99% (while the other cores are free)

[terriko]

We've been seeing some out of memory issues that are likely related: #4710

@shivamv300 Other people have reported that disabling OSV solves the problem, but I see you've already tried that. The other thing that works in some cases is upgrading to python 3.13 (which has other memory improvements). I'm hideously sick so I haven't been able to work on figuring out a root cause, so any info and analysis is very welcome!