Skip to content

Revise nut-scanner parallelization of scanning #2511

Open
@jimklimov

Description

@jimklimov

Earlier work (~2017 for 42ITy) saw addition of pthreads support for networked scans, so that large subnets could be scanned in finite time. IIRC, this used separate thread pools per "bus", so scans involving e.g. NetXML and SNMP and Old NUT would use 2x-3x the requested limit of threads.

With work on issue #2244 (PR #2509) allowing several IP address ranges to be requested for scanning, it was discovered that each range-scan request, even if for just one IP address, suffers the timeout (5 sec by default) if some addresses do not reply. Then the fanned-out threads from one IP address range request are all reaped and another range for the same bus is inspected.

This issue proposes to separate the thread-pool creation and reaping from the scanning method, so that many range scans can be requested and get reaped only afterwards. As an option, maybe converge to one thread pool shared by all buses (so limiting the amount of threads to what the user allowed/requested).

It could be nice to come up with a way to reduce copy-paste in the scanning methods about different threading implementations. It could in fact be part of moving to some single shared thread pool implemented in whatever somewhat abstracted manner (e.g. like we have IP address iterator methods, we could have some thread iterator - or block on it waiting for a free slot).

On a related note, it can help to update the IP address iteration method to handle not one range, but the array of such ranges (introduced with PR #2509), so we would call the scanning method once and avoid the problem of waiting for a timeout of one range before we can start the next.

Also note that sem_init() is not implemented on MacOS; some other methods should be used. See e.g. https://stackoverflow.com/questions/1413785/sem-init-on-os-x

Also, IPMI scanning is not parallelized at all

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions