Merge pull request #2679 from hmpf/modernize-scripts

Modernize scripts in bin/

Author: hmpf

.github/workflows/build-and-test.yml

@@ -49,7 +49,7 @@ jobs:
       - name: "Install test runner dependencies"
         run: |
           set -xe
-          python -m pip install --upgrade 'pip==23.1.0' setuptools wheel 'tox<4' tox-gh-actions coverage virtualenv snmpsim 'pyasn1<0.5.0'
+          python3 -m pip install --upgrade 'pip==23.1.0' setuptools wheel 'tox<4' tox-gh-actions coverage virtualenv snmpsim 'pyasn1<0.5.0'
           sudo apt-get install -y nbtscan
 
       # virtualenv seems to currently be embedding a broken version of
@@ -80,7 +80,7 @@ jobs:
           export DISPLAY=:99
           ulimit -c unlimited
           sudo sysctl -w kernel.core_pattern=/tmp/core-%e.%p.%h.%t
-          python -m tox
+          python3 -m tox
 
       - name: Upload Selenium driver logs (${{ matrix.python-version }})
         if: always()

NOTES.rst

@@ -11,6 +11,38 @@ please go to https://github.com/uninett/nav/milestones .
 NAV 5.9
 =======
 
+Changed names of NAV command line programs
+------------------------------------------
+NAV has switched to a more canonical way of installing Python command line
+scripts, or "binaries".  This means that all NAV command line programs that
+previously ended with a ``.py`` extension now have been stripped of that
+extension.  Any custom cron jobs or scripts you have that may reference these
+NAV commands must be updated in order to continue working.
+
+It also means that you need to make sure your :file:`daemons.yml` configuration
+file is up-to-date after an upgrade, as well as the NAV cronjob snippets in the
+:file:`cron.d/` configuration directory.
+
+These commands are affected and no longer have a ``.py`` extension:
+
+* ``alertengine``
+* ``autoenable``
+* ``collect_active_ip``
+* ``emailreports``
+* ``logengine``
+* ``macwatch``
+* ``mailin``
+* ``maintengine``
+* ``netbiostracker``
+* ``pping``
+* ``radiusparser``
+* ``servicemon``
+* ``smsd``
+* ``snmptrapd``
+* ``sortedstats_cacher``
+* ``start_arnold``
+* ``t1000``
+
 Web security
 ------------
 

doc/hacking/hacking.rst

@@ -113,7 +113,6 @@ A rough guide to the source tree:
 =================================  =================================================================
 Directory                          Description
 =================================  =================================================================
-:file:`bin/`                       NAV 'binaries'; executable scripts and programs.
 :file:`contrib/`                   User contributed NAV tools. NAV doesn't depend on these, and any
                                    maintenance of them is left up to the original developers. We do
                                    not offer support for these tools.

doc/reference/arnold.rst

@@ -54,15 +54,15 @@ several views available. They are as follows:
 In addition you have two actions you can use - *Search* and *Manual detention*.
 
 - **Search** lets you search for detentions given some search parameters.
-- **Manual detention** lets you manually detain a port given an IP- or 
+- **Manual detention** lets you manually detain a port given an IP- or
   MAC-address.
 
 Predefined detentions
 ---------------------
-The only way to use a predefined detention is by using the ``start_arnold.py``
+The only way to use a predefined detention is by using the ``start_arnold``
 shell script. After creating a predefined detention you usually want to create a
 cron-job for running the script with some input parameters. See section about
-`start_arnold.py`_.
+`start_arnold`_.
 
 
 Using the scripts
@@ -71,48 +71,48 @@ Using the scripts
 Arnold consists of three scripts, which all are located in the ``nav/bin``
 directory.
 
-- **autoenable.py** enables ports based on the autoenable variable available for
+- **autoenable** enables ports based on the autoenable variable available for
   both manual and predefined detentions.
-- **start_arnold.py** is used in combination with predefined detentions to
+- **start_arnold** is used in combination with predefined detentions to
   invoke a series of detentions.
-- **t1000.py** verifies that the MAC-addresses that should be offline are not
+- **t1000** verifies that the MAC-addresses that should be offline are not
   active on other ports. If a detained MAC-address is online on another port, it
   will try to detain it there aswell.
 
 More details about the different scripts can be seen below.
 
-autoenable.py
--------------
+autoenable
+----------
 
-*autoenable.py* fetches all detained ports with an autoenable-value and enables
+*autoenable* fetches all detained ports with an autoenable-value and enables
 each of those detentions if the time is due. It can be run manually or as a
 periodic cron job.
 
 The simplest way of running automatic enabling periodically is to create a file
-containing cron configuration that calls the *autoenable.py* program as often as
+containing cron configuration that calls the *autoenable* program as often as
 you would like::
 
-  0 * * * * some_prefix/nav/bin/autoenable.py  # Run every hour on the hour
+  0 * * * * some_prefix/nav/bin/autoenable  # Run every hour on the hour
 
 Save this snippet in a file called ``autoenable`` in NAV's ``etc/cron.d/``
 directory. That way, you can add it to the navcron user's crontab by calling
 ``nav start autoenable``.
 
-start_arnold.py
----------------
+start_arnold
+------------
 
-When a predefined detention is created you can use *start_arnold.py* to invoke a
+When a predefined detention is created you can use *start_arnold* to invoke a
 series of detentions based on the input to the script.
 
 If the file or list of addresses exist locally then you can pipe it in using for
 instance ``cat``::
-  
-  # cat scanresult.txt | nav/bin/start_arnold_py -i
+
+  # cat scanresult.txt | nav/bin/start_arnold -i
 
 or you can do it from a remote server using ssh commands::
 
-  # cat scanresult.txt | ssh [email protected]:nav/bin/start_arnold_py -i
-  
+  # cat scanresult.txt | ssh [email protected]:nav/bin/start_arnold -i
+
 To avoid having to type passwords you want to create public keys, like described
 for instance `here <http://www.linuxproblem.org/art_9.html>`_.
 
@@ -123,19 +123,19 @@ Each line in this file is assumed to consist of an IP- or MAC-address and
 optionally a comment (separated by a space). For each valid address a detention
 will be made. Lines starting with *#* will be skipped.
 
-t1000.py
---------
+t1000
+-----
 
-This script needs to be set up to run in the same way as `autoenable.py`_.
+This script needs to be set up to run in the same way as `autoenable`_.
 
-*t1000.py* fetches all detained ports and checks if the MAC-address which was
+*t1000* fetches all detained ports and checks if the MAC-address which was
 behind the detained port is active on another port. If it is, it enforces the
 detention on that port aswell. Depending on options given at detention-time it
 will either remove the detention on the old port or just leave it.
 
 .. warning:: This does not detain the new port immediately after a detained
    computer has moved to it, because it takes some time before NAV discovers the
-   new location of the MAC-address. This combined with the interval t1000.py
+   new location of the MAC-address. This combined with the interval ``t1000``
    runs in could give the user quite some time with access before being detained
    again. This on-and-off behavior of internet access has been known to cause
    confusion and annoyance among the users - use this script knowing that.

doc/reference/mailin.rst

@@ -24,11 +24,11 @@ connections from outside the server, or no messages will come through.
 
 Pick an e-mail address on your NAV server to send 3rd party alerts to, for
 example `[email protected]`.  Mail received at this address should be
-piped through the ``mailin.py`` program.  This can usually be accomplished by
+piped through the ``mailin`` program.  This can usually be accomplished by
 adding an e-mail alias to :file:`/etc/aliases`, like this::
 
   cat >> /etc/aliases <<EOF
-  mailin: "| /usr/local/nav/bin/mailin.py"
+  mailin: "| /usr/local/nav/bin/mailin"
   EOF
   newaliases
 

doc/reference/radius-install.rst

@@ -185,7 +185,7 @@ Copy the script to a location of your choice on the server where the
 that executes this script as often as you would like to make sure that the
 script is actually running. For example::
 
-  0 * * * *  /path/to/radiusparser.py
+  0 * * * *  /path/to/radiusparser
 
 This will run the script every hour, but if it detects that it is already
 running, it quits and leaves the running script alone.
@@ -210,14 +210,14 @@ accounting log table accumulating forever.  To arrange for periodic
 deletion of old records, create a cron snippet in NAV's ``etc/cron.d/``
 directory called ``radiusclean`` and add the following lines to it::
 
-  50 5 * * 6 /path/to/navclean.py --radiusacct -E "3 months" -f
-  45 5 * * 6 /path/to/navclean.py --radiuslog  -E "1 month" -f
+  50 5 * * 6 /path/to/navclean --radiusacct -E "3 months" -f
+  45 5 * * 6 /path/to/navclean --radiuslog  -E "1 month" -f
 
 To insert the cron snippet into `navcron`'s crontab, run::
 
   sudo nav start radiusclean
 
-This will run the ``navclean.py`` program once a week, deleting all radius
+This will run the ``navclean`` program once a week, deleting all radius
 accounting entries older than three months and all radius error log entries
 older than one month.  Feel free to change the intervals to something you
 think is suitable for your organization.

doc/reference/radius.rst

@@ -21,10 +21,10 @@ Files
 doc/README.txt            What you are probably reading right now :]
 doc/INSTALL.txt           How to install the radius accounting subsystem
 
-bin/navclean.py           Modified version of navclean.py, that also handles
+bin/navclean              Modified version of navclean.py, that also handles
                           the radius accounting table. Should probably me made
                           to also handle the log table, or dropped all together.
-bin/radiusparser.py       radius.log parser that inserts data into the database.
+bin/radiusparser          radius.log parser that inserts data into the database.
 
 sql/accounting_table.sql  Script for creating the accounting table
 sql/log_table.sql         Script for creating the log table
@@ -62,7 +62,7 @@ It should be pretty obvious that we have ripped extensively from NAV's
 MachineTracker subsystem, and at the time writing this, even the graphics in the
 tables are from MachineTracker.
 
-There have yet to be confirmed reports on this stuff working (or even being 
+There have yet to be confirmed reports on this stuff working (or even being
 tried) on any other setup than what it was developed on, which is:
 
 * NAV v3.3.3
@@ -94,9 +94,9 @@ Hopefully it will be pretty self explanatory. You choose your search criteria
 and click the search button. The sessions that get matched are all sessions
 that "touch" your specified time interval in any way. At the bottom of the
 page, you will get a summary for your search, giving you the total amount of
-uploaded/downloaded data for all sessions matching your search criterias. This 
-feature is currently made a bit useless from the duplicate sessions in the 
-database (see `Known Issues`_). 
+uploaded/downloaded data for all sessions matching your search criterias. This
+feature is currently made a bit useless from the duplicate sessions in the
+database (see `Known Issues`_).
 
 In the search results, you will sometimes see sessions whose `Session Stop`
 field contains text in stead of the actual stop time. Here is an explanation
@@ -114,7 +114,7 @@ of what they mean:
 
 For this feature to work correctly, the variable ``REAUTH_TIMEOUT`` in
 ``radius_config.py`` must be set correctly
- 
+
 There are links from `Username`, `Realm`, `Assigned IP` and `NAS IP`
 to new searches. I.e. clicking on a username will show all sessions for this
 user, using the already specified time interval. A click on a session id will
@@ -141,7 +141,7 @@ This chart will lie a little, since it sums up all sessions that ended within
 the specified number of days. The reason for doing this is that we only get any
 numbers on how much data the user has sent/received when the session ends. Thus
 there is really no way to know for sure, just from the radius accounting log,
-when during the session the data was transferred. 
+when during the session the data was transferred.
 
 Of course, we could always limit the search to sessions that only started
 inside our search interval, but then a lot of long sessions might slip under
@@ -165,18 +165,18 @@ Known Issues
 Accounting module: Duplicate entries for some sessions
 ------------------------------------------------------
 
-Sometimes, when a Start message is immediately followed by an Alive message for 
-the same session, FreeRADIUS inserts the session into the database twice, the 
-only difference between them seem to be a few hundreds of a second on the Start 
+Sometimes, when a Start message is immediately followed by an Alive message for
+the same session, FreeRADIUS inserts the session into the database twice, the
+only difference between them seem to be a few hundreds of a second on the Start
 time. This seems to be caused by the following scenario:
 
 1) FreeRADIUS receives a Start packet and inserts a new entry/session in the db
-2) FreeRADIUS receives an Alive packet for the same session *immediately* after 
+2) FreeRADIUS receives an Alive packet for the same session *immediately* after
    the Start packet, and queries the database to see if the `unique-session-id`
    already exists.
 3) The query doesn't return anything, since postgresql hasn't had time to
-   complete the `INSERT`-query for the Start packet, and 
-   ``accounting_update_query_alt`` is thus run, inserting a new row. 
+   complete the `INSERT`-query for the Start packet, and
+   ``accounting_update_query_alt`` is thus run, inserting a new row.
 
 How to get around this? I'm not quite sure. Maybe someone with more experience
 with (Postgre)SQL could look at some kind of table locking, if this wouldn't

doc/reference/smsd.rst

@@ -12,7 +12,7 @@ dispatcher handler layer.
 Usage
 =====
 
-For more help on usage, run ``smsd.py --help`` and see the ``smsd.conf`` config
+For more help on usage, run ``smsd --help`` and see the ``smsd.conf`` config
 file.
 
 

doc/reference/snmptrapd.rst

@@ -20,7 +20,7 @@ Usage
 =====
 
 snmptrapd can be started and stopped using the regular `nav start` and
-`nav stop` commands.  The :program:`snmptrapd.py` program can also be run in
+`nav stop` commands.  The :program:`snmptrapd` program can also be run in
 the foreground on the command line, logging all its activites to the
 standard output instead of the log file.
 
@@ -30,7 +30,7 @@ the port is bound.
 
 ::
 
-    usage: snmptrapd.py [-h] [-d] [-c COMMUNITY] [address [address ...]]
+    usage: snmptrapd [-h] [-d] [-c COMMUNITY] [address [address ...]]
 
     NAV SNMP Trap daemon
 

pyproject.toml

@@ -25,6 +25,42 @@ dependencies = [
 [project.urls]
 Homepage = "https://nav.uninett.no/"
 
+[project.scripts]
+alertengine = "nav.bin.alertengine:main"
+autoenable = "nav.bin.autoenable:main"
+collect_active_ip = "nav.bin.collect_active_ip:main"
+emailreports = "nav.bin.emailreports:main"
+eventengine = "nav.bin.eventengine:main"
+ipdevpolld = "nav.bin.ipdevpolld:main"
+logengine = "nav.bin.logengine:main"
+macwatch = "nav.bin.macwatch:main"
+mailin = "nav.bin.mailin:main"
+maintengine = "nav.bin.maintengine:main"
+nav = "nav.bin.navmain:main"
+navcheckservice = "nav.bin.navcheckservice:main"
+navclean = "nav.bin.navclean:main"
+navdf = "nav.bin.navdf:main"
+navdump = "nav.bin.navdump:main"
+naventity = "nav.bin.naventity:main"
+navoidverify = "nav.bin.navoidverify:main"
+navpgdump = "nav.bin.navpgdump:main"
+navsnmp = "nav.bin.navsnmp:main"
+navstats = "nav.bin.navstats:main"
+navsyncdb = "nav.bin.navsyncdb:main"
+navsynctypes = "nav.bin.navsynctypes:main"
+navtopology = "nav.bin.navtopology:main"
+navuser = "nav.bin.navuser:main"
+netbiostracker = "nav.bin.netbiostracker:main"
+pping = "nav.bin.pping:main"
+radiusparser = "nav.bin.radiusparser:main"
+servicemon = "nav.bin.servicemon:main"
+smsd = "nav.bin.smsd:main"
+snmptrapd = "nav.bin.snmptrapd:main"
+sortedstats_cacher = "nav.bin.sortedstats_cacher:main"
+start_arnold = "nav.bin.start_arnold:main"
+t1000 = "nav.bin.t1000:main"
+thresholdmon = "nav.bin.thresholdmon:main"
+
 [tool.setuptools]
 include-package-data = true
 zip-safe = false

python/nav/bin/__init__.py

@@ -19,7 +19,7 @@
 Autoenable removes detention for computers that are done with detention.
 
 Usage:
-sudo -u $NAV_USER ./autoenable.py
+sudo -u $NAV_USER autoenable
 
 autoenable is meant to be run as a cronjob. It checks the configured
 arnold-database for any detained ports and opens them if they have a