[Bug]: nb_inventory.py not working with Netbox 3.2.x #775

HardTy · 2022-04-21T14:14:22Z

Ansible NetBox Collection version

v3.7.0

Ansible version

ansible [core 2.12.4]
  config file = /Users/rene/github/riege/ansible-localdns/ansible.cfg
  configured module search path = ['/Users/rene/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/rene/Library/Python/3.9/lib/python/site-packages/ansible
  ansible collection location = /Users/rene/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.12 (main, Mar 26 2022, 15:51:15) [Clang 13.1.6 (clang-1316.0.21.2)]
  jinja version = 3.0.1
  libyaml = True

NetBox version

v3.2.1

Python version

3.9

Steps to Reproduce

Try to get a inventory from Netbox like with ansible-inventory command.

$ ansible-inventory --list -i netbox-inventory.yml
 -vvv
ansible-inventory [core 2.12.4]
  config file = /Users/rene/github/riege/ansible-localdns/ansible.cfg
  configured module search path = ['/Users/rene/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/rene/Library/Python/3.9/lib/python/site-packages/ansible
  ansible collection location = /Users/rene/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible-inventory
  python version = 3.9.12 (main, Mar 26 2022, 15:51:15) [Clang 13.1.6 (clang-1316.0.21.2)]
  jinja version = 3.0.1
  libyaml = True
Using /Users/rene/github/riege/ansible-localdns/ansible.cfg as config file
host_list declined parsing /Users/rene/github/riege/ansible-localdns/netbox-inventory.yml as it did not pass its verify_file() method
script declined parsing /Users/rene/github/riege/ansible-localdns/netbox-inventory.yml as it did not pass its verify_file() method
Fetching: https://netbox.infraops.dev.riege.cloud/api/status
Fetching: https://netbox.infraops.dev.riege.cloud/api/docs/?format=openapi
toml declined parsing /Users/rene/github/riege/ansible-localdns/netbox-inventory.yml as it did not pass its verify_file() method
[WARNING]:  * Failed to parse /Users/rene/github/riege/ansible-localdns/netbox-inventory.yml with auto
plugin: Incorrect JSON payload:  <!DOCTYPE html> <html> <head>     <meta charset="utf-8"/>
<title>NetBox API</title>                                            <link rel="icon" type="image/png"
href="/static/drf-yasg/swagger-ui-dist/favicon-32x32.png"/>                    <link rel="stylesheet"
type="text/css" href="/static/drf-yasg/style.css"/>         <link rel="stylesheet" type="text/css"
href="/static/drf-yasg/swagger-ui-dist/swagger-ui.css">                         </head>  <body
class="swagger-body">          <div id="swagger-ui"></div>          <script id="swagger-settings"
type="application/json">{"docExpansion": "list", "deepLinking": false, "showExtensions": true,
"defaultModelRendering": "model", "defaultModelExpandDepth": 1, "defaultModelsExpandDepth": 1,
"showCommonExtensions": true, "supportedSubmitMethods": ["get", "put", "post", "delete", "options",
"head", "patch", "trace"], "displayOperationId": true, "persistAuth": false, "refetchWithAuth": false,
"refetchOnLogout": false, "fetchSchemaWithQuery": true, "validatorUrl": null}</script> <script
id="oauth2-config" type="application/json">{}</script>       <script src="/static/drf-yasg/swagger-ui-
dist/swagger-ui-bundle.js"></script>     <script src="/static/drf-yasg/swagger-ui-dist/swagger-ui-
standalone-preset.js"></script>     <script src="/static/drf-yasg/insQ.min.js"></script>     <script
src="/static/drf-yasg/immutable.min.js"></script>     <script src="/static/drf-yasg/url-
polyfill.min.js"></script>     <script src="/static/drf-yasg/swagger-ui-init.js"></script>          <a
id="oauth2-redirect-url" href="/static/drf-yasg/swagger-ui-dist/oauth2-redirect.html"
class="hidden"></a>       <div id="django-session-auth" class="hidden">                      <input
type="hidden" name="csrfmiddlewaretoken"
value="*****">
<div class='btn authorize'>                     <a id="auth" class="header__btn"
href="/login/?next=/api/docs/" data-sw-translate>
Django Login                                              </a>                 </div>
</div>  </body>  </html>
  File "/Users/rene/Library/Python/3.9/lib/python/site-packages/ansible/inventory/manager.py", line 290, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/Users/rene/Library/Python/3.9/lib/python/site-packages/ansible/plugins/inventory/auto.py", line 58, in parse
    plugin.parse(inventory, loader, path, cache=cache)
  File "/Users/rene/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 1887, in parse
    self.main()
  File "/Users/rene/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 1759, in main
    self.fetch_api_docs()
  File "/Users/rene/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 1387, in fetch_api_docs
    openapi = self._fetch_information(
  File "/Users/rene/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 392, in _fetch_information
    raise AnsibleError("Incorrect JSON payload: %s" % raw_data)
[WARNING]:  * Failed to parse /Users/rene/github/riege/ansible-localdns/netbox-inventory.yml with yaml
plugin: Plugin configuration YAML file, not YAML inventory
  File "/Users/rene/Library/Python/3.9/lib/python/site-packages/ansible/inventory/manager.py", line 290, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/Users/rene/Library/Python/3.9/lib/python/site-packages/ansible/plugins/inventory/yaml.py", line 112, in parse
    raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory')
[WARNING]:  * Failed to parse /Users/rene/github/riege/ansible-localdns/netbox-inventory.yml with ini
plugin: Invalid host pattern '---' supplied, '---' is normally a sign this is a YAML file.
  File "/Users/rene/Library/Python/3.9/lib/python/site-packages/ansible/inventory/manager.py", line 290, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/Users/rene/Library/Python/3.9/lib/python/site-packages/ansible/plugins/inventory/ini.py", line 136, in parse
    raise AnsibleParserError(e)
[WARNING]: Unable to parse /Users/rene/github/riege/ansible-localdns/netbox-inventory.yml as an
inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
{
    "_meta": {
        "hostvars": {}
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    }
}

It is the same behavior with ansible-playbook command.

Expected Behavior

Get a working Ansible inventory.

Observed Behavior

See "Steps to Reproduce".

The text was updated successfully, but these errors were encountered:

nomaster · 2022-04-21T14:48:51Z

It seems that the "Accept: application/json" header is missing in the request

update: we investigated the issue and discovered that the request sets the 'content-type' header, which is technically not the right one. The header hat should be set is the (missing) 'Accept', see RFC2616

If no Accept header field is present, then it is assumed that the
client accepts all media types. If an Accept header field is present,
and if the server cannot send a response which is acceptable
according to the combined Accept field value, then the server SHOULD
send a 406 (not acceptable) response.

ansible_modules/plugins/inventory/nb_inventory.py

Lines 1865 to 1869 in 5d6cb68

    
           self.headers = { 
        
               "User-Agent": "ansible %s Python %s" 
        
               % (ansible_version, python_version.split(" ", maxsplit=1)[0]), 
        
               "Content-type": "application/json", 
        
           }

rodvand · 2022-04-22T07:09:55Z

Thanks for reporting this. I am not able to reproduce the issue on my own NetBox instance, or the demo instance, both running 3.2.1.

I'm using the simplest nb_inventory config I can think of:

plugin: netbox.netbox.nb_inventory
api_endpoint: https://demo.netbox.dev
token: TOKEN

ryanmerolle · 2022-04-25T21:29:36Z

Please share your netbox_inventory.yml contents, pynetbox version, and requests version.

ryanmerolle · 2022-04-26T20:54:56Z

Also, if this ever worked for you @HardTy, what version NetBox and ansible netbox collections was it working with?

HardTy · 2022-04-27T08:30:11Z

We found the problem. We use Varnish as cache for the Netbox Api to speed it up. But it seems there was a change with API which hit us. The setup works with Netbox versions before 3.2.x. So we need the 'Accept' header as described by @nomaster and in the link PR 777.

rodvand · 2022-04-27T20:32:08Z

@HardTy Interesting. Do you know why it fails with Varnish as a cache? I've mentioned in the #777 PR that introducing the Accept header fails with the demo install (in my tests).

sc68cal · 2022-04-27T20:34:45Z

My suspicion is that Varnish is expecting the Accept header to be set, which is different than what the NetBox http server expects/requires.

ryanmerolle · 2023-02-25T05:14:53Z

@HardTy how goes it, should this be closed out?

ryanmerolle · 2023-02-25T05:38:35Z

adding @twink0r since they raised a PR

pouriyabp · 2023-05-06T13:00:08Z

I have this problem too, any solution?

rodvand · 2023-05-30T11:44:00Z

I also have this problem:

Rocky Linux 9
ansible [core 2.14.2]
  config file = /srv/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.11/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.11.2 (main, Feb 16 2023, 00:00:00) [GCC 11.3.1 20221121 (Red Hat 11.3.1-4)] (/usr/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True

ansible-inventory -i .inventory-netbox.yml --list
[WARNING]:  * Failed to parse /srv/.inventory-netbox.yml with auto plugin: pytz must be installed to
use this plugin
[WARNING]:  * Failed to parse /srv/.inventory-netbox.yml with yaml plugin: Plugin configuration YAML
file, not YAML inventory
[WARNING]: No inventory was parsed, only implicit localhost is available

This is not the same issue. Follow the collection requirements (install pytz as the output mentions is missing) and you should be fine.

k4mil666 · 2023-05-30T11:55:40Z

The most interesting thing is that I have this module installed.

[kk@prod-ansible3 inventory]$ pip list | grep pyt
dbus-python 1.2.18
python-dateutil 2.8.1
pytz 2021.1

[kk@prod-ansible3 inventory]$ python -c "import pytz"

k4mil666 · 2023-05-30T12:06:28Z

I had to create a virtual environment (venv) and only then the installation in this environment works correctly.

python3 -m venv ansible_env
source ansible_env/bin/activate
pip install ansible pytz

Thank you very much for your interest. Regards.

cleverer · 2023-06-14T20:55:15Z

For anyone else "just" installing ansible following the debian install instructions:

What helped for me was to manually install pip and pytz globally:

# Original Ansible install worked with this:
apt install ansible

# nb_inventory also needs:
apt install python3-pip
pip install pytz

HardTy added the bug Something isn't working label Apr 21, 2022

twink0r mentioned this issue Apr 21, 2022

Use the Accept Header to indicate that the response should be JSON #777

Open

3 tasks

ryanmerolle changed the title ~~nb_inventory.py not working with Netbox 3.2.x [Bug]:~~ [Bug]: nb_inventory.py not working with Netbox 3.2.x Apr 25, 2022

ryanmerolle closed this as completed Apr 27, 2022

rodvand reopened this Apr 27, 2022

ryanmerolle added the pending closure label Feb 25, 2023

This comment was marked as off-topic.

Sign in to view

ryanmerolle closed this as completed May 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: nb_inventory.py not working with Netbox 3.2.x #775

[Bug]: nb_inventory.py not working with Netbox 3.2.x #775

HardTy commented Apr 21, 2022

nomaster commented Apr 21, 2022 •

edited

Loading

rodvand commented Apr 22, 2022

ryanmerolle commented Apr 25, 2022

ryanmerolle commented Apr 26, 2022

HardTy commented Apr 27, 2022 •

edited

Loading

rodvand commented Apr 27, 2022

sc68cal commented Apr 27, 2022

ryanmerolle commented Feb 25, 2023

ryanmerolle commented Feb 25, 2023

pouriyabp commented May 6, 2023

This comment was marked as off-topic.

rodvand commented May 30, 2023

k4mil666 commented May 30, 2023

k4mil666 commented May 30, 2023

cleverer commented Jun 14, 2023

[Bug]: nb_inventory.py not working with Netbox 3.2.x #775

[Bug]: nb_inventory.py not working with Netbox 3.2.x #775

Comments

HardTy commented Apr 21, 2022

Ansible NetBox Collection version

Ansible version

NetBox version

Python version

Steps to Reproduce

Expected Behavior

Observed Behavior

nomaster commented Apr 21, 2022 • edited Loading

rodvand commented Apr 22, 2022

ryanmerolle commented Apr 25, 2022

ryanmerolle commented Apr 26, 2022

HardTy commented Apr 27, 2022 • edited Loading

rodvand commented Apr 27, 2022

sc68cal commented Apr 27, 2022

ryanmerolle commented Feb 25, 2023

ryanmerolle commented Feb 25, 2023

pouriyabp commented May 6, 2023

This comment was marked as off-topic.

rodvand commented May 30, 2023

k4mil666 commented May 30, 2023

k4mil666 commented May 30, 2023

cleverer commented Jun 14, 2023

nomaster commented Apr 21, 2022 •

edited

Loading

HardTy commented Apr 27, 2022 •

edited

Loading