[BUG] 3005 Does not Support Napalm 4.x

[ITJamie]

Description
Latest dev build from this pipeline: https://gitlab.com/saltstack/open/salt-pkg/-/pipelines/613600623
napalm proxy grains is throwing an error related to napalm.version

This is a follow on from #62393

Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)
Latest dev build from this pipeline: https://gitlab.com/saltstack/open/salt-pkg/-/pipelines/613600623
a working device to connect to salt-proxy

• on-prem machine
• VM (Virtualbox, KVM, etc. please specify)
• VM running on a cloud service, please be explicit and add details
• container (Kubernetes, Docker, containerd, etc. please specify)
• or a combination, please be explicit
• jails if it is FreeBSD

Steps to Reproduce the behavior
https://github.com/napalm-automation/napalm-salt/tree/master/saltstack - a quick example of a salt setup with napalm
install napalm with salt pip (salt-pip install napalm)
Start a proxy module and watch /var/log/salt/proxy. the following error will appear

2022-08-15 19:46:58,096 [salt.loader      :1156][CRITICAL][1065324] Failed to load grains defined in grain file napalm.version in function <LoadedFunc name='napalm.version'>, error:
Traceback (most recent call last):
  File "salt/loader/__init__.py", line 1150, in grains
    ret = funcs[key](**kwargs)
  File "salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "salt/loader/lazy.py", line 1228, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "salt/loader/lazy.py", line 1243, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/opt/saltstack/salt/run/salt/grains/napalm.py", line 156, in version
    return {"version": _get_grain("os_version", proxy=proxy)}
  File "/opt/saltstack/salt/run/salt/grains/napalm.py", line 98, in _get_grain
    grains = _retrieve_grains_cache(proxy=proxy)
  File "/opt/saltstack/salt/run/salt/grains/napalm.py", line 71, in _retrieve_grains_cache
    GRAINS_CACHE = proxy["napalm.get_grains"]()
  File "salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "salt/loader/lazy.py", line 1228, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "salt/loader/lazy.py", line 1243, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/opt/saltstack/salt/run/salt/proxy/napalm.py", line 258, in get_grains
    return call("get_facts", **{})
  File "/opt/saltstack/salt/run/salt/proxy/napalm.py", line 348, in call
    __context__["napalm_device"]["network_device"], method, *args, **kwargs
  File "salt/loader/context.py", line 78, in __getitem__
    return self.value()[item]
  File "salt/utils/context.py", line 221, in __getitem__
    return self._dict()[key]
KeyError: 'napalm_device'

Expected behavior
No errors related to proxy process

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Salt Version:
          Salt: 3005rc2+86.g54e2c9fa06

Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.0
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
        Python: 3.9.13 (main, Aug 15 2022, 10:08:44)
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: debian 11 bullseye
        locale: utf-8
       machine: x86_64
       release: 5.10.0-12-amd64
        system: Linux
       version: Debian GNU/Linux 11 bullseye

[s0undt3ch]

This issue is confirmed but it's not related to the onedir builds of Salt.

Currently, Salt does not support napalm 4.x.

If napalm 3.x is installed, this issue does not happen.

[mirceaulinic]

I haven't tested 3005 thoroughly yet, but I'm seeing a similar (if not the exact same) error in 3004.2. The Proxy Minion starts correctly however, it just seems like this is caused by the chicken-and-egg problem PMs have, and the error is thrown when the proxy isn't fully loaded yet, so it's just cosmetic. In my environment I've worked around this by patching the napalm.py proxy module:

diff --git a/salt/proxy/napalm.py b/salt/proxy/napalm.py
index 7e81ded4d5..bed6ae456e 100644
--- a/salt/proxy/napalm.py
+++ b/salt/proxy/napalm.py
@@ -248,6 +248,8 @@ def get_device():
     """
     Returns the network device object.
     """
+    if "napalm_device" not in __context__:
+        return {}
     return __context__["napalm_device"]["network_device"]
 
 
@@ -344,6 +346,8 @@ def call(method, *args, **kwargs):
         # thus the NAPALM methods will be called with their defaults
         if warg is None:
             kwargs.pop(karg)
+    if "napalm_device" not in __context__:
+        return {"result": False, "comment": "Not initialised yet", "out": None}
     return salt.utils.napalm.call(
         __context__["napalm_device"]["network_device"], method, *args, **kwargs
     )

[ichilton]

Hi,

Currently, Salt does not support napalm 4.x.
If napalm 3.x is installed, this issue does not happen.

Actually, I seem to be getting messages like this with napalm 3.3.1...

  File "/usr/lib/python3/dist-packages/salt/proxy/napalm.py", line 258, in get_grains
    return call("get_facts", **{})
  File "/usr/lib/python3/dist-packages/salt/proxy/napalm.py", line 348, in call
    __context__["napalm_device"]["network_device"], method, *args, **kwargs
  File "/usr/lib/python3/dist-packages/salt/loader/context.py", line 78, in __getitem__
    return self.value()[item]
  File "/usr/lib/python3/dist-packages/salt/utils/context.py", line 221, in __getitem__
    return self._dict()[key]
KeyError: 'napalm_device'

>>> import napalm
>>> napalm.__version__
'3.3.1'