sls with empty output leads to "requisites were not found"

[skandyla]

Hi,

In this example I have two states, which retrieve some data from pillar.
One state is depend from another.
The error occurs when the first state doesn't get any data from pillar, so it produces zero output, and the second state fails with The following requisites were not found: require: sls:

          ID: custompkgs_httpd
    Function: pkg.installed
        Name: httpd
      Result: False
     Comment: The following requisites were not found:
                                 require:
                                     sls: repos.custom

The first state:
salt/repos/custom.sls:

{% if grains['os'] == 'CentOS' %}                                                    
{% for yumrepo in salt['pillar.get']('yumrepos',{}) %}                               
{{ yumrepo }}_repo:                                                                  
  file.managed:                                                                      
    - name: /etc/yum.repos.d/{{ yumrepo }}.repo                                      
    - contents_pillar: yumrepos:{{ yumrepo }}                                        
    - mode: 600                                                                      
    - user: root                                                                     
{% endfor %}                                                                         
{% endif %}   

The second state:
salt/packages/init.sls:

include:
  - repos.custom

{% for package, status in pillar.get('custompkgs', {}).items() %}                    
custompkgs_{{ package }}:                                                            
  pkg:                                                                               
    - name: {{ package }}                                                            
    - {{ status }}
    - require:
      - sls: repos.custom
{% endfor %} 

The second state is depend from the first, because I quite often deploy my private repositories and install custom packages which are depend from that private repositories.

Test's pillar data for the host:

custompkgs:                                                                          
  httpd: installed

When the first state (repos.custom) provide some output (I mean {% for package, status in pillar.get('custompkgs', {}).items() %} ) even without any changes, everything works fine.

Fox example, if I add some data to the pillar:

yumrepos:                                                                           
  tmp: |                                                                            
    #tmp 

everything works good.

Versions:

# salt-ssh --versions
Salt Version:
           Salt: 2015.8.5

Dependency Versions:
         Jinja2: 2.7.3
       M2Crypto: 0.21.1
           Mako: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.7.0
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: centos 7.2.1511 Core
        machine: x86_64
        release: 3.10.0-327.3.1.el7.x86_64
         system: CentOS Linux 7.2.1511 Core

[jfindlay]

@skandyla, thanks for reporting. Since your first state expects pillar data by using contents_pillar, I would expect it to fail to render and thus not be available for the second state. Do you have an alternative expectation?

[skandyla]

@jfindlay,
actually contents_pillar is another issue there, not related to the bug, I'm talking about. Sure the state rendering fails if contents_pillar is absent. But in this example this state does a job only if pillar data yumrepos: exist. So, execution of a module doesn't pass to the inside of a loop, where contents_pillar is defined.

The first state works with or without any content in pillar.
It's some kind of state, which are applied to all of my hosts (70+). In case some data is defined at pillar, the state does a job. In other case, it does nothing. And it actually working that way.

salt/repos/custom.sls (short version)

{% for yumrepo in salt['pillar.get']('yumrepos',{}) %}
# do a job
{% endfor %}  

Example of launching this state for some host without any related data at pillar:

# time salt-ssh --wipe "example.com" state.sls repos.custom
example.com:

Summary for example.com
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1

# time salt-ssh --wipe  "example.com" pillar.get yumrepos
example.com:

The problem is the second state.
salt/packages/init.sls:

include:
  - repos.custom

{% for package, status in pillar.get('custompkgs', {}).items() %}                    
custompkgs_{{ package }}:                                                            
  pkg:                                                                               
    - name: {{ package }}                                                            
    - {{ status }}
    - require:
      - sls: repos.custom
{% endfor %} 

This state shows an error:

...
     Comment: The following requisites were not found:
                                 require:
                                     sls: repos.custom

But the repos.custom state is working fine on himself, it just do nothing in case of absent pillar data. The problem is require: - sls: repos.custom doesn't understand that.

Finally, I've get it working buy introducing some function outside for loop:

{% if grains['os'] == 'CentOS' %}                                                    

# this is dummy function. yum-utils is already present at my system.
# But this forces the state to produce some response code
# in case of absent pillar data at 'yumrepos'
yum-utils:    
  pkg.installed                                                                                       

{% for yumrepo in salt['pillar.get']('yumrepos',{}) %}                               
{{ yumrepo }}_repo:                                                                  
  file.managed:                                                                      
    - name: /etc/yum.repos.d/{{ yumrepo }}.repo                                      
    - contents_pillar: yumrepos:{{ yumrepo }}                                        
    - mode: 600                                                                      
    - user: root                                                                     
{% endfor %}                                                                         
{% endif %} 

In that case, the second state (packages/init.sls) is working good. It's find the require dependence to be resolved. (There are also no data at pillar)
But I don't think that this is a good solution.

I suppose there is some bug in sls rendering subsystem.

[jfindlay]

@skandyla, thanks for the extra explanation.

[kiemlicz]

Also suffering from this bug
Do you have any updates about this?

[OrangeDog]

I think I've got a similar issue. This is including https://github.com/saltstack-formulas/letsencrypt-formula via gitfs.

include:
  - letsencrypt

...
    - require:
      - sls: letsencrypt

    Comment: The following requisites were not found:
                                require:
                                    sls: letsencrypt

Salt 2018.3.4

Using sls: 'letsencrypt.*' works though.