[BUG] First-run of new logrotate hourly job evaluates necessity incorrectly, so rotation never actually works.

[mdschmitt]

Your setup

Formula commit hash / release tag

tag: v0.13.2

Versions reports (master & minion)

Master:
Salt Version:
          Salt: 3004.1

Dependency Versions:
          cffi: 1.14.0
      cherrypy: Not Installed
      dateutil: 2.8.0
     docker-py: 4.0.2
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.2
       libgit2: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 0.5.6
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.20
      pycrypto: Not Installed
  pycryptodome: 3.6.1
        pygit2: 1.7.1
        Python: 3.7.10 (default, Jun  3 2021, 00:02:01)
  python-gnupg: Not Installed
        PyYAML: 5.3.1
         PyZMQ: 17.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.2.3

System Versions:
          dist: amzn 2
        locale: UTF-8
       machine: x86_64
       release: 4.14.177-139.254.amzn2.x86_64
        system: Linux
       version: Amazon Linux 2

Minion:

Salt Version:
          Salt: 3004.1

Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: Not Installed
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 0.5.6
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.6.1
        pygit2: Not Installed
        Python: 3.7.10 (default, Jun  3 2021, 00:02:01)
  python-gnupg: Not Installed
        PyYAML: 4.2
         PyZMQ: 17.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.2.3

System Versions:
          dist: amzn 2
        locale: UTF-8
       machine: x86_64
       release: 4.14.273-207.502.amzn2.x86_64
        system: Linux
       version: Amazon Linux 2

Pillar / config used

---
logrotate:
  jobs:
    my_app:
      path:
        - {{ global.dirs.logs }}/myapp-stdout.log
      config:
        - hourly
        - missingok
        - rotate 720
        - dateext
        - dateformat .%Y-%m-%d-%H00
        - compress
        - compresscmd /usr/bin/zstd
        - compressext .zst
        - compressoptions -9 -T1 --long
        - uncompresscmd /usr/bin/unzstd
        - notifempty
        - olddir {{ global.dirs.old_logs }}
        - sharedscripts
        - postrotate
        - pkill --signal USR1 --euid {{ global.users.appuser }} --full "app-process"
        - endscript

---

Bug details

Describe the bug

When running the logrotate formula with an hourly logrotate job, everything is laid down correctly but if rotation is not forced to run the first time (i.e. "initialize" the job) it stays stuck in a state of returning log does not need rotating (log has been already rotated) when evaluating status forever, thus filling up the disk space and crashing my app. :(

I think this has something to do with the /var/lib/logrotate/logrotate.hourly.status file not already existing before the job is populated.

Steps to reproduce the bug

Pretty much just "lay down logrotate for a fresh application with an hourly config option, watch to see if logrotate -d /etc/logrotate.d/job_name ever returns log needs rotating, even after it should rotate the log. In my experience, it never does.

Expected behaviour

Logfiles rotate, even if they have never done so before.

Attempts to fix the bug

The only thing that fixes this is manually running logrotate -f /etc/logrotate.d/my_app because that forces rotation and after that, the status file is created/set correctly.
That, or adding a -f flag to the actual logrotate line in /etc/cron.daily/logrotate (or wherever your system has the cronjob definition).

Additional context

n/a

[mdschmitt]

Hm, I think this has to do with the fact that a custom logrotate status file is used for the hourly jobs. If that file has nothing in it to begin with, logrotate doesn't know what has or has not happened, so it defaults to evaluating jobs as "not needing rotation" even though they should get rotated. The one-time addition of the -f flag forces the job and thereby creates the needed statusfile with contents that are then valid and all successive runs succeed.