prometheus_tail_monitor does not refresh deleted/rotated files.

[SarthakSahu]

We are using tail plugin with "pos_file_compaction_interval" option. With this option pos_file is getting updated by removing all deleted tailed file information.
We are using the prometheus_tail_monitor plugin to get the information of tailed files.

Issue: When the tailed files are getting deleted/rotated, even though the pos_file is updated, But metrics fluentd_tail_file_inode and fluentd_tail_file_position are not updated and still shows the information of deleted/rotated files. As a result of this the amount of data exposed by these metrics are increasing continuously.

Found a similar discussion in google groups.
https://groups.google.com/g/fluentd/c/KsoAoTp62dA

[archupriya3]

Any update on this issue?

[aggarwalShivani]

Hi, I am facing the same issue with td-agent-4.3.1-1.el8.x86_64 (fluent-plugin-prometheus-2.0.2, prometheus-client-2.1.0). Any updates on fixing this?

[archupriya3]

Any updates on this issue? We are facing the same issue with 4.4.2 td-agent rpm as well.

[moharana-subhashree]

Any updates on this issue? We are facing the same issue with 4.5.0 td-agent rpm.

[moharana-subhashree]

Any updates on this issue? We are facing the same issue with fluent-package-5.0.2-1 rpm.

[skumarp7]

We are also facing this issue with fluent-package-5.1.0-1 rpm. Any update on this issue?

[anupg336]

Any update on this issue? We are also seeing the same problem with the fluent-package-6.0.0-1 rpm.

[anupg336]

We are facing this issue with fluent-package-6.0.1-1 rpm also. Any update on this?

[Watson1978]

I think the root cause is that the Prometheus client library retains gauge metrics in memory indefinitely unless they are explicitly removed.
Currently, the prometheus_tail_monitor only adds or updates metrics for the files it is currently tailing, but it lacks the logic to clean up the metrics for files that have been rotated or deleted.