Bad Documentation (if you ask me)

When you looks at the document of logrotate, the first page about the settings shows an example as follow:

/var/log/news/* {
    monthly
    rotate 2
    olddir /var/log/news/old
    missingok
    postrotate
        kill -HUP `cat /var/run/inn.pid`
    endscript
    nocompress
}

Making your think that it's okay to have an asterisk in this way, right? PLAINLY WRONG!

The logrotate utility actually has no heuristic to detect whether a file was already rotated or compressed so with an asterisk, it will discover all the files in a folder and view them all as current .log files.

In other words, it will fail very badly because it will take the already rotated files and rotate them. Then it will compress those. In other words, it will take a file named old-news.log and change it to:

old-news-log
old-news.log.1

On the following run, it will see both files so you end up with:

old-news.log
old-news.log.1
old-news.log.1.1
old-news.log.2.gz

Next time around, again, it takes all the files and rotate them all, no matter whether they are rotated files already:

old-news.log
old-news.log.1
old-news.log.1.1
old-news.log.1.2.gz
old-news.log.2.gz
old-news.log.2.gz.1
old-news.log.3.gz

See where we're going here? Somewhere in the land of let's multiple your files like crazy because it's a lot of fun (or something like that, at least...)

I have a computer which had that problem and it generated some 3.4 million log files, most of which were empty. Starting with four files accelerated the process to the millions. Ouch!

Whenever logrotate runs, it first reads all the filenames of a folder before processing any one of them. So it was stuck reading those 3 million files for a few hours and then it would rotate all of those millions in another set of millions. Fun fun fun!

How do you fix that one?

Well... assuming you know the extension of your files, you put that extension in the wildcard. For example, if all your log files end with .log, you only want to rotate those with the *.log pattern.

/var/log/news/*.log { ... }

Without that pattern, you're going to get a CPU running for pretty much nothing. If your logs are large, also, that's going to be double the trouble. In my case, most of the files were very small (like 50Kb for the large ones, before compression, and most of the others were empty.)

In my case, though, I have multiple extensions. What you can do in that case, is put one wild card per line as in:

/var/log/mail/*.err
/var/log/mail/*.info
/var/log/mail/*.log
/var/log/mail/*.warn
{
   ... # your logrotate rules go here
}

Once that was fixed, logrotate ran in no time. Like with full debug, etc. it ran through al the files in less than 2 seconds!