Today I noticed that my fail2ban wasn't running.

Looking into it, I could see that an error was happening on startup. This pretty much always mean that something's wrong in the configuration files.

sudo systemctl start fail2ban
Job for fail2ban.service failed because the control process exited with error code.
See "systemctl status fail2ban.service" and "journalctl -xe" for details.

However, somehow, systemd would not show me the error with the status or checking the journal and it would not write anything to the /var/log/fail2ban.log file which stayed totally empty.

prompt % systemctl status fail2ban
● fail2ban.service - Fail2Ban Service
   Loaded: loaded (/lib/systemd/system/fail2ban.service; enabled; vendor preset: enabled)
   Active: inactive (dead) (Result: exit-code) since Sat 2017-05-27 03:30:42 UTC; 9min ago
     Docs: man:fail2ban(1)
  Process: 15483 ExecStart=/usr/bin/fail2ban-client -x start (code=exited, status=255)

May 27 03:30:42 m2osw.com systemd[1]: fail2ban.service: Control process exited, code=exited status=255
May 27 03:30:42 m2osw.com systemd[1]: Failed to start Fail2Ban Service.
May 27 03:30:42 m2osw.com systemd[1]: fail2ban.service: Unit entered failed state.
May 27 03:30:42 m2osw.com systemd[1]: fail2ban.service: Failed with result 'exit-code'.
May 27 03:30:42 m2osw.com systemd[1]: fail2ban.service: Service hold-off time over, scheduling restart.
May 27 03:30:42 m2osw.com systemd[1]: Stopped Fail2Ban Service.
May 27 03:30:42 m2osw.com systemd[1]: fail2ban.service: Start request repeated too quickly.
May 27 03:30:42 m2osw.com systemd[1]: Failed to start Fail2Ban Service.
May 27 03:36:50 m2osw.com systemd[1]: Stopped Fail2Ban Service.

In order to find the configuration error, I instead had to run the start process by hand. So first I made sure it was stopped, and then I ran the start command as found in the /etc/systemd/system/multi-user.target.wants/fail2ban.service file:

prompt % sudo /usr/bin/fail2ban-client stop
ERROR  Failed to access socket path: /var/run/fail2ban/fail2ban.sock. Is fail2ban running?

And then:

prompt % sudo /usr/bin/fail2ban-client -x start
ERROR  No file(s) found for glob /var/log/mail.log
ERROR  Failed during configuration: Have not found any log file for postfix jail

Now I can see why and where the error is happening.

We moved all the mail logs under /var/log/mail/... so the path needed to be updated to /var/log/mail/mail.log.

Once that was done, the normal systemctl command worked as expected:

prompt % sudo systemctl start fail2ban

Finally, this did not generate any error.