terrarum

Introduction

Sébastien Han posted a good article on how to use rsyslog with OpenStack. Coincidentally, I recently configured rsyslog for an OpenStack project and figured I'd publish my configuration, too.

• Introduction
• Configuring rsyslog
  • Configuring the Server
  • Configuring the Client
• OpenStack and syslog
• Sorting and Organizing
• Conclusion

Configuring rsyslog

I have three puppet manifests that I use to configure rsyslog:

• base.pp
• client.pp
• server.pp

base.pp is simply declared in both client.pp and server.pp.

Configuring the Server

server.pp is used to configure a server to be an rsyslog-server. To apply it, simply do:

class { 'admin::rsyslog::server': }

Although that seems simple enough, the heavy work is in the two files that the manifest uses:

• cloud-rsyslog.logrotate is a script used by logrotate to, of course, rotate the logs.
• server.conf.erb is an rsyslog configuration file that has a tremendous amount of stuff going on. I'll describe this later on.

Configuring the Client

Client configuration is just a simple as server configuration:

class { 'admin::rsyslog::client':
  server => 'rsyslog.example.com',
}

OpenStack and syslog

As Sébastien described, each OpenStack component can be configured to send their logs to syslog. You can see this being done in the following manifests (search for "syslog"):

• compute node
• cloud controller

For Swift, the proxy-logging middleware sends the logs to syslog.

Sorting and Organizing

When an OpenStack component sends a log to the rsyslog server, the log is passed through the filters in the server.conf file. Based on if the log is from Swift (special case), the log facility, and the hostname, the log is directed to a specific log file in a specific directory.

I have logs going into the following directories:

• /var/log/rsyslog/%HOSTNAME%/nova.log: all logs related to nova from that host.
• /var/log/rsyslog/%HOSTNAME%/glance.log: all logs related to glance from that host.
• /var/log/rsyslog/%HOSTNAME%/cinder.log: all logs related to cinder from that host.
• /var/log/rsyslog/%HOSTNAME%/keystone.log: all logs related to keystone from that host.
• /var/log/rsyslog/%HOSTNAME%/swift.log: all logs related to nova from that host.
• /var/log/rsyslog/%HOSTNAME%/syslog.log: all other logs from that host.
• /var/log/rsyslog/nova.log: all nova logs from all hosts.
• /var/log/rsyslog/swift.log: all swift logs from all hosts.
• /var/log/rsyslog/cinder.log: all cinder logs from all hosts.

Conclusion

rsyslog provides a great deal of flexibility when it comes to logging. Puppet, as usual, comes to the rescue in making the complexities of such services manageable.

While this solution works for me, I have plans to revisit OpenStack logs in combination with logstash in the future.