11 October 2013

How to test your Logstash configuration

When pushing more and more types of events to Logstash, the configuration easily became quite complex and hard to manage. It's easy to make both syntax and logical errors. So, testing your logstash configuration before bringing it to your server is a good thing.

Step 1: Syntax check

We start with a quick win: see if the configuration file you created is valid. Logstash has a built-in command option for it:

java -jar logstash.jar agent --configtest --config <yourconfigfile>

It's there since 1.1.10. To avoid mistakes, you can add it to your init script.

Step 2: logstash rspec

For logical errors, logstash rspec comes to help:

java -jar logstash.jar rspec <your spec file>

Cool to see find out those nifty, slightly hidden features of Logstash!

How do you write those tests? Let's take an example:

require "test_utils"

describe "haproxy logs" do
  extend LogStash::RSpec

  config <<-CONFIG
  filter {
    grok {
       type            => "haproxy"
       add_tag         => [ "HTTP_REQUEST" ]
       pattern         => "%{HAPROXYHTTP}"
    date {
       type            => 'haproxy'
       match           => [ 'accept_date', 'dd/MMM/yyyy:HH:mm:ss.SSS' ]

  sample({'@message' => '<150>Oct 8 08:46:47 syslog.host.net haproxy[13262]: [08/Oct/2013:08:46:44.256] frontend-name backend-name/server.host.net 0/0/0/1/2 404 1147 - - ---- 0/0/0/0/0 0/0 {client.host.net||||Apache-HttpClient/4.1.2 (java 1. 5)} {text/html;charset=utf-8|||} "GET /app/status HTTP/1.1"',
          '@source_host' => '',
          '@type' => 'haproxy',
          '@source' => 'tcp://',

      }) do

    insist { subject["@fields"]["backend_name"] } == [ "backend-name" ]
    insist { subject["@fields"]["http_request"] } == [ "/app/status" ]
    insist { subject["tags"].include?("HTTP_REQUEST") }
    insist { subject["@timestamp"] } == "2013-10-08T06:46:44.256Z"
    reject { subject["@timestamp"] } == "2013-10-08T06:46:47Z"

You can see the pattern:

  1. Specify filter configuration. Filters are the most important (and easiest) to test.
  2. Add some sample input. This is what the filter would get from the input side. In this example the sample message is a haproxy log message.
  3. Write expectations for the sample input. In this example we test if the fields are grokked out correctly and if the timestamp has been extracted correctly from the accept_date field.

When you run the test, this is the output:

$ java -jar logstash.jar rspec --color example_spec.rb

Finished in 0.028 seconds
1 example, 0 failures

Cool! Of course you can test a whole configuration file instead of a snippet in the spec file:

describe "haproxy logs" do
  extend LogStash::RSpec

  file = File.open("path/to/etc/logstash_filters.conf", "rb")
  contents = file.read
  config  contents

  # sample block goes here

Make sure you leave the input { } and output { } parts out of the configuration files.

Bonus points: test your puppet-templated Logstash configuration file

I a next post, I will talk about how to test puppet templated configuration files.