Semantic Logging

Insight Into the Business

Created by Henrik Feldt / @henrikfeldt

I am
@henrikfeldt

Software architect at
Intelliplan

Intelliplan

  • Recruitment, staffing and CRM software
  • 7 hard-core developers
  • ~1k tx/s normal load
  • Functional Programming (F#, Clojure) and C#

Aims with Logging

Why log?

  • Debugging/correcting
  • Trending
  • Insight

What is Insight?

  1. Planning – the Future
  2. Action – the Present
  3. Reaction – inevitable changes

Business Value

  • Value through features, performance, usability
  • Create easy to use GUIs: user experience

How to have Insight?

With semantic logging

What is Logging? (1)

Metrics

+ Log Lines

+ HealthChecks

What is Logging? (2)

Messaging

Let's make assumptions!

Messaging = Eventing

The writing on the wall

Eventing is here to stay!

Logging = eventing

  • Who did what when?
See how often an operation is retried by the user

Logging -> Metrics

You can project metrics from log lines

Introducing Logary

https://github.com/logary

Logary Architecture (1)

Logary Architecture

Logary Architecture (2)

  • A target = an actor/agent
  • Non-intrusive, never throws exceptions
  • Fully async logging w/ deferral
  • Production for ~8 months

Normal Logging


"user sign-in"
|> Log.debugStrTag "authentication"
|> Log.setDatas [
  "user_id",     "34778"
  "token_thumb", "deafbeef"
]
|> Log.log logger
						

Metrics


"my.app.web.authentication.signin" |> Log.incr logger
"my.app.web.authentication.session_timeout" |> Log.incr logger

"my.app.fulfillment.order_shipped" |> Log.incr logger
("my.infra.queues.length", 10.) ||> Log.decrBy logger
						

Health Checks


let pingHafSe () = async {
  use p = new Ping()
  let awaitPong = Async.AwaitEvent(p.PingCompleted, p.SendAsyncCancel)
  try
    p.SendAsync("haf.se", 1000, obj())
    let! complete = awaitPong
    if complete.Cancelled then
      return Unhealthy <| Failure.Create("Ping was cancelled")
    elif not (complete.Error = null) then
      return Unhealthy <| Failure.Create("Ping got error", Some <| complete.Error)
    else return Healthy
  with e ->
      return Unhealthy <| Failure.Create("Exception when pinging haf.se", Some(e)) }

let h = fromFn "ping haf.se" pingHafSe
let gotUnhealthy = untilPred 10000 <| fun i -> match h.Check () with Unhealthy _ -> true | _ -> false

C# Support


var x = LogaryFactory.New("Logary Specs",
        with => with
            .Target<Graphite.Builder>(
                "graphite",
                conf => conf.Target.ConnectTo("127.0.0.1", 2131)
            )
            .Target<Debugger.Builder>("debugger")
            .Target<Logstash.Builder>(
                "ls",
                conf => conf.Target
                    .Hostname("localhost")
                    .Port(1936)
                    .EventVersion(Logstash.EventVersion.One)
                    .Done())
        );
           	

Putting it together

  1. Instrument your code
  2. Put up your infrastructure today!
  3. Analyse users' behaviour
  4. Build features they want and need
  5. Profit!