Building an Activity Feed System with Storm

One of many wonderfully functional recipes from the Clojure Cookbook

clj_cookbookEditor’s Note: The Clojure Cookbook is a recently published book by experienced Clojurists Luke VanderHart and Ryan Neufeld. It seeks to be a practical collection of tasks for intermediate Clojure programmers. In addition to providing their own recipes, Ryan and Luke accepted contributions from a number of people in the community. One of those contributors was Travis Vachon–in this excerpt from the Cookbook, Travis gives you a tried and true recipe for working with Clojure and Storm.


You want to build an activity stream processing system to filter and aggregate the raw event data generated by the users of your application.


Streams are a dominant metaphor for presenting information to users of the modern Internet. Used on sites like Facebook and Twitter and mobile apps like Instagram and Tinder, streams are an elegant tool for giving users a window into the deluge of information generated by the applications they use every day.

As a developer of these applications, you want tools to process the firehose of raw event data generated by user actions. They must offer powerful capabilities for filtering and aggregating data and must be arbitrarily scalable to serve ever-growing user bases. Ideally they should provide high-level abstractions that help you organize and grow the complexity of your stream-processing logic to accommodate new features and a complex world.

Clojure offers just such a tool in Storm, a distributed real-time computation system that aims to be for real-time computation what Hadoop is for batch computation. In this section, you’ll build a simple activity stream processing system that can be easily extended to solve real-world problems.

First, create a new Storm project using its Leiningen template:

In the project directory, run the default Storm topology (which the lein template has generated for you):

This generated example topology just babbles example messages incoherently, which probably isn’t what you want, so begin by modifying the “spout” to produce realistic events.

Read more…

Comment |

Controlling Drones with Clojure

OSCON 2013 Speaker Series

Carin Meier (@carinmeier) is Artisan at Neo, Founder of Gigsquid Software and OSCON 2013 Speaker. In this interview we talk about her love of Clojure and how she created a library to control her AR drone with it!

Key highlights include:

  • Clojure, a modern Lisp? [Discussed at 0:20]
  • Immutable data structures make Clojure powerful [Discussed at 1:01]
  • Yes, you can program an AR drone in Clojure [Discussed at 2:04]
  • But, how do you get started? It just takes three lines of code [Discussed at 3:47]

For the code behind the drone and Carin’s language Babar (inspired by Elephant 2000) check out her github page.

You can view the full interview here:

Read more…

Comment |
Clojure’s advantage: Immediate feedback with REPL

Clojure’s advantage: Immediate feedback with REPL

Chas Emerick on how Clojure can make a difference to developers.

REPL is built into Clojure, and you can connect to any running Clojure process and modify and execute code. In this interview, "Clojure Programming" co-author Chas Emerick discusses the possibilities this introduces for Clojure developers.

Comment |
Editorial Radar: Functional languages

Editorial Radar: Functional languages

The benefits of functional languages and functional language techniques.

O'Reilly editors Mike Loukides and Mike Hendrickson discuss the advantages of functional programming languages and how functional language techniques can be deployed with almost any language.

Comment: 1 |
A rough guide to JVM languages

A rough guide to JVM languages

Java is as much about the JVM as it is the language.

This overview of JVM-based programming compares the relative strengths of the major languages.

Comments: 11 |
Clojure: Lisp meets Java, with a side of Erlang

Clojure: Lisp meets Java, with a side of Erlang

Stuart Sierra on why Clojure is catching on.

Stuart Sierra digs into Clojure: what it is, how it works, and why it's attracting Java developers.

Comments Off |