March 15th, 2014

Protocols in ClojureScript

Protocols are Clojure's mechanism for specifying re-usable type interfaces.

Protocols are used heavily in Clojure and ClojureScript?you can find them in many popular libraries. As with many developers, lately I've been using David Nolen's React wrapper library Om. It makes heavy use of Protocols for both its own implementation and in some suggested usage patterns. And recently I've had occasion to make good use of Protocols for other purposes in my own projects. So now seemed like a good time to write a post giving an overview of what Protocols are and describing some simple usage patterns for those just coming into Clojure or ClojureScript.

(I will use "Clojure" as an all-encompassing label for Clojure and ClojureScript, but I will take care to differentiate between Clojure as hosted on the JVM and ClojureScript targeting JavaScript when I need to distinguish between implementation-specific features.)

What are Protocols?

So I took a step back and said, what part of Java did I need in order to implement Clojure and its data structures, what could I do without, and what semantics was I willing to support - for Clojure - i.e. not in terms of interop. What I ended up with was - a high-performance way to define and implement interfaces. What I explicitly left out was - concrete derivation and implementation inheritance. (Rich Hickey talking about introducing Protocols into Clojure)

Coming into Clojure with a Ruby background, the most obvious analog for me when first learning about Protocols was Ruby mixins. Protocols are also comparable to Java Interfaces. As with Mixins and Interfaces, Protocols provide a light-weight way provide a type with an interface?and because we are working with Clojure, in this case "an interface" means, simply, a set of function signatures.

(defprotocol IFoo
  (foo [this])
  (bar [this]))

Protocols give us a way to do polymorphic type dispatch in Clojure.

(deftype Foo1 []
  IFoo
  (foo [this] (println "Foo1"))
  (bar [this] (println "Bar1")))

(deftype Foo2 []
  IFoo
  (foo [this] (println "Foo2"))
  (bar [this] (println "Bar2")))

=> (foo (Foo1.))
Foo1
nil
=> (foo (Foo2.))
Foo2
nil

As you see I declare a type which implements the Protocol via deftype (this is not the only way?I can also extend the Protocols with defrecord or with reify?I'll describe these further below).

Protocols provide a solution to the expression problem?

The Expression Problem is a new name for an old problem. The goal is to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts). (Philip Wadler, 12 November 1998*)

Here is a quick example of how Protocols allow you to provide new functions over previously defined datatypes?

(deftype Foo [])
(deftype IncompatibleBar [])

(defprotocol ExpressYourself
  (foos-and-bars-in-harmony [this]))

(extend-type Foo
  ExpressYourself
  (foos-and-bars-in-harmony [this] "I am a foo!"))

(extend-type IncompatibleBar
  ExpressYourself
  (foos-and-bars-in-harmony [this] "I am an incompatible bar!"))

=> (foos-and-bars-in-harmony (Foo.))
"I am a foo!"
=> (foos-and-bars-in-harmony (IncompatibleBar.))
"I am an incompatible bar!"

This is an admittedly very simple example, but shows how simple it is to provide new functions on a type, after that type has already been defined. (For a deeper dive into the Expression Problem and how Clojure Protocols provide a solution, check out this piece.)

This is a very powerful mechanism. I often take advantage of Protocols when I know that an underlying implementation is going to need to change, but I can be confident that the interface will remain consistent (I suppose you can call this dependency injection).

For example, say we have a function for dealing with sushi.

(defprotocol Sushi
  (wash-my-hands [this])
  (eat-sushi [this]))

(defn deal-with-sushi [eater]
  (wash-my-hands eater)
  (eat-sushi eater))

(deftype NeverEatenSushiBeforeAmerican []
  Sushi
  (wash-my-hands [this]
    (println "I would if I was actually going to eat that raw fish."))
  (eat-sushi [this]
    (println "Well, I guess I'll try the salmon")))

(deftype American []
  Sushi
  (wash-my-hands [this]
    (println "Er, where's the sink?"))
  (eat-sushi [this]
    (println "Hey, I'm hip, I'll use chopsticks!")))

(deftype Japanese []
  Sushi
  (wash-my-hands [this]
    (println "Oshibori for the win."))
  (eat-sushi [this]
    (println "Actually dude we just use our fingers.")))

=> (deal-with-sushi (NeverEatenSushiBeforeAmerican.))
I would if I was actually going to eat that raw fish.
Well, I guess I'll try the salmon

=> (deal-with-sushi (American.))
Er, where's the sink?
Hey, I'm hip, I'll use chopsticks!

=> (deal-with-sushi (Japanese.))
Oshibori for the win.
Actually dude we just use our fingers.

Then later on we can define another type implementing the same protocol...and we need change nothing in our original deal-with-sushi function.

(deftype Dog []
  Sushi
  (wash-my-hands [this]
    (println "Not even sure why I implemented this function"))
  (eat-sushi [this]
    (println "Get food in mouth now")))

=> (deal-with-sushi (Dog.))
Not even sure why I implemented this function
Get food in mouth now

defrecord and reify

defrecord can be thought of as an augmentation of deftype which provides some built-in hash-map like features (you can see that this is more or less what defrecord is by examining the Clojure and ClojureScript codebases). For example, it provides a bit of syntactic sugar out of the box so that you can refer to fields in a hash-map like fashion.

(defrecord Foo3 [field1 field2]
  IFoo
  (foo [this] (println "Foo3, field1: " field1))
  (bar [this] (println "Bar3, field2: " field2)))

(def foo3 (Foo3. 1 2))

=> (:field1 foo3)
1
=> (:field2 foo3)
2
=> (foo foo3)
Foo3, field1:  1
nil
=> (bar foo3)
Bar3, field2:  2
nil

reify provides a way to generate an instance of an anonymous type which implements a Protocol or set of Protocols. These can be handy when you need a custom one-off implementation of some Protocol but don't care about storing the type for re-use.

(def quick-foo
  (reify
    IFoo
    (foo [this] (println "Quick, gimme a foo!"))))

=> (foo quick-foo)
Quick, gimme a foo!
nil

You can see this used in the examples in Om, for example.

(defn mouse-view [app owner]
  (reify
    om/IWillMount
    (will-mount [_]
      (let [mouse-chan
            (async/map
              (fn [e] [(.-clientX e) (.-clientY e)])
              [(listen js/window EventType/MOUSEMOVE)])]
        (go (while true
              (om/update! app :mouse (<! mouse-chan))))))
    om/IRender 
    (render [_]
      (dom/p nil
        (when-let [pos (:mouse app)]
          (pr-str (:mouse app)))))))

We can also add Protocols to a previously defined type using extend-type, as in the Expression Problem examples above.

(defprotocol IFoo
  (foo [this]))

(deftype FooItAgain []
  IFoo
  (foo [this] (println "same old foo.")))

(defprotocol INewFoo
  (a-new-fn [this])

(extend-type FooItAgain
  INewFoo
  (a-new-fn [this] (println "realized I needed this!")))

=> (foo (FooItAgain.))
same old foo.
nil
=> (a-new-fn (FooItAgain.))
realized I needed this!
nil

...and extend-protocol is simply a bit of syntactic sugar which allows us to add a protocol to a number of types at once.

;;
;; ...continued from above.
;;

(deftype YetAnotherFoo []
  IFoo
  (foo [this] (println "yep.")))

(deftype FooMeToTheMoon []
  IFoo
  (foo [this] (println "...")))

(extend-protocol INewFoo
  YetAnotherFoo
  (a-new-fn [this] (println "need it here too."))

  FooMeToTheMoon
  (a-new-fn [this] (println "...and here.")))

=> (a-new-fn (YetAnotherFoo.))
need it here too.
nil
=> (a-new-fn (FooMeToTheMoon.))
...and here.
nil

Reusing Implementations: Differences Between Clojure and ClojureScript

With Clojure on the JVM we have a simple facility to re-use implementations via the extend function.

(defprotocol ILister
  (list-forwards [this])
  (list-backwards [this]))

(def base-fns
  {:list-forwards
   (fn [this] (println "forwards: " (val (first this))))

   :list-backwards
   (fn [this] (println "backwards: " (-> this first val reverse)))})

(defrecord ListLister [l])
(defrecord StringLister [s])

(extend ListLister ILister base-fns)

(extend StringLister
  ILister
  (assoc base-fns
    :list-backwards
    (fn [this]
      (println "backwards: " (clojure.string/join "" (-> this first val reverse))))))

(def ll (ListLister. '(1 2 3 4)))
(def sl (StringLister. "My string."))

=> (list-forwards ll)
forwards:  (1 2 3 4)
nil
=> (list-forwards sl)
forwards:  My string.
nil
=> (list-backwards ll)
backwards:  (4 3 2 1)
nil
=> (list-backwards sl)
backwards:  .gnirts yM
nil

Unfortunately, we don't have extend in ClojureScript. However, with recent versions of ClojureScript (since r2138) we have the specify function. This lets us create an extension of a named instance (as opposed to an anonymous instance, as with reify) of a ClojureScript type. We can create a solution similar to the Clojure extend example above using factory-style functions built with specify:

(defprotocol IForwardsLister
  (list-forwards [this]))

(defprotocol IBackwardsLister
  (list-backwards [this]))

(defrecord DefaultForwardsLister [coll]
  IForwardsLister
  (list-forwards [this] (println "forwards: " coll)))

(defn make-string-lister [s]
  (specify (DefaultForwardsLister. s)
    IBackwardsLister
    (list-backwards [this]
      (println "backwards: " (clojure.string/join "" (-> this first val reverse))))))

(defn make-list-lister [l]
  (specify (DefaultForwardsLister. l)
    IBackwardsLister
    (list-backwards [this]
      (println "backwards: " (clojure.string/join "" (-> this first val reverse))))))

(def sl (make-string-lister "My string."))
(def ll (make-list-lister '(1 2 3 4)))

=> (list-forwards sl)
forwards:  My string.
nil
=> (list-forwards ll)
forwards:  (1 2 3 4)
nil
=> (list-backwards sl)
backwards:  .gnirts yM
nil
=> (list-backwards ll)
backwards:  4321

A better example can be seen again in Om's source code.

(defn ^:private to-cursor* [val state path]
  (specify val
    IDeref
    (-deref [this]
      (if-not *read-enabled*
        (get-in @state path)
        (throw (js/Error. (str "Cannot deref cursor during render phase: " this)))))
    ICursor
    (-path [_] path)
    (-state [_] state)
    ITransact
    (-transact! [this korks f]
      (transact* state (into path korks) f this))
    IEquiv
    (-equiv [_ other]
      (check
        (if (cursor? other)
          (= val (-value other))
          (= val other))))))

I should also note that there is a mutating version of specify, appropriately named specify!, should you need such a thing?I'll leave it as an exercise for the reader to determine when that may be appropriate.

There is one more method I will mention available for providing extendable default implementations in ClojureScript?simply extending the default Object Protocol (in Clojure you can use java.lang.Object). Re-working the above example again?

(defprotocol ILister
  (list-forwards [this])
  (list-backwards [this]))

(extend-type js/Object
  ILister
  (list-forwards [this]
    (println "forwards: " (-> this first val)))
  (list-backwards [this]
    (println "backwards: " (-> this first val reverse))))

(defrecord ListLister [l] ILister)

(defrecord StringLister [s]
  ILister
  (list-backwards [this]
    (println "backwards: " (clojure.string/join "" (-> this first val reverse)))))

(def sl (StringLister. "My string."))
(def ll (ListLister. '(1 2 3 4)))

=> (list-forwards sl)
forwards:  My string.
nil
=> (list-forwards ll)
forwards:  (1 2 3 4)
nil
=> (list-backwards sl)
backwards:  .gnirts yM
nil
=> (list-backwards ll)
backwards:  (4 3 2 1)

While this is closer to the Clojure example in elegance, modifying core language features is something I tend to shy away from on principle. Despite it being slightly clunkier I think I would probably choose the specify solution before this.

Where To Now?

There is a lot left to be said about Protocols, but hopefully this gives you a basic idea of how they are used and illustrates some of the idiosyncracies involved when using them in ClojureScript.

For further study, I recommend reading the chapter "Datatypes and Protocols" from the book Clojure Programming. It provides a number of helpful examples of Protocol implementations, and goes into some detail regarding JVM-specific optimizations and performance concerns.

Additionally, Chapter 9, "Combining data and code" from The Joy of Clojure also provides a great introduction to Protocols, but takes a different tack by contextualizing them in a discussion of Namespaces and the other method Clojure itself provides for powerful type-dispatch, multimethods. The soon-to-be-released second edition of The Joy of Clojure also includes some discussion of ClojureScript-specific details of Protocols in it's chapter "Why Clojurescript"?I recommend checking it out.

Last but not least, it is worth reading through the clojure.org documentation for Protocols and Datatypes for a concise (albeit dense) summary of the major concepts involved and philosophy behind Protocols and Datatypes in Clojure. It is also worth reading Rich Hickey's original email to the Clojure mailing list introducing Datatypes and Protocols for some more context and how these features fit into Clojure overall.