cl-fix

Table of Contents

1. Overview

NOTE: cl-fix is under active development. It is NOT recommended for production use yet, though it is ready for some basic testing and early feedback. See 'Current Status' below.

cl-fix enables you to write FIX-based systems using Common Lisp. FIX (Financial Information eXchange) is a protocol that underpins the communications between many major players in the finance and capital markets space.

This library will enable you to write anything from trading bots wired up to a broker, to simulators for testing other FIX-based systems (e.g. simulating a broker or exchange's endpoint). The primary value-add in using a language such as Common Lisp is that you'll be able to iterate and prototype quicker in Lisp than in other languages.

cl-fix is NOT an ultra-low-latency library suitable for HFT. Performance will be optimised in later rounds of development. First and foremost design goal is to have you iterating quickly against FIX endpoints.

There are 2 major functions that this library offers:

  1. Define FIX dictionaries. The dictionary macros define classes and de/serialisation functions for defined messages.
  2. Establish FIX sessions with remote endpoints to handle messages. That is, initiate or accept FIX connections and implement workflows using your dictionaries.

2. Current status

As stated above, cl-fix is under active development. Before I'd consider it production-grade, I want to complete (at least) the following:

  • Run a cl-fix based system for at least one whole trading week against my broker
  • Basic performance tests and baselining

Interested in helping out? See the contributor guide!

3. Key links

Item Link
Project home page https://fix9.net/cl-fix/
Source code https://gitlab.com/fix9/cl-fix
Discussions https://groups.io/g/cl-fix

4. Installation

Firstly, install the fix9 distribution into quicklisp:

(ql-dist:install-dist "http://fix9.net/dists/fix9.txt")

Then:

(ql:quickload '(:cl-fix :cl-fix/fix44))

Cl-fix is also included in the official quicklisp distribution, but, it only snaps it on a monthly basis. The fix9 dist is updated far more frequently, so must be added to ensure you have the latest updates. This will likely matter less in the future when the codebase stabilises and changes are less frequent.

To update your fix9 dist, just like with the official quicklisp dist, execute:

(ql:update-dist "fix9")

5. Dictionaries

Dictionaries are where you define the fields and messages your app will be exchanging with its peers. Dictionaries are the Google Protobuf equivalent to '.proto' files. Key difference from Protobufs is that we're using Lisp macros to do our definition, which in turn does our code-gen for us.

This section outlines how to define a dictionary, then use the serializers and deserializers that are generated for that dictionary.

5.1. Define a dictionary

Generally you'll derive your dictionary from the FIX specification for your endpoint. However, in the source, there is a standard FIX 4.4 dictionary which should get you 90% of the way there for many endpoints.

Rather than repeat the contents of that file here, you are encouraged to read through it and its comments. It is over here: ./dict/fix44.lisp.

Using the dictionary macros in cl-fix results in a few functions within that namespace:

  • SERIALIZE-FIX function: serializes a message instance to a stream for this dictionary.
  • READ-FIX function: reads one FIX message from a given stream and returns it in its object form for this dictionary.
  • STR Returns a string representation of a given message for this dictionary.

5.2. Serializer/Deserializer usage

(ql:quickload '(:cl-fix :cl-fix/fix44))

;; Serialize
(fix44:str (util:make-fix-msg :logon
                              :msg-seq-num 45
                              :sender-comp-id "me"
                              :target-comp-id "them"
                              :sending-time (local-time:now)
                              :username "bob"
                              :password "secret"))
;; Deserialize
(with-input-from-string (s *) (fix44:read-fix s))

6. Establishing sessions

Establishing a session is a matter of 3 broad steps:

  1. Set up your connector (either an initiator or acceptor)
  2. Set up your sessions, and add them to the connector
  3. Start your connector

This section walks you through how to do that.

6.1. Set up a connector

To establish a session, you will first need a CONNECTOR.

There are 2 connector types:

  1. An INITIATOR, which initiates an outgoing connection to an endpoint; and
  2. An ACCEPTOR, which accepts an incoming connection.

You can create an initiator as so:

(defparameter *connector* (make-instance 'cl-fix:initiator :host "some.example.host"
                                                           :port 8383
                                                           :dict-key :fix44))

6.2. Set up and add sessions

Next, add one or more sessions to your connector. Adding a session to an INITIATOR versus ACCEPTOR has slightly different semantics:

  • Adding a session to an INITIATOR means that upon connecting to the remote endpoint, each session will send its respective LOGON(A) message.
  • Adding a session to an ACCEPTOR means that when the connector receives an incoming connection, it expects to receive a LOGON(A) message, which will then be dispatched to the correct SESSION object.

Adding a session requires you to first create one, then add it to the corresponding connector. You'll also want to bind it to a message handler. You'll typically subclass the in-built SESSION class as follows (continuing from the above example):

;; Define a handler
(defun my-handler (session msg)
  (log:info "Received a ~A message" (util:fix-msg-type msg))
  (case (util:fix-msg-type msg)
    (:market-data-snapshot  (log:info "Yay - got some market data"))
    (:execution-report      (log:info "Yay - got an execution report"))
    (t                      (log:warn "Unhandled message: ~A" msg))))

;; Define session class defaults
(defclass my-session (cl-fix:session)
  ()
  (:default-initargs
   :dict-key :fix44
   :sender-comp-id "this-system"
   :target-comp-id "remote-system"
   :handler-fn #'my-handler))

;; Make an instance and add to connector
(defparameter *session* (make-instance 'my-session))
(cl-fix:add-session *connector* *session*)

6.3. Start your connector

You can then start your connector. Again, the behaviour is slightly different depending on what type of connector:

  • In the case of an initiator, starting the connector will automatically start the underlying sessions once connected to the remote endpoint. Once the session is started from an initiator, it will send out a LOGON(A) message as per the session's values.
  • In the case of an acceptor, we listen for an incoming connection. Upon receiving an incoming connection, we expect a LOGON(A) message to be sent by the remote end. That message will be matched to an added session based on key fields in the message (namely: target and sender comp ID's, and target and sender sub ID's)

Starting a connector is simple:

(cl-fix:start-connector *connector*)

6.4. Stopping or restarting your connector

Should you need to make some source code updates then re-iterate, you'd typically stop your connector, reload sources and start it up again:

;; Stop connector
(cl-fix:stop-connector *connector*)

;; ... Make some source code changes ...

;; Start connector
(cl-fix:start-connector *connector*)

6.5. Full examples

For some fully working examples, take a look at the following files:

7. Notes and caveats

Some notes and caveats to be aware of:

  • cl-fix will always write out UTCTIMESTAMP with milliseconds - even if the millisecond component is "000".

8. FAQ

8.1. The endpoint I'm connecting to requires SSL. Is this supported?

SSL support will be added in the future. However, until then, it's possible to use cl-fix behind stunnel.

An example, minimal stunnel configuration (stunnel.conf):

foreground = yes
[quote-endpoint]
accept = 127.0.0.1:1111
client = yes
connect = target.example.com:2222

Then, when constructing your initiator, ensure you connect to your local stunnel port:

(make-instance 'cl-fix:initiator :dict-key :fix44 :host "127.0.0.1" :port 1111)

8.2. Nobody in finance uses Common Lisp. Why bother?

Intellectual curiosity, and rethinking problems I've encountered that could have used fast(er) iteration.

I've been in situations where iterating and prototyping at "Lisp speed" against FIX endpoints would have been extremely beneficial. I'm also experimenting with trading infrastructure against my broker, somewhat inspired by Durenard's book "Professional Automated Trading".

I'm fully aware that the vast majority of developers in this space are using Java, C++, or C#. I myself am a Java developer. The intersection of "programmers who work in finance", and "Common Lisp users" may likely just be myself. Hopefully not! Perhaps this library will make others come out of the woodwork.

8.3. I've found a bug. What do I do?

Please see ./CONTRIBUTING.html for information on how to raise it.

8.4. What does fix9 mean?

Nothing, really. I wanted a short, memorable domain name on which to publish code, and then to get on with life. I might invent a better back-story in the future.