cl-fix
Table of Contents
1. Overview
NOTE: cl-fix is under active development. It is NOT recommended for production use yet, though it is ready for some basic testing and early feedback. See 'Current Status' below.
cl-fix
enables you to write FIX-based systems using Common Lisp. FIX
(Financial Information eXchange) is a protocol that underpins the
communications between many major players in the finance and capital markets space.
This library will enable you to write anything from trading bots wired up to a broker, to simulators for testing other FIX-based systems (e.g. simulating a broker or exchange's endpoint). The primary value-add in using a language such as Common Lisp is that you'll be able to iterate and prototype quicker in Lisp than in other languages.
cl-fix is NOT an ultra-low-latency library suitable for HFT. Performance will be optimised in later rounds of development. First and foremost design goal is to have you iterating quickly against FIX endpoints.
There are 2 major functions that this library offers:
- Define FIX dictionaries. The dictionary macros define classes and de/serialisation functions for defined messages.
- Establish FIX sessions with remote endpoints to handle messages. That is, initiate or accept FIX connections and implement workflows using your dictionaries.
2. Current status
As stated above, cl-fix is under active development. Before I'd consider it production-grade, I want to complete (at least) the following:
- Run a cl-fix based system for at least one whole trading week against my broker
- Basic performance tests and baselining
Interested in helping out? See the contributor guide!
3. Key links
Item | Link |
---|---|
Project home page | https://fix9.net/cl-fix/ |
Source code | https://gitlab.com/fix9/cl-fix |
Discussions | https://groups.io/g/cl-fix |
4. Installation
Firstly, install the fix9 distribution into quicklisp:
(ql-dist:install-dist "http://fix9.net/dists/fix9.txt")
Then:
(ql:quickload '(:cl-fix :cl-fix/fix44))
Cl-fix is also included in the official quicklisp distribution, but, it only snaps it on a monthly basis. The fix9 dist is updated far more frequently, so must be added to ensure you have the latest updates. This will likely matter less in the future when the codebase stabilises and changes are less frequent.
To update your fix9 dist, just like with the official quicklisp dist, execute:
(ql:update-dist "fix9")
5. Dictionaries
Dictionaries are where you define the fields and messages your app will be exchanging with its peers. Dictionaries are the Google Protobuf equivalent to '.proto' files. Key difference from Protobufs is that we're using Lisp macros to do our definition, which in turn does our code-gen for us.
This section outlines how to define a dictionary, then use the serializers and deserializers that are generated for that dictionary.
5.1. Define a dictionary
Generally you'll derive your dictionary from the FIX specification for your endpoint. However, in the source, there is a standard FIX 4.4 dictionary which should get you 90% of the way there for many endpoints.
Rather than repeat the contents of that file here, you are encouraged to read through it and its comments. It is over here: ./dict/fix44.lisp.
Using the dictionary macros in cl-fix results in a few functions within that namespace:
SERIALIZE-FIX
function: serializes a message instance to a stream for this dictionary.READ-FIX
function: reads one FIX message from a given stream and returns it in its object form for this dictionary.STR
Returns a string representation of a given message for this dictionary.
5.2. Serializer/Deserializer usage
(ql:quickload '(:cl-fix :cl-fix/fix44)) ;; Serialize (fix44:str (util:make-fix-msg :logon :msg-seq-num 45 :sender-comp-id "me" :target-comp-id "them" :sending-time (local-time:now) :username "bob" :password "secret")) ;; Deserialize (with-input-from-string (s *) (fix44:read-fix s))
6. Establishing sessions
Establishing a session is a matter of 3 broad steps:
- Set up your connector (either an initiator or acceptor)
- Set up your sessions, and add them to the connector
- Start your connector
This section walks you through how to do that.
6.1. Set up a connector
To establish a session, you will first need a CONNECTOR
.
There are 2 connector types:
- An
INITIATOR
, which initiates an outgoing connection to an endpoint; and - An
ACCEPTOR
, which accepts an incoming connection.
You can create an initiator as so:
(defparameter *connector* (make-instance 'cl-fix:initiator :host "some.example.host" :port 8383 :dict-key :fix44))
6.2. Set up and add sessions
Next, add one or more sessions to your connector. Adding a session to an INITIATOR
versus
ACCEPTOR
has slightly different semantics:
- Adding a session to an
INITIATOR
means that upon connecting to the remote endpoint, each session will send its respective LOGON(A) message. - Adding a session to an
ACCEPTOR
means that when the connector receives an incoming connection, it expects to receive a LOGON(A) message, which will then be dispatched to the correctSESSION
object.
Adding a session requires you to first create one, then add it to the corresponding
connector. You'll also want to bind it to a message handler. You'll typically subclass the
in-built SESSION
class as follows (continuing from the above example):
;; Define a handler (defun my-handler (session msg) (log:info "Received a ~A message" (util:fix-msg-type msg)) (case (util:fix-msg-type msg) (:market-data-snapshot (log:info "Yay - got some market data")) (:execution-report (log:info "Yay - got an execution report")) (t (log:warn "Unhandled message: ~A" msg)))) ;; Define session class defaults (defclass my-session (cl-fix:session) () (:default-initargs :dict-key :fix44 :sender-comp-id "this-system" :target-comp-id "remote-system" :handler-fn #'my-handler)) ;; Make an instance and add to connector (defparameter *session* (make-instance 'my-session)) (cl-fix:add-session *connector* *session*)
6.3. Start your connector
You can then start your connector. Again, the behaviour is slightly different depending on what type of connector:
- In the case of an
initiator
, starting the connector will automatically start the underlying sessions once connected to the remote endpoint. Once the session is started from an initiator, it will send out a LOGON(A) message as per the session's values. - In the case of an
acceptor
, we listen for an incoming connection. Upon receiving an incoming connection, we expect a LOGON(A) message to be sent by the remote end. That message will be matched to an added session based on key fields in the message (namely: target and sender comp ID's, and target and sender sub ID's)
Starting a connector is simple:
(cl-fix:start-connector *connector*)
6.4. Stopping or restarting your connector
Should you need to make some source code updates then re-iterate, you'd typically stop your connector, reload sources and start it up again:
;; Stop connector (cl-fix:stop-connector *connector*) ;; ... Make some source code changes ... ;; Start connector (cl-fix:start-connector *connector*)
6.5. Full examples
For some fully working examples, take a look at the following files:
- the initiator example
- the acceptor example
7. Notes and caveats
Some notes and caveats to be aware of:
- cl-fix will always write out UTCTIMESTAMP with milliseconds - even if the millisecond component is "000".
8. FAQ
8.1. The endpoint I'm connecting to requires SSL. Is this supported?
SSL support will be added in the future. However, until then, it's possible to use cl-fix behind stunnel.
An example, minimal stunnel configuration (stunnel.conf):
foreground = yes [quote-endpoint] accept = 127.0.0.1:1111 client = yes connect = target.example.com:2222
Then, when constructing your initiator, ensure you connect to your local stunnel port:
(make-instance 'cl-fix:initiator :dict-key :fix44 :host "127.0.0.1" :port 1111)
8.2. Nobody in finance uses Common Lisp. Why bother?
Intellectual curiosity, and rethinking problems I've encountered that could have used fast(er) iteration.
I've been in situations where iterating and prototyping at "Lisp speed" against FIX endpoints would have been extremely beneficial. I'm also experimenting with trading infrastructure against my broker, somewhat inspired by Durenard's book "Professional Automated Trading".
I'm fully aware that the vast majority of developers in this space are using Java, C++, or C#. I myself am a Java developer. The intersection of "programmers who work in finance", and "Common Lisp users" may likely just be myself. Hopefully not! Perhaps this library will make others come out of the woodwork.
8.3. I've found a bug. What do I do?
Please see ./CONTRIBUTING.html for information on how to raise it.
8.4. What does fix9 mean?
Nothing, really. I wanted a short, memorable domain name on which to publish code, and then to get on with life. I might invent a better back-story in the future.