Question: How can you handle the trading and the non-trading (e.g accounting) aspects of the business successfully?  (Many people reported that they didn’t have enough time to focus on strategies)

Answer: Automate everything that can be automated.  It is possible to automate every single step in the trading process, to the extent that you are not involved in the day-to-day work.  On OSX and Linux the cron system starts processes automatically.

Once you can mentally free yourself from those tasks, the next step is to automate the research process.  Parameter tuning is simple.  Writing larger strategies takes time, but fortunately a well-written python implementation really streamlines this process.

Sidenote: the python package PuDB turned out to be indispensible.

Question: How do you sleep at night?

Answer: It’s really hard.  You know your money is sitting with a broker that has power to steal from you, especially in light of controversies surrounding MF Global and Pensen Financial.

In my case, since a significant portion of my wealth is locked up, it’s pretty terrifying.  Fortunately I think I can find a job if necessary (and, furthermore, it would seem that others have found jobs at banks and hedge funds after closing their own operations)

Question: How are you handling the lack of volumes in the financial markets?

Answer: Needless to say, a new set of strategies must be deployed.  I will discuss them in a future post, but 5 AM seems like the right time to end this post

Answer: It’s easy to say that rebates are unfair, possibly because of the woeful lack of simple explanations of the rules.  So consider this my attempt at a simple explanation.

For a trade to occur, a person must have sent an order (for this example, suppose the order is a buy order) before the trade occurred.  That buy order is sitting in the exchange’s order book.  Another person (in financial jargon, the “active party”) comes later and sends a sell order at the same price (or lower).

When that trade occurs, the second person is acting based on more relevant information.  Assuming the second person isn’t an idiot, he is sending the order because he believes the price is going to fall (yes there are people who have to trade for other reasons, such as hedging another exposure, but the majority of trades are not hedging trades).  So the first person (who is the “passive party” in the transaction) is ostensibly disadvantaged.  The trade is a bad trade for the passive party.

We can actually measure this effect quantitatively.  We will call the active party “instantaneously correct” if the next price move is favorable for him.  It should be obvious that the active party is correct if and only if the passive party is incorrect (since the passive party has the opposite exposure).

So the natural question is: how frequently is the passive party correct (active party is stupid)?  This is a simple test: just count each trade (comparing the mid-market price at the time of trade to the next value), weighted by share volume.  Limiting to the actual trading day, assuming positions are unwound in the closing cross, the active party is correct more than 99.95% of the time (measured since September 1).  To put it differently, the market maker is almost always wrong.

The subsequent questions should be straightforward, but just in case …

Question: Why don’t we just ask the exchanges to put some forced lifetime, preventing market makers from canceling orders too quickly?

Answer: that would just increase spreads, taxing the very people who are pushing for this proposal.

To understand why, look at the mechanics of the trade.  The ability to take a risk (by sending a buy order at a higher price or sending a sell order at a lower price) is a function of how quickly the trader can respond to an adverse event by canceling the order.  So if there is a forced lifetime, anyone sending an order is taking a risk tat the order may have to sit for some period of time.  And if you believe that you may be picked off at any time (after all, given when major events have occurred and given the quick market response, the risk is significant) you would not send orders at aggressive prices unless you strongly believed that particular trade would be profitable if picked off.  On the average, market makers would quote larger spreads, which would represent a larger expense to the other traders.

Question: Is ultra low latency trading risk-free?

Answer: No, and that’s why a rebate is needed and why order lifetimes should not be enforced

To understand why, take a look at certain news events during the day that cause price dislocations.  As the news comes out, every major trader is generally trading in the same direction.  Thus, the people who take the wrong side of the trade end can end up losing more than 1% of the position value.  When put in this light, the rebate does little to compensate for the risk, but it is enough to allow the passive party to take a risk.

Answer: A month

If you stare at a blank vi terminal and expect to magically spit out C code, you will get nowhere.  Same with assembly or C++ or with most languages.  So how do you get off the ground running?

Enter Python.

If you asked me a year ago, I probably would have said “Enter Perl”, but Python is my new favorite language.

It should take no more than a day to put together a full stack in python.  At a high level:

• python struct module gives you a very easy way to unpack binary messages.  major gripe: what is the python equivalent of unpack(‘x8 S>/A’) ??
• The simple networking commands have almost direct python analogues
• There’s a module or builtin for pretty much any other piece you need

Building a straightforward, correct implementation gives you a baseline.  You can then put together some simple regression tests.  For the order book, a simple regression test involves replaying a simple data segment (for example, you can record a day’s data and just filter for a specific ticker), passing the data through a driver program, and dumping what the book looks like (all of the orders and their relative priorities)

Now, with the baseline, you can attack a C or C++ implementation.  Most people would fire up vi and start from scratch.  However, there’s a much slicker way to do this: use the python/C API to rebuild piece by piece in C, replacing pure python implementations of features with the C equivalent.  For the C++ hackers, boost::python allows you to reimplement python classes with C++ equivalents.

Since we have the original pure python implementation, at each stage of translation we can verify that the new program is consistent with the old one.

Suddenly, a week later you have this hybrid Python and C implementation, with core features implemented in C and python used as a glue.

It should take less than a day to remove the python scaffold and connect the underlying C components with a C driver program.  And now you have a pure C implementation

Now that you have a C implementation, you can go back and optimize components as relevant.  Now, is this the most efficient route?  Probably not.  However, two weeks into the process you should have a fully functioning C implementation.

How do you reduce elements to x86 assembly?  For that, you can either use inline assembly (gcc uses the __asm__ keyword) or use an assembler like nasm to produce ELF object files, linking them with the C code using ld.

Summarizing, the low-level code only enters the fray after the design has been prototyped, and the cascade of implementations allow you to develop a comfort level with each step of the process before delving further.  The steps are:

• Pure python implementation (1 week)
• Python gluing PythonC/boost::python wrappers to C++ implementations (1 week)
• C/C++ gluing the native C/C++ implementations (3 Days)
• Tweaking the C implementations where relevant (2 Days)
• Replacing the C code with optimized assembly routines (1 Week)

And the most awesome part is, at each stage you have a fully functional platform!

Question: How do you backtest or simulate strategies?

Answer: In python

The first python implementation involves a prototype that uses callbacks and other high level model-friendly techniques (simple event reactor) that allow me to quickly put together and test strategies.  Once I am happy with it, I can implement it using the low-level API (which, for various reasons, avoids the dynamic features like virtual functions that make the callback implementation so flexible).

I use 3 levels of simulations

• Python prototyping for testing strategies.  These are designed to be throwaway idea tests, complete with introspective features.  However, latency testing is futile.
• C in-process simulator.  Each strategy program can be built in production or simulation mode.  The simulation binaries actually can be used in production trading, but there is some extra code that allow you to choose to run in simulation or in production using a command-line flag.  This allows you to test the source code to make sure that the implementation is correct.  Again, since the simulator uses a time stepping model, latency testing is futile.
• Live simulator.  A separate server simulates the exchange, reading in a data file and interspersing strategy orders with real orders and simulating the order book.  This gives you a more accurate measure of performance, but may take longer depending on whether you choose to simulate in real time or 2x or 10x time.

Generally, it flows in that order.  An idea is first modeled in python and then implemented and verified in C.  Finally, performance is tested using a simulation server.

Answer: Ideally, everything is written in-house

People sometimes think of this as a sort of “NIH Syndrome”, that there’s some sort of ingrained aversion to using others’ code.  But that can’t be the case (after all, they are using an operating system not developed in-house).

At a very high level, suppose you are pushing 10M shares of volume per day.  A simple mistake at any level could blow up your operation in a matter of seconds:

- A broken risk check could inadvertently allow you to exceed your capital limits

- A broken order gateway may bomb out before you can complete your trade, leaving a large undesired naked exposure

- A broken strategy may over-hedge, incurring significant losses

- A broken feed may be pumping wrong prices, resulting in ridiculous orders and trades

- A broken reporting infrastructure may report the wrong position, forcing you to make poor decisions

- A broken kernel module or tweak can result in a kernel panic, leaving your machine a mess at the most critical time

When you are trading with your own money, these risks will keep you up at night.  You need to be absolutely confident that every piece is working properly.  As a concrete example, suppose that the feed is reporting bad prices.  Then, you might find that buying a particular stock is profitable and proceed to buy when in fact the trades are losing money.  In the blink of an eye, a strategy may end up buying $1M worth of a stock.  Now, of course, that’s why you have exposure control (but you are assuming that it will work

This is in stark contrast to how most technology vendors approach the problem.  To most vendors, 99.99% uptime is acceptable.  A blip is treated as a rare hiccup.  However, Murphy’s Law says that systems will run amok at the worst possible moments.  And in those moments, multi-million dollar losses aren’t uncommon.

At the end of the day, the only way to ensure that there are no problems is to build it yourself  You should understand every trick employed, every potential landmine, every design decision.  It’s easy to take a shortcut and buy an off-the-shelf solution, but the cost of the first crash far exceeds the development time for a self-written solution.

As I discussed in another forum, there are special strategies that can only be deployed with custom components.  No off-the-shelf feed handler and order gateway can run the feed spike trading strategy (as far as I have seen, third party feeds don’t give you the ability to tweak when events are triggered).

Having said that, there are a few pieces that could safely be outsourced.  These, as you can imagine, are not on the critical path.  For example:

- Software and tools for visualization (charting and order book visualization tools)

- Reporting and compliance software (to generate those reports)

Question: Why are you prejudiced against vendors?

Answer: Most vendors fall into one of the following categories:

- Built by a pure technology firm without shaking down the system with real trading.  It’s easy to write the software, but the only way to be sure the systems work is to actually trade with the system.  I am very wary of vendors with < 3 customers, because you know you are essentially beta-testing their system while risking your own capital.

- Built by a trading firm.  Why would the trading firm sell their platform if it is making money for the firm?  So here’s a sense in which some set of trading strategies fail to work on the platform, ostensibly because of a latency issue.

- Built by a trader.  This is the worst case, because the IP is of questionable origin.  There are at least 4 cases which come to mind involving people trying to steal IP to run the system elsewhere.  The most recent case involves a kid my age.  And there’s really no need to invite trouble by using that platform.

I guess the best summary is: Imagine you are running this on your own.  Would you really trust a third party feed, whose code you haven’t seen, not to blow up at the worst possible moment?  Do you trust the third party order gateway provider to keep up with changes with the exchanges?  Do you trust the reporting infrastructure to handle new exchange features like half-penny prices?

Question: How can you trust the Linux kernel or libc or gcc?

Answer: Because I’ve been through most of the source and modified relevant parts of the code  Yes, it takes that level of dedication.  When a slight hiccup threatens the solvency of your business, you have to put in enough energy to trust every piece of the system, on the hardware and on the software side.

Answer: The question cannot be answered as stated.  This requires you to know when quotes leave the exchange and when the software stack sees the quotes, which isn’t well defined because both use separate time sources.

To understand the difficulty, assume for the moment that there is no clock drift (both clocks are running at the same rate).  Imagine that the actual transmission time (measured from either time source) is (L) microseconds and that the difference between the clock times is (D) microseconds, with the sender running D microseconds slower than the receiver.  Then, the sender sends a message at some time (T).  According to the receiver, the message is sent at time T+D and is received at time T+D+L.  Supposing the sender marked the message with its local time T, the receiver can compute D+L = (receiver local recv time) – (time in the message).  However, without knowing D in advance, you cannot compute L with certainty.  This situation is muddied when you take into account clock drift and other oddities.

To put numbers to the situation, you could know that the difference between local recv time and exchange send time is 1 millisecond, but you can’t determine whether the latency is 1 msec or actually 50 usec (with a 950 usec skew)

Question: If you can’t answer that question, what numbers can / should you quote?

Answer: This question rankles me in particular because some unnamed vendors like to quote average or optimistic times which aren’t accurately measured.

1. The start and stop times should be determined from the same clock source.  That means same machine.  You can try as hard as you wish (e.g. using ptp) to synchronize machines, but the best results use the same machine for start and stop).

2. The times should use a high-precision timing source.  The obvious first point is that NTP should be disabled.  Nothing should be tampering with the clock.  Whenever possible, avoid time sources for which the cost of measuring the time significantly affects the experiment.  As an example, the gettimeofday system call can cost you dearly.

3. There should be no assumption regarding any clock source for which you have no control.  For example, for a long time the BATS exchange would use multiple machines to generate quotes (which were coalesced by a machine later on), resulting in weird time anomalies where you would see three consecutive quotes with times T, T+1, T respectively.  Of course, if you looked on a specific ticker, the quotes were in-order.  And someone who doesn’t bother considering time source issues before reporting about timing anomalies (are you really surprised who complained about this?) shouldn’t be commenting about time issues.

And we should be quoting the following numbers: median, short-tail (90%), long-tail (at least 99%).  Definitely not mean and standard deviation (stop the madness)  Why?  Because the most profitable opportunities generally are long-tail events (99.99%+) so the standard deviation doesn’t really capture the effects.

Question: What are your times?

Answer: Using the rdtsc x86 instruction on an appropriate configured server (disabling power savings mode in bios, forcing highest clock speed in software, etc) using a dual xeon  X5650, Solarflare SFN5122 with appropriate affinity settings, and a custom tuned Linux kernel (one day I’ll clean up and post a patch):

A) Order entry round-trip

START: strategy sends an order

STOP: strategy receives order acknowledgement

PROCESS:

- Strategy tic

- Send order to gateway to buy 1 share of BAC

- Gateway receives request and generates outbound OUCH message

- Risk check processes the request and sends it to the exchange

- Exchange receives it, processes it, and then sends us an acknowledgement

- Gateway receives response, parses it, and sends a message to strategy

- Strategy framework acknowledges message, informs strategy that order was received

- Strategy toc

Median = 65 usec, 90th% = 74 usec, 99th%= 88 usec

A) Order to feed acknowledgement

START: strategy sends an order

STOP: strategy receives quote for the original order

PROCESS:

- Strategy tic

- Send order to gateway to buy 1 share of BAC

- Gateway receives request and generates outbound OUCH message

- Risk check processes the request and sends it to the exchange

- Exchange receives it, processes it, and then sends us an acknowledgement

- Gateway receives response, parses it, and sends a message to strategy

- Strategy framework acknowledges message, informs strategy that order was received

- Strategy notes when the ID generated by the exchange

- Quote engine receives quote and informs strategy

- Strategy framework sees the quote, updates order book, and notifies strategy

- Strategy compares ID to marked ID, sees a match, and toc

Median = 78 usec, 90th% = 85 usec, 99th%= 98 usec

Note that strategies which don’t care about each individual quote (lots of speed strategies just care about top of book updates) perform better because they don’t see every quote.

Question: What is your internal latency?

Answer: This measurement requires a second server to act as a fake exchange and mark the times.  You have to connect the two servers together and use the fake exchange to generate a fake quote.  A fake strategy, receiving the fake quote, would decide to send an order.  The second server marks the other end.  This does include wire latency, but those are basically negligible in this test.

Median = 11 usec, 90th% = 12 usec, 99th%= 14 usec

To get these aggressive numbers, there were many tricks designed to reduce the amount of work per order.  For example, when sending out orders, templates of the required messages are precomputed in advance so that only a few fields need to be populated before shooting the order (and risk checks are optimized so as to minimize the number of full checks in the critical path).  And yes, a lot of hard pieces are in hand-rolled x64 assembly …

I’m pretty sure there are people out there who can beat this (on PC or on custom hardware); please post some numbers for our benefit

Answer: Yes, although it’s been tough.  I really could use some help, and one day I will write a multi-page rant about the utter incompetency of financial recruiters …

It’s been months.  I apologize.  I didn’t intend to abandon the blog, if that’s of any consolation

Something short for today: accounting for HFT.

Question: What are the per-trade costs and commissions in equities trading?

Answer: The per-trade costs break down as follows:

1. “Brokerage Commissions” — for a standard sponsored access relationship, this refers to a commission.  For self-clearing, this refers to the amortized costs.
2. “Taxes” — monies that flow to government or regulatory agencies
3. “Exchange Costs” — monies that flow to or from the exchanges.

Before that, a few friends asked me about a blog post on the Nanex site: http://www.nanex.net/StrangeDays/08252011.html (they found the site through a zerohedge repost)

On August 25, 2011, over the course of 13 hours (46,800 seconds or 46.8 B microseconds) there were 628629 messages corresponding to DELL.  The first DELL quote of the day was at 7:02:51 AM (someone tried to buy 3000 shares at 14.48) and the last quote was at 19:59:51 (a sell order at 14.90 was canceled).

Close inspection of these quotes reveals something very disturbing:  Nanex’s system can’t handle the recent quote volumes.

So let’s start from the obvious questions:

Question: What happened?

It looks like someone was speed testing.  10K orders per second translates roughly to 100 usec per order, consistent with order entry latencies in the NASDAQ colocation.

Question: Was the strategy running on other tickers as well?

AFAICT no.

The exchange assigns an internal ID to each order on the market.  For NASDAQ in particular, which is where this event happened, The strategy started saturating the market from the order id 465998436, and more than 99% of the next five thousand order id’s were for orders on DELL.

Question: Nanex alleges that you would need 40 Gb/s bandwidth to handle the message rate if it indeed hit every single ticker.  Is this true?

Let’s try to get a better handle on the math.  Sending and canceling an order, on the ITCH 4.1 feed, takes 49 bytes (30 for the add and 19 for the cancel).  There are packet header messages, but the exchange will coalesce multiple messages into a larger packet (up to 64K).  As a result, we can amortize the cost to less than a byte per message, so lets call it 50 bytes for a round trip.  Physically, since the network roundtrip time is O(100 usec), lets suppose there are 5000 add-and-cancel units.  This yields 50 * 5000 = 250000 bytes = 2Mb per ticker.  Over 4000 stocks, you get 8 Gb/sec, well within the 10G capability (and this is taking grossly pessimistic estimates).

As far as the lower bandwidth customers are concerned, the feeds are shaped so that consumers will not drop packets (they will be slightly delayed to ensure that downstream clients don’t drop).  The network systems should be able to handle the flow.

Question: Is this particular trading illegal?

Depends on who you ask.  Based on a strict interpretation of the Securities Act of 1934, given that the order size is an order of magnitude smaller than the bid and offer sizes, and given that no transactions actually occurred and no price was directly manipulated, there are grounds for arguing that this is not illegal.

My interpretation isn’t the only one at play here.  People like Joe Saluzzi (the CNBC guest who bitches and moans about HFT, currently blogging at http://blog.themistrading.com/) would argue that orderly markets are being disrupted by a flood of orders, and certainly some judges would buy the anti-HFT arguments.

Question: Why would Nanex write this blog post?

I will give them credit for one thing: “This cannot be dismissed as a computer problem or glitch. This can’t be explained as stupidity or some oversight. It is not pinging for hidden liquidity. And it’s certainly not price discovery. As far as we can tell, it’s not adding liquidity or narrowing the bid/ask spread.” is a true statement.

I suspect that their systems weren’t able to handle the volume.  They are, after all, selling software.  I can envision a customer complaining that their feeds were delayed.  As a technology provider, it is easy for them to write a blog post blaming the markets

Should a customer use the raw data feed?  Without at least 1G pipes, it makes absolutely no sense to get the raw feed.  The performance of a processed feed, especially given that the data rate will be orders of magnitude lower than the raw feed, far exceeds the raw feed when connecting over 100Mb/s connection.

FYI: I am amazed at how many people stumbled upon this blog looking for a how to build a faster limit order book.  It’s very possible that a lot of people are suffering, so I probably should tone down my criticism a little bit.  However, they are selling software to handle the market data …

I was going to talk about latency numbers last week, but in light of the last few trading days (especially Thursday’s market collapse and today’s slide), a lot of people emailed me asking about risk. Someone asked me about Fixnetix, claiming “20+ pre-risk checks taking less than a 100 nano seconds“.  And since time is really really hard to come by (yes, we all saw what happened over the last few trading days), I decided to answer the most commonly asked questions that I could pound out in a few minutes on my iPhone:

Question: How do you measure and approach risk?

There are quite a few measures of risk, some of which are covered in many books. VaR (value at risk) is one commonly thrown about. But for the types of strategies we are discussing, the portfolio gross exposure is the main criteria (sum of the absolute dollar positions).

Question: how much collateral do you need?

Answer: Quite simply, if you are closing positions at the end of each day, it makes no sense not to use as much intraday leverage as possible.  After all, since you are not holding overnight positions (where major moves can happen with no ability to react), you dont pose as much of a risk to a broker as a trader who holds positions overnight.  And if you don’t need to commit capital to the risk deposit, you can use the extra money to pay other expensive bills.

Question: How are you making money in this environment?

Answer: That will have to wait

Question: How do you calculate the risk quickly?  is 100nsec achievable

Answer: Unless they are referring to average case with some very nasty tails, I call BS.  Let’s suppose you are running a basket strategy involving 500 tickers (S&P index).  You need to take the total sum of dollar positions across all tickers.  Evaluating that sum itself, whilst maintaining a map with ticker as the keyword, is expensive.  Even if you incrementally update (update the gross position every time the price changes), and you are pumping a million orders per second, and prices are changing left and right as they did over the last few days, the numbers dont add up.

tl;dr: The most plausible explanation is that they aren’t doing all of the required checks.

Question: How do you decide when to take a loss on a trade (if  you hold a position for a long time)?

Answer: Unfortunately there is no quantitative way of defining a hardstop.  That’s where CNBC comes in handy  In all seriousness, it’s very hard to quantify exactly how the decision is made, but it mostly stems from understanding the strategies, how they behave, and what conditions could cause aberrant behavior.  I really wish there were something more algorithmic involved, but in my experience algorithmic methods will be more conservative (resulting in taking losses early rather than waiting until conditions improve).

Question: How do you efficiently implement the order book?

Answer: I realized I wrote hash implementation, but the starting point is how to most efficiently implement a hash function.

All implementations require a hash off the top (for symbols — you can’t mix orders from different symbols).  Some extremely parsimonious implementations may require another table for order id, since some basic info like symbol and side are fixed throughout the life of the order.

Some people have asked for some hints, and also asked that I don’t spoil it immediately, so the full solution will have to wait.  However, I will mention one thing: most strings you will have to hash are generally 8 characters long.  For example, if you decide to work with CUSIPs and throw out the check digit, you are left with 8 characters.

That being said, and I’m really disappointed I have to mention this (a lot of self-professed C / C++ experts were unaware that this was even possible):

Assuming that you are dealing with ascii 8-bit characters (true of all data interchange formats that I’m aware of), you can convert between 8-character strings and 64-bit numbers:

char *t = “VEYRONB “;

uint64_t u =*(uint64_t *)t

I believe C++ has a special keyword reinterpret_cast:

uint64_t u = *reinterpret_cast<uint64_t *>(t);

Lots of basic operations are easy to implement.  As a warmup to the people who asked, I suggested they implement some stuff from libc (like strncmp) using this representation.

Question: Are you going to take this site down, like the WK Selph character?

Answer: Unequivocally no.  Unless, of course, this turns out to be a personal liability, in which case as much as possible will be preserved.

This gets a .1 because it’s too long for a comment, too short for ‘Part 4′, and I can barely stay awake at this point (haven’t slept since Wednesday, unfortunately)

Question: Can you provide market data to play with?

Answer: I would have to register as a vendor of record for various exchanges and other companies, and even then I wouldn’t be able to just share it with the rest of the world.  That being said, I had the following link in my history:

ftp://emi.nasdaq.com/ITCH/

This is for NASDAQ data in ITCH 4.1 format, but other NASDAQ OMX exchanges (e.g. BX) use the same format.

There are many vendors of record who will be able to resell market data at reasonable prices.

Question: Can you provide timestamped market data?

Answer: All of the aforementioned links are provided by the exchanges themselves. They only contain the exchange timestamps.

Question: What if I pay you?

Answer: There is significant economic overhead to become a vendor of record.  This might be the kind of thing for kickstarter ( http://www.kickstarter.com/profile/proposals/83047 )

Email me (intransigence123@gmail.com) if you are interested in such a service.  Would you be willing to pay, say, 10$ a month + some initial signing fee (I’m guessing it will be something like $100) for current market data?

Question: How can a small guy make money when companies like Goldman Sachs have phalanxes of traders and developers scanning the markets for each asset class?

Answer: technology. And cleverness.  But let’s focus on technology for the moment.

At a high level the trading platform must do the following:

1) receive market data and order information from the exchange
2) organize the data in a meaningful way
3) decide to do something.
4) tell the exchange what you want to do.

Discount brokerage houses like interactive brokers provide tools to abstract away some of the nuances of 1,2,4 by providing an API.

In many of the banks and larger hedge funds, there are actual traders who do not necessarily care about the nitty gritty implementation details, so on-staff developers try hard to provide a low latency API. Still, at the end of the day, they are providing a wrapper that introduces delays in the critical path.  This allows for a class of “burst” strategies (which I vaguely outlined in a HN post).

Many of those larger entities have been around for a very long time, and it’s hard to migrate a hundred-man team from one platform to a new one.  There are so many operational risks involved, and missing a trading day could cost millions of dollars.  This is true in many software houses, and Joel Spolsky wrote about this more than a decade ago.

Question: So you are starting from a blank slate.  How do you start developing?

Answer: For various reasons, (2) is the natural entry point (historical data can be used in lieu of real data, strategy requires data, and building an order book simulator can initially obviate the need for true exchange order entry), so let’s start here.

At the end of the day, no matter what asset class, for each symbol there are four few numbers we are interested in:

- Price at which I can buy the asset (Pb)

- Price at which I can sell the asset (Pa)

- Amount that I can buy (Qb)

- Amount that I can sell (Qa)

Note: There are nontrivial amounts of information that can be calculated.  A common interpolated fair-value measure used in many predictive signals, called the weighted-mid-market-price, can be calculated as WMM = (Pb * Qa + Pa * Qb) / (Qa + Qb), but I will save this for a more quant-oriented post.

Data comes in one of a few forms.  The easiest form to handle is the price-level feed.  The Consolidated Quote feed (CQS), which is a regulatory feed that describes the top of book prices for each exchange, is a price feed.  Each message tells you, for some ticker, the price and quantity at the top of the book (so there’s no need to do any extra legwork).  You could store this information in a simple C struct for each ticker, using some sort of dictionary to simplify the ticker to struct mapping.  Updating is straight forward: just overwrite the last data point.

A more complicated form is the order feed.  Check the NASDAQ ITCH 4.1 Specification for an example.  The exchange manages all of the open buy and sell orders in two priority-sorted structures, where new orders are placed in their proper place based on their relative priority (the comparison function involves the prices and the time that the order was sent).  Historically, the actual mechanism involved a physical ledger, hence the name “Order Book”.

The trade mechanism is straightforward: When a new buy order is sent, if there is someone willing to sell at the order price or lower, a trade occurs.  The highest priority sell order is offset with the buy, and then the next orders is considered until the entire buy order is filled or there are no sell orders at the buy price.

Most of these exchanges use a price-time priority rule.  To understand, lets use a simple example:

Alice sends an order to buy 100 shares of BAC at 1.01

Bob sends an order to buy 100 shares of BAC at 1.02

Eve sends an order to buy 100 shares of BAC at 1.01

Now, Sally sends an order to sell 150 shares of BAC at 1.00.  Since Sally’s sell price is lower than the largest buy price (1.02), a trade will occur.  Who will actually end up buying Sally’s shares?

We start at the best price and walk down.  In this case, since Bob’s order is the largest price, his order is filled first.  Bob’s order for 100 shares is fully executed, and Sally receives a notification that 100 shares were filled.  Then, there are 50 shares left for Sally at 1.00.  Now, Alice and Eve’s orders are at the same price.

Order feeds are direct in the sense that it shows you every limit order** and action that enters the exchange.  The traditional way to make sense of order data is to effectively replicate what the exchange does: maintain the order book structure and update every time a new message comes in.  There are three fundamental types of messages:

- New Order (someone sends a limit order, and the exchange accepts it without offsetting against another order)

- Cancel Order (someone cancels part or all of the order and the exchange removes it from their book)

- Trade (an actual trade occurs)

Other actions, such as ‘Replace Order’, can be described in terms of the base actions (in this case, a Replace is an atomic Cancel + Add).

Question: Price-level feeds make sense.  How do you process Order feeds?

Answer: Depending on the type of strategy you want to run, you may need quick access to certain pieces of information.  Most strategies just need to know the 4 pieces of information from above.  Some require more.  The development pathway, as you can imagine, is:

What does the strategy do? –> What pieces of information do I need?  When do I need it?  How frequently must it be updated? –> What is the best order book structure?

Question: What are some common types of order book structure?

Answer: I may not end up describing every part of the structure.  For example, to optimize order lookup and still keep the structure, the most obvious approach is to keep a parallel hash table mapping order ids to order information to basic order information.  If some details sound vague, it’s because I’ve glossed over most implementation details — and yes, the devil is in the details here.

- List-of-Lists or Array-of-List: All of the orders for a price level are stored in a linked list, and these linked lists are arranged in another linked list or array.  Visually, an array of list would look like this:

102 –> Bob

101 –> Alice –> Eve

This structure makes it very easy to answer questions like “how many shares do people want to buy at price 1.01?” or “How many shares are behind my order?” It also benefits from locality (action at or close to the inside market can be processed quickly).  However, other operations take many steps or require too much memory (how would you implement adding an order?  Most likely, it either involves a hash table lookup  or walking the linear price list)

- Priority queue (e.g. heap): Just forget the elaborate structures and keep all of the orders in sorted order.  Locality is a bit harder to exploit, but there are definitely fewer moving parts than the list of lists.

- Skip lists: These structures involve multiple linked lists with increasing distance so as to reduce the number of steps required to walk to a particular order.

Now, some operations are fast in some implementations but could require a linear scan or other slow operation in other implementations.  For example, there’s no simple way given the priority queue implementation to count how many shares are behind yours.

Question: So how do you efficiently implement the underlying primitives (such as hash tables)?

Answer: That will come in the next post

**Caveat: Not every order is transmitted in the feed.  In particular, for venues that support it, hidden orders are not disseminated until a trade occurs, and the order-level feeds explicitly mark those trades in a different way from normal trades.