Parse FIX Messages Part 1

For those unfamiliar with the FIX protocol, see:

• https://en.wikipedia.org/wiki/Financial_Information_eXchange
• https://www.fixtrading.org/

The FIX Protocol transfers data uncompressed and in ASCII form.  The following data types are transferred like so:

Integer Value

To send the Integer 1,423, the TCP stream would look like this:

Index	ASCII	Hex
0	‘1’	0x31
1	‘4’	0x34
2	‘2’	0x32
3	‘3’	0x33

Date Time Value

To send the Date Time value pair “June 4th, 1998 2:58:48 PM”, the TCP stream would look like this:

Index	ASCII	Hex
0	‘1’	0x31
1	‘9’	0x39
2	‘9’	0x39
3	‘8’	0x38
4	‘0’	0x30
5	‘6’	0x36
6	‘0’	0x30
7	‘4’	0x34
8	‘-‘	0x2D
9	‘1’	0x31
10	‘4’	0x34
11	‘:’	0x3A
12	‘5’	0x35
13	‘8’	0x38
14	‘:’	0x3A
15	‘4’	0x34
16	‘8’	0x38
17	<SOH>	0x1

A very simple thing I can do is to read the incoming TCP stream and look for any integer and datetime values and replace them with a 32-bit Integer for Integer Values, and two 32-bit Integers for the Date Time Values.  So:

Incoming FIX stream looks like this:

(
is how the SOH or ASCII character 1 appears in a text editor)

9=154 52=19980604-14:58:48

Integer Value

Integer value does not really save much for a small number, but that is okay for now:

Integer field in ASCII format takes up 6 bytes:

(And the text based version in case you want to cut-and-paste)

39 3D 9A 01

After converting this using an FPGA, it would take up only 4 bytes:

Date Time Value

Now the Date Time value however has a lot of space for saving… it can go from using 21 bytes to using 12 bytes.  Additionally, the real savings is in the time it takes to parse this on a regular processor and in the time it takes to convert it to an appropriate type in the user-mode application.

35 32 3D 01 30 E1 3C 00 02 39 B8 01

The FPGA-converted value only uses 11 bytes:

What does this mean? If we are using a programming language that stores Date Time values as 2 32-bit Integers, we can convert the ASCII stream directly into this format.  If on the other hand we are using a custom data type that stores it using the UNIX epcoh time, we can convert it directly to that.

Or, what if we don’t care about the Date and want to discard that part of the message altogether? We can do that all in the FPGA as the data is coming in.  No need for us to even read the entire message in to memory, we can just do this as it comes in.

So I created a LabVIEW FPGA project that reads a text file which contains the raw FIX message or messages, and passes them down to the FPGA where I parse the data and convert each field, value pair into an optimized key, value pair and to send it back up to the host.

Here is a screenshot of a sub-VI that parses data coming in and converts it into an Integer Value.  So let’s say we are reading a FIX message and want to get the Integer value of the field number:

(for reference)

8=FIX.<Version><SOH> … <FieldNum>=<FieldValue><SOH> … 

What does the above LabVIEW FPGA screenshot mean?

It means that on each clock cycle, Data_in has an unsigned 8-bit integer representing the data coming in from the data stream, and Reset_in is a boolean value telling us if we should reset the current value being stored/saved.

As for the output, the Valid boolean is set once and only once and that is when the SOH character is encountered, and the Integer Value is set to the appropriate value and is held that way until Reset_in is set to true again.

Now what would this look like in VHDL?  I don’t know, but it will take me some time to implement it, and some more time to create a test bench, and even more time to create test values and cases for this.  But with LabVIEW? I just drew it and used existing LabVIEW tools to pass data through it.  Go LabVIEW.

In upcoming articles, I will expand this code to handle entire FIX protocol messages, which include:

Heartbeat

8=FIX.4.1
9=112
35=0
49=BRKR
56=INVMGR
34=235
52=19980604-07:58:28
112=19980604-07:58:28
10=157

LogOn

8=FIX.4.2
9=61
35=A
34=1
49=EXEC
52=20121105-23:24:06
56=BANZAI
98=0
108=30
10=003

Indicator of Interest (IOI)

8=FIX.4.2
9=154
35=6
49=BRKR
56=INVMGR
34=236
52=19980604-07:58:48
23=115685
28=N
55=SPMI.MI
54=2
27=200000
44=10100.000000
25=H
10=159

For those that wish to follow, find me on github at:

https://github.com/fpganow/FIXParser