\n
So what now. \u00a0We know what a NASDAQ ITCH 4.1 Market Data Message looks like. \u00a0The format is very simple, there is some – yes – ASCII data in the message format, and all messages are preceded by the message length. \u00a0Message length preceding the message makes it very easy to interpret a feed from inside an FPGA.<\/p>\n
What to do first? Well, what does eXtreme Programming say to do? It says keep it simple.<\/p>\n
So, I am working on a LabVIEW 2016 program that does the following:<\/p>\n
- \n
- Open a NASDAQ ITCH file<\/li>\n
- Send it one byte at a time into the FPGA over the PCIe bus<\/li>\n
- FPGA reads the feed, skips over messages types that it does not know how to handle, and parses only messages of a specific type.<\/li>\n
- Send back to the host computer a statistic – any statistic for now.<\/li>\n<\/ol>\n
If you are unfamiliar with National Instruments products and LabVIEW, go here to learn more:<\/p>\n
- \n
- http:\/\/www.ni.com\/labview\/<\/a><\/li>\n
- https:\/\/www.youtube.com\/user\/Labview<\/a><\/li>\n<\/ul>\n
Step 1 – Open a NASDAQ ITCH file<\/strong><\/p>\n
The first step is to create a LabVIEW for Windows – as opposed to LabVIEW for FPGA – and to read in an entire NASDAQ ITCH 4.1 file. \u00a0This file is quite big, so I\u00a0wrote a quick Python script (used Ryan Day’s code as a guide:\u00a0https:\/\/github.com\/rday\/ITCH41<\/a>), and I generated a file with the following:<\/p>\n
- \n
- 1 Timestamp message<\/li>\n
- 5 Timestamp messages<\/li>\n
- 50 Timestamp messages<\/li>\n<\/ul>\n
The Python script can also output the seconds field from each message read in. \u00a0See the python script here:<\/p>\n
https:\/\/github.com\/JohnStratoudakis\/LabVIEW_Fpga\/blob\/master\/MarketData\/MarketData_01\/parseItch.py<\/a><\/p>\n
Now I have three files, named: T.itch, T.5.itch, and T.50.itch, and I will write a LabVIEW program to send all data from one of the files above in to the FPGA.<\/p>\n
Step 2 – Send Data From One File to FPGA via DMA FIFO<\/strong><\/p>\n
This requires knowledge of LabVIEW for Windows and some knowledge of LabVIEW for FPGA. \u00a0I wrote a simple User Interface that allows you to select a file. \u00a0That file is then sent to the FPGA using a Host-to-Target FIFO 1 byte at a time. \u00a0Since viewing a LabVIEW vi requires that you have LabVIEW installed on your computer, I took some screen shots of the LabVIEW code and placed them here:<\/p>\n
- \n
- https:\/\/github.com\/JohnStratoudakis\/LabVIEW_Fpga\/blob\/master\/MarketData\/MarketData_01\/Host-Send-Data.png<\/a><\/li>\n
- https:\/\/github.com\/JohnStratoudakis\/LabVIEW_Fpga\/blob\/master\/MarketData\/MarketData_01\/Host-Top-Level_1.png<\/a><\/li>\n<\/ul>\n
Step 3 – Interpret the Feed Inside the FPGA<\/strong><\/p>\n
Right now I have generated a file that contains, 1, 5, and 50 Timestamp messages. \u00a0So that means, for each messaged that is encountered, I will extract the timestamp, which will be in seconds, save it in to a local FPGA variable, and send the value back up to the host.<\/p>\n
Step 4 – Send Data Back to Host<\/strong><\/p>\n
The statistic will be the seconds portion of each message that was passed in. \u00a0The seconds field is a 32-bit integer, so the Target-to-Host DMA FIFO will be a 32-bit integer. \u00a0Here is a quick screen shot of the FPGA top-level loop, which reads the input data stream one byte at a time:<\/p>\n
![](\"http:\/\/fpganow.com\/wp-content\/uploads\/2017\/02\/FPGA-Entire-Loop-1024x547.png\")
<\/a><\/p>\nThe purple colored box is how the FPGA receives the data from the host. \u00a0In a live application, the purple colored box, also known as the “Read (FIFO Method)” can have this data come directly from a 10 gigabit connection, or from another loop inside the FPGA.<\/p>\n
![](\"http:\/\/fpganow.com\/wp-content\/uploads\/2017\/02\/FIFO-Read.png\")
<\/a><\/p>\n(Read more about this method on National Instruments website:\u00a0https:\/\/zone.ni.com\/reference\/en-XX\/help\/371599H-01\/lvfpgahost\/fpga_method_fifo_read\/<\/a>)<\/p>\n
As the data comes in, a counter is started at 0, and depending on the element count, the data is stored in different output variables. \u00a0The first 2 bytes are the message length, the 3rd is the message type, the 4th, 5th, 6th, and 7th elements are the Seconds portion of the message.<\/p>\n
The counter is then compared to the message length variable, and when they are equal, the output variable “Message Done” is set to true. \u00a0Here is the bottom portion of the screen shot from above:<\/p>\n
![](\"http:\/\/fpganow.com\/wp-content\/uploads\/2017\/02\/Fpga-End-of-Message.png\")
<\/a><\/p>\nAfter the end of the message is read, we have a case structure, which is similar to an if statement but for FPGAs, which will read the appropriate variables and send them back up to the host via a DMA-FIFO. \u00a0Now this DMA-FIFO can be configured to send data up to the host computer, or to another DMA-FIFO inside the FPGA. \u00a0For now we are going to send this up to the host for analysis.<\/p>\n
Take a look at the right-half of the original FPGA vi screenshot. \u00a0This element executes once, and reads the Seconds variable and sends it up to the host.<\/p>\n
In part 3, I will add another feature – Add Order with MPID, so we can now know in the FPGA when a new order is entered for a particular security, what side that order is on, and how many shares\/price. \u00a0This is more meaningful information that can be used to trade the markets, especially during a Donald Trump speech!<\/p>\n","protected":false},"excerpt":{"rendered":"
Skip directly to the source code on Github.com here: https:\/\/github.com\/JohnStratoudakis\/LabVIEW_Fpga\/tree\/master\/MarketData\/MarketData_01 So what now. \u00a0We know what a NASDAQ ITCH 4.1 Market Data Message looks like. \u00a0The format is very simple, there is some – yes – ASCII data in the message format, and all messages are preceded by the message length. \u00a0Message length preceding the … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","footnotes":""},"categories":[7,6,5],"tags":[],"class_list":["post-77","post","type-post","status-publish","format-standard","hentry","category-fpga","category-labview","category-market-data","masonry-post","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50"],"_links":{"self":[{"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/posts\/77","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/comments?post=77"}],"version-history":[{"count":9,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/posts\/77\/revisions"}],"predecessor-version":[{"id":92,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/posts\/77\/revisions\/92"}],"wp:attachment":[{"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/media?parent=77"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/categories?post=77"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/tags?post=77"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- https:\/\/github.com\/JohnStratoudakis\/LabVIEW_Fpga\/blob\/master\/MarketData\/MarketData_01\/Host-Top-Level_1.png<\/a><\/li>\n<\/ul>\n
- https:\/\/github.com\/JohnStratoudakis\/LabVIEW_Fpga\/blob\/master\/MarketData\/MarketData_01\/Host-Send-Data.png<\/a><\/li>\n
- https:\/\/www.youtube.com\/user\/Labview<\/a><\/li>\n<\/ul>\n
- http:\/\/www.ni.com\/labview\/<\/a><\/li>\n