{"id":60,"date":"2017-01-19T02:55:15","date_gmt":"2017-01-19T02:55:15","guid":{"rendered":"http:\/\/fpganow.com\/?p=60"},"modified":"2017-04-23T17:41:01","modified_gmt":"2017-04-23T17:41:01","slug":"filter-market-data-inside-an-fpga","status":"publish","type":"post","link":"https:\/\/fpganow.com\/index.php\/2017\/01\/19\/filter-market-data-inside-an-fpga\/","title":{"rendered":"Filter Market Data Messages in an FPGA &#8211; part 1"},"content":{"rendered":"<p>So I went to NASDAQs ftp site and downloaded the entire ITCH feed for November\u00a09th, 2013. \u00a0The file was large &#8211; 319MB compressed, and you can download it yourself from here:<\/p>\n<p><a href=\"ftp:\/\/emi.nasdaq.com\/ITCH\/11092013.NASDAQ_ITCH41.gz\">ftp:\/\/emi.nasdaq.com\/ITCH\/11092013.NASDAQ_ITCH41.gz<\/a>.<\/p>\n<p>NASDAQ has a very simple document describing the specification here:<\/p>\n<p><a href=\"http:\/\/nasdaqtrader.com\/content\/technicalsupport\/specifications\/dataproducts\/NQTV-ITCH-V4_1.pdf\">http:\/\/nasdaqtrader.com\/content\/technicalsupport\/specifications\/dataproducts\/NQTV-ITCH-V4_1.pdf<\/a><\/p>\n<p>I skimmed over the specification to get an idea of how Market Data works. \u00a0What I basically understand is that at the start of the trading day, NASDAQ sends a list of all securities that will be available to trade for that day following by a bunch of messages indicating changes to prices being offered to buy or sell for the security as well as actual trades.<\/p>\n<p>The basic format of an ITCH 4.1 Market Data message is the size of the message, followed by the data, where the first byte of the data is the message type. \u00a0So using this information we can easily decode an entire ITCH feed, paying attention only to messages that interest us.<\/p>\n<p><strong>Timestamp Message<\/strong><\/p>\n<table style=\"width: 240px;\">\n<tbody>\n<tr>\n<td style=\"width: 126px;\">0x05<\/td>\n<td style=\"width: 221px;\">Length<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 126px;\">0x54<\/td>\n<td style=\"width: 221px;\">Message Type &#8216;T&#8217;<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 126px;\">0x00<\/td>\n<td style=\"width: 221px;\">Second &#8211; byte 1<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 126px;\">0x00<\/td>\n<td style=\"width: 221px;\">Second &#8211; byte 2<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 126px;\">0x58<\/td>\n<td style=\"width: 221px;\">Second &#8211; byte 3<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 126px;\">0xb7<\/td>\n<td style=\"width: 221px;\">Second &#8211; byte 4<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The ITCH standard says that all Integer fields are in Big-Endian format, so the timestamp in the message above is interpreted as 0x00 00 58 b7, or 22,711. See a nice online hex to dec converter here:<\/p>\n<p><a href=\"http:\/\/www.binaryhexconverter.com\/hex-to-decimal-converter\">http:\/\/www.binaryhexconverter.com\/hex-to-decimal-converter<\/a><\/p>\n<p>Now 22,711 is the number of seconds since midnight, here is another online tool to convert this to a normal time in hours, minutes, and seconds:<\/p>\n<p><a href=\"https:\/\/www.tools4noobs.com\/online_tools\/seconds_to_hh_mm_ss\/\">https:\/\/www.tools4noobs.com\/online_tools\/seconds_to_hh_mm_ss\/<\/a><\/p>\n<p>So 22,711 is 06:18:31. \u00a0That is pretty early in the morning, so it looks like this particular NASDAQ ITCH feed starts with pre-market trading.<\/p>\n<p>I also used this online tool to convert ASCII to Hex:<\/p>\n<p><a href=\"http:\/\/www.asciitohex.com\/\">http:\/\/www.asciitohex.com\/<\/a><\/p>\n<p>The next message in this feed:<\/p>\n<p><strong>System Event Message<\/strong><\/p>\n<table style=\"width: 413px;\">\n<tbody>\n<tr>\n<td style=\"width: 116px;\">0x06<\/td>\n<td style=\"width: 282px;\">Length<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 116px;\">0x53<\/td>\n<td style=\"width: 282px;\">Message Type &#8216;S&#8217;<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 116px;\">0x11<\/td>\n<td style=\"width: 282px;\">Timestamp byte 1(nanoseconds since last Timestamp Seconds)<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 116px;\">0xcd<\/td>\n<td style=\"width: 282px;\">Timestamp byte 2<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 116px;\">0x6c<\/td>\n<td style=\"width: 282px;\">Timestamp byte 3<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 116px;\">0xc9<\/td>\n<td style=\"width: 282px;\">Timestamp byte 4<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 116px;\">0x4f<\/td>\n<td style=\"width: 282px;\">Event Code<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The possible event codes are:<\/p>\n<ul class=\"ili-indent\">\n<li>Daily\n<ul>\n<li>&#8216;O&#8217; &#8211; (0x4f) &#8211; Start of Messages<\/li>\n<li>&#8216;S&#8217; &#8211; (0x53) &#8211; Start of System Hours<\/li>\n<li>&#8216;Q&#8217; &#8211; (0x51) &#8211; Start of Market Hours<\/li>\n<li>&#8216;M&#8217; &#8211; (0x4d) &#8211; End of Market Hours<\/li>\n<li>&#8216;C&#8217; &#8211; (0x43) &#8211; End of Messages<\/li>\n<\/ul>\n<\/li>\n<li>As Needed &#8211; In the event of an emergency market condition\n<ul>\n<li>&#8216;A&#8217; &#8211; (0x41) &#8211; Emergency Market Condition &#8211; Halt<\/li>\n<li>&#8216;R&#8217; &#8211; (0x52) &#8211; Emergency Market Condition &#8211; Quote Only Period<\/li>\n<li>&#8216;B&#8217; &#8211; (0x42) &#8211; Emergency Market Condition &#8211; Resumption<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>I included the &#8220;As Needed&#8221; Event Codes, because that is when things will get real bad, and you will probably want your FPGA to get ready to liquidate everything in your portfolio&#8230; More on that in the future, but for now we must stayed focused on trading Trump and his tweets.<\/p>\n<p>So 0x4f means start of messages. \u00a0Okay, continuing, I see\u00a0a few more system event messages as well as some Timestamp messages. \u00a0I skip these for now and come to the next message which sounds interesting and is a Stock Directory Message<\/p>\n<p><strong>Stock Directory Message<\/strong><\/p>\n<table style=\"width: 767px;\">\n<tbody>\n<tr>\n<td style=\"width: 89.6094px;\"><em><span style=\"text-decoration: underline;\">byte #<\/span><\/em><\/td>\n<td style=\"width: 91.2188px;\"><em><span style=\"text-decoration: underline;\">length<\/span><\/em><\/td>\n<td style=\"width: 107.219px;\"><em><span style=\"text-decoration: underline;\">data<\/span><\/em><\/td>\n<td style=\"width: 435.609px;\"><em><span style=\"text-decoration: underline;\">description<\/span><\/em><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">0<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x14 (decimal: 20)<\/td>\n<td style=\"width: 435.609px;\">Length<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">1<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x52<\/td>\n<td style=\"width: 435.609px;\">Message Type &#8216;R&#8217;<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">2-5<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">1d \u00a048 bd c7<\/td>\n<td style=\"width: 435.609px;\">Timestamp Nanoseconds<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">6-13<\/td>\n<td style=\"width: 91.2188px;\">8<\/td>\n<td style=\"width: 107.219px;\">41 20 20 20 20 20 20 20<\/td>\n<td style=\"width: 435.609px;\">Stock (0x20 is a space, 0x41 is A) So this is for Agilent (http:\/\/finance.yahoo.com\/quote\/A?p=A)<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">14<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">4e (&#8216;N&#8217;)<\/td>\n<td style=\"width: 435.609px;\">Market Category &#8211; simply the exchange:<\/p>\n<ul class=\"ili-indent\">\n<li style=\"list-style-type: none;\">\n<ul class=\"\">\n<li>N &#8211; NYSE<\/li>\n<li>A &#8211; NYSE Amex<\/li>\n<li>P &#8211; NYSE Arca<\/li>\n<li>Q &#8211; NASDAQ Global Select Market<\/li>\n<li>G &#8211; NASDAQ Global MarketSM<\/li>\n<li>S &#8211; NASDAQ Capital Market<\/li>\n<li>Z &#8211; BATS BZX Exchange<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">15<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">20 (space)<\/td>\n<td style=\"width: 435.609px;\">Financial Status Indicator &#8211; Indicates when a firm is not in compliance with NASDAQ continued listing requirements. \u00a0This sounds like a way to find distressed stocks that are about to be delisted &#8211; lots of volatility with low volume.<\/p>\n<ul class=\"ili-indent\">\n<li>D &#8211; Deficient<\/li>\n<li>E &#8211; Delinquent<\/li>\n<li>Q &#8211; Bankrupt<\/li>\n<li>S &#8211; Suspended<\/li>\n<li>G &#8211; Deficient and Bankrupt<\/li>\n<li>H &#8211; Deficient and Delinquent<\/li>\n<li>J &#8211; Delinquent and Bankrupt<\/li>\n<li>K &#8211; Deficient, Delinquent and Bankrupt<\/li>\n<li>Space &#8211; Company is in compliance<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">16-19<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">00 00 00 64<\/p>\n<p>(decimal: 100)<\/td>\n<td style=\"width: 435.609px;\">Round lot size<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">20<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">4e (N)<\/td>\n<td style=\"width: 435.609px;\">Round lots only &#8211; indicates if NASDAQ only accepts orders in round lot size<\/p>\n<ul class=\"ili-indent\">\n<li>Y &#8211; only round lots are accepted in this stock<\/li>\n<li>N &#8211; odd\/mixed lots are allowed<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Okay, so we can now decode a couple of message types. \u00a0But this is not important and is only distracting us from the goal. \u00a0We can decode all of these messages inside a MicroBlaze Soft Core Processor running embedded C++ inside the FPGA, and then send a message to the rest of the FPGA telling it how to deal with the rest. \u00a0So let&#8217;s keep scanning this Market Data file and find a message that helps us with trading.<\/p>\n<p>I encounter a message of type &#8220;Reg SHO Short Sale Price Test Restricted Indicator&#8221;. \u00a0I did some googling and found that Reg SHO is yet another loophole-filled attempt by regulators to prevent naked shorting. \u00a0My favorite sentence from this Investopedia page\u00a0<a href=\"http:\/\/www.investopedia.com\/terms\/r\/regsho.asp\">http:\/\/www.investopedia.com\/terms\/r\/regsho.asp<\/a>\u00a0is:<\/p>\n<p>&#8220;a broker has reasonable belief that the equity to be short sold can be borrowed and delivered to a short seller on a specific date before short selling can occur&#8221;<\/p>\n<p>Anyway, let&#8217;s keep scanning this file. \u00a0I find a message of type &#8216;H&#8217; &#8211; Stock Trading Action Message, this sounds pretty good&#8230;<\/p>\n<p><strong>Stock Trading Action\u00a0Message<\/strong><\/p>\n<table style=\"width: 767px;\">\n<tbody>\n<tr>\n<td style=\"width: 89.6094px;\"><em><span style=\"text-decoration: underline;\">byte #<\/span><\/em><\/td>\n<td style=\"width: 91.2188px;\"><em><span style=\"text-decoration: underline;\">length<\/span><\/em><\/td>\n<td style=\"width: 107.219px;\"><em><span style=\"text-decoration: underline;\">data<\/span><\/em><\/td>\n<td style=\"width: 435.609px;\"><em><span style=\"text-decoration: underline;\">description<\/span><\/em><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">0<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x13 (decimal: 19)<\/td>\n<td style=\"width: 435.609px;\">Length<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">1<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x48<\/td>\n<td style=\"width: 435.609px;\">Message Type &#8216;H&#8217;<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">2-5<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">1d 50 bd 0d<\/td>\n<td style=\"width: 435.609px;\">Timestamp Nanoseconds<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">6-13<\/td>\n<td style=\"width: 91.2188px;\">8<\/td>\n<td style=\"width: 107.219px;\">41 42 2d 20 20 20 20 20<\/td>\n<td style=\"width: 435.609px;\">Stock Symbol: AB-<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">14<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x54 (T)<\/td>\n<td style=\"width: 435.609px;\">Trading State<\/p>\n<ul class=\"ili-indent\">\n<li>H &#8211; Halted across all U.S. equity markets \/ SROs<\/li>\n<li>P &#8211; Paused across all U.S. equity markets \/ SROs (NASDAQ-listed securities only)<\/li>\n<li>Q &#8211; Quotation only period for cross-SRO halt or pause<\/li>\n<li>T &#8211; Trading on NASDAQ<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">15<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">20 (space)<\/td>\n<td style=\"width: 435.609px;\">Reserved<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">16-19<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">20 20 20 20<\/td>\n<td style=\"width: 435.609px;\">Trading Action Reason &#8211; I guess all spaces means &#8220;no reason&#8221; or &#8220;nothing to worry about&#8221;<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Okay, so it looks like the Trading Action Message just tells us when trading is officially open&#8230;etc. \u00a0Nothing of use here for the FPGA plan. \u00a0Anyway, on to the next message type that I find.<\/p>\n<p>I continue running\u00a0my analysis script and I find the following:<\/p>\n<ul class=\"ili-indent\">\n<li>Add Order with MPID<\/li>\n<li>Add Order Message<\/li>\n<li>Order Delete Message<\/li>\n<\/ul>\n<p>This time I will not analyze the &#8220;Add Order with MPID&#8221; message unless it appears to have something of use. \u00a0The Message Type is &#8216;F&#8217;, and it looks like this message will be useful to us. \u00a0Additionally, MPID stands for &#8220;Market Participant\u00a0Identifier&#8221; and appears to simply be your broker. \u00a0See:<\/p>\n<p><a href=\"https:\/\/www.interactivebrokers.com\/en\/index.php?f=705\">https:\/\/www.interactivebrokers.com\/en\/index.php?f=705<\/a><\/p>\n<p>So, here is the analysis:<\/p>\n<p><strong>Add Order with MPID Attribution<\/strong><\/p>\n<table style=\"width: 767px;\">\n<tbody>\n<tr>\n<td style=\"width: 89.6094px;\"><em><span style=\"text-decoration: underline;\">byte #<\/span><\/em><\/td>\n<td style=\"width: 91.2188px;\"><em><span style=\"text-decoration: underline;\">length<\/span><\/em><\/td>\n<td style=\"width: 107.219px;\"><em><span style=\"text-decoration: underline;\">data<\/span><\/em><\/td>\n<td style=\"width: 435.609px;\"><em><span style=\"text-decoration: underline;\">description<\/span><\/em><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">0<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x22 (decimal: 34)<\/td>\n<td style=\"width: 435.609px;\">Length<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">1<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x46<\/td>\n<td style=\"width: 435.609px;\">Message Type &#8216;F&#8217;<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">2-5<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">\n<p class=\"p1\"><span class=\"s1\">02 43 d1 46<\/span><\/p>\n<\/td>\n<td style=\"width: 435.609px;\">Nanoseconds portion of timestamp<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">6-13<\/td>\n<td style=\"width: 91.2188px;\">8<\/td>\n<td style=\"width: 107.219px;\">00 00 00 00 00 00 16 b2<\/td>\n<td style=\"width: 435.609px;\">Order number &#8211; unique and is assigned during order entry<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">14<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x42<\/td>\n<td style=\"width: 435.609px;\">Type of order<\/p>\n<ul class=\"ili-indent\">\n<li>&#8216;B&#8217; &#8211; (0x42) &#8211; Buy<\/li>\n<li>&#8216;S&#8217; &#8211; (0x53) &#8211; Sell<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">15-18<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">00 00 00 64 (decimal: 100)<\/td>\n<td style=\"width: 435.609px;\">Number of shares<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">19-26<\/td>\n<td style=\"width: 91.2188px;\">8<\/td>\n<td style=\"width: 107.219px;\">\n<p class=\"p1\"><span class=\"s1\">5a 56 5a 5a 54 20 20 20<\/span><\/p>\n<\/td>\n<td style=\"width: 435.609px;\">The stock symbol:<\/p>\n<p>ZVZZT<\/p>\n<p>Funny&#8230; this looks like a test symbol, take a look at the Bloomberg quote:<\/p>\n<p><a href=\"https:\/\/www.bloomberg.com\/quote\/ZVZZT:US\">https:\/\/www.bloomberg.com\/quote\/ZVZZT:US<\/a><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">27-30<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">\n<p class=\"p1\"><span class=\"s1\">00 02 97 ac <\/span><\/p>\n<\/td>\n<td style=\"width: 435.609px;\">Display price, converts to: $16.99<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">31-34<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">\u00a04c 45 48 4d<\/td>\n<td style=\"width: 435.609px;\">NASDAQ MPID: LEHM (Is that Lehman Brothers?)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>So I am looking at the NASDAQ ITCH 4.1 specification (as you have obviously realized by now) and I see that the order immediately before &#8220;Add Order with MPID Attribution&#8221; is &#8220;Add Order &#8211; No MPID Attribution&#8221;, which is exactly the same as the message above, but without the last field. \u00a0The only other difference is that the message type is &#8216;A&#8217;.<\/p>\n<p><strong>A word about Display Price<\/strong><\/p>\n<p>So the specification says that you convert all Integer price values and treat them as fixed pointed numbers with the first 6 places representing the integer portion and the remaining 4 representing decimal digits, and this leads to a maximum price of 200,000.0000.<\/p>\n<p>So in the example message above:<\/p>\n<p>000297ac is equal to 169900, which is 16.9900, or simply $16.99.<\/p>\n<p>(see\u00a0<a href=\"http:\/\/www.binaryhexconverter.com\/hex-to-decimal-converter\">http:\/\/www.binaryhexconverter.com\/hex-to-decimal-converter<\/a>)<\/p>\n<p>Okay, so we know when the market opens and trading is started, stopped&#8230;etc We know when a new order is added to the NASDAQ order book. \u00a0What is next? Order Delete!<\/p>\n<p>Order Delete is different from Order Cancel. \u00a0Order Delete means that the entire order is removed or deleted from the Order Book, Order Cancel is whenever a portion of an order is cancelled.<\/p>\n<p><strong>Order Delete Message<\/strong><\/p>\n<table style=\"width: 767px;\">\n<tbody>\n<tr>\n<td style=\"width: 89.6094px;\"><em><span style=\"text-decoration: underline;\">byte #<\/span><\/em><\/td>\n<td style=\"width: 91.2188px;\"><em><span style=\"text-decoration: underline;\">length<\/span><\/em><\/td>\n<td style=\"width: 107.219px;\"><em><span style=\"text-decoration: underline;\">data<\/span><\/em><\/td>\n<td style=\"width: 435.609px;\"><em><span style=\"text-decoration: underline;\">description<\/span><\/em><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">0<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x0D (decimal: 13)<\/td>\n<td style=\"width: 435.609px;\">Length<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">1<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x44<\/td>\n<td style=\"width: 435.609px;\">Message Type &#8216;D&#8217;<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">2-5<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">\n<p class=\"p1\"><span class=\"s1\">21 95 1b cf<\/span><\/p>\n<\/td>\n<td style=\"width: 435.609px;\">Nanoseconds portion of timestamp<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">6-13<\/td>\n<td style=\"width: 91.2188px;\">8<\/td>\n<td style=\"width: 107.219px;\">\n<p class=\"p1\"><span class=\"s1\">00 00 00 00 00 00 031 6b<\/span><\/p>\n<\/td>\n<td style=\"width: 435.609px;\">Order reference number<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><strong>Order Cancel\u00a0Message<\/strong><\/p>\n<table style=\"width: 767px;\">\n<tbody>\n<tr>\n<td style=\"width: 89.6094px;\"><em><span style=\"text-decoration: underline;\">byte #<\/span><\/em><\/td>\n<td style=\"width: 91.2188px;\"><em><span style=\"text-decoration: underline;\">length<\/span><\/em><\/td>\n<td style=\"width: 107.219px;\"><em><span style=\"text-decoration: underline;\">data<\/span><\/em><\/td>\n<td style=\"width: 435.609px;\"><em><span style=\"text-decoration: underline;\">description<\/span><\/em><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">0<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x (decimal: )<\/td>\n<td style=\"width: 435.609px;\">Length<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">1<\/td>\n<td style=\"width: 91.2188px;\">1<\/td>\n<td style=\"width: 107.219px;\">0x58<\/td>\n<td style=\"width: 435.609px;\">Message Type &#8216;X&#8217;<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">2-5<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">\n<p class=\"p1\"><span class=\"s1\">21 dd db e1<\/span><\/p>\n<\/td>\n<td style=\"width: 435.609px;\">Nanoseconds portion of timestamp<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">6-13<\/td>\n<td style=\"width: 91.2188px;\">8<\/td>\n<td style=\"width: 107.219px;\">\n<p class=\"p1\"><span class=\"s1\">00 00 00 00 00 25 42 b7<\/span><\/p>\n<\/td>\n<td style=\"width: 435.609px;\">Order reference number<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89.6094px;\">14-17<\/td>\n<td style=\"width: 91.2188px;\">4<\/td>\n<td style=\"width: 107.219px;\">\n<p class=\"p1\"><span class=\"s1\">00 00 00 <\/span><span class=\"s1\">0a <\/span><\/p>\n<\/td>\n<td style=\"width: 435.609px;\">Number of shares being removed from order. \u00a0Hmmm&#8230; is someone backing out of a position? Scared!!! that means sell! Eh, maybe not, they only removed 10 shares. \u00a0But what if they are trading a high-priced stock like Amazon or Google? Oops, I meant Alphabet&#8230; and what about Berkshire Hathaway?<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Okay, so we have Add Order, Remove Order, Shorten Order. \u00a0What is left? Order Execute and Order Modify. \u00a0I will go with Order Executed.<\/p>\n<p><strong>Order Executed\u00a0Message<\/strong><\/p>\n<table style=\"width: 766px;\">\n<tbody>\n<tr>\n<td style=\"width: 89px;\"><em><span style=\"text-decoration: underline;\">byte #<\/span><\/em><\/td>\n<td style=\"width: 91px;\"><em><span style=\"text-decoration: underline;\">length<\/span><\/em><\/td>\n<td style=\"width: 107px;\"><em><span style=\"text-decoration: underline;\">data<\/span><\/em><\/td>\n<td style=\"width: 435px;\"><em><span style=\"text-decoration: underline;\">description<\/span><\/em><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">0<\/td>\n<td style=\"width: 91px;\">1<\/td>\n<td style=\"width: 107px;\">0x19 (decimal: )<\/td>\n<td style=\"width: 435px;\">Length<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">1<\/td>\n<td style=\"width: 91px;\">1<\/td>\n<td style=\"width: 107px;\">0x45<\/td>\n<td style=\"width: 435px;\">Message Type &#8216;E&#8217;<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">2-5<\/td>\n<td style=\"width: 91px;\">4<\/td>\n<td style=\"width: 107px;\">\n<p class=\"p1\"><span class=\"s1\">18 45 cd 07<\/span><\/p>\n<\/td>\n<td style=\"width: 435px;\">Nanoseconds portion of timestamp<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">6-13<\/td>\n<td style=\"width: 91px;\">8<\/td>\n<td style=\"width: 107px;\">\n<p class=\"p1\"><span class=\"s1\">\u00a0<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\">00 00 00 00 00 00 e9 ca<\/span><\/p>\n<\/td>\n<td style=\"width: 435px;\">Order reference number<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">14-17<\/td>\n<td style=\"width: 91px;\">4<\/td>\n<td style=\"width: 107px;\">\u00a000 00 03 e8<\/td>\n<td style=\"width: 435px;\">Number of shares executed<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">18-25<\/td>\n<td style=\"width: 91px;\">8<\/td>\n<td style=\"width: 107px;\">\n<p class=\"p1\"><span class=\"s1\">00 00 00 00 00 00 00 01<\/span><\/p>\n<\/td>\n<td style=\"width: 435px;\">\u00a0NASDAQ generated day-unique match number.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><strong>Order Executed\u00a0with Price Message<\/strong><\/p>\n<table style=\"width: 766px;\">\n<tbody>\n<tr>\n<td style=\"width: 89px;\"><em><span style=\"text-decoration: underline;\">byte #<\/span><\/em><\/td>\n<td style=\"width: 91px;\"><em><span style=\"text-decoration: underline;\">length<\/span><\/em><\/td>\n<td style=\"width: 107px;\"><em><span style=\"text-decoration: underline;\">data<\/span><\/em><\/td>\n<td style=\"width: 435px;\"><em><span style=\"text-decoration: underline;\">description<\/span><\/em><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">0<\/td>\n<td style=\"width: 91px;\">1<\/td>\n<td style=\"width: 107px;\">0x1e (decimal: )<\/td>\n<td style=\"width: 435px;\">Length<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">1<\/td>\n<td style=\"width: 91px;\">1<\/td>\n<td style=\"width: 107px;\">0x43<\/td>\n<td style=\"width: 435px;\">Message Type &#8216;C&#8217;<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">2-5<\/td>\n<td style=\"width: 91px;\">4<\/td>\n<td style=\"width: 107px;\">\n<p class=\"p1\"><span class=\"s1\">01 74 a8 ca<\/span><\/p>\n<\/td>\n<td style=\"width: 435px;\">Nanoseconds portion of timestamp<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">6-13<\/td>\n<td style=\"width: 91px;\">8<\/td>\n<td style=\"width: 107px;\">\n<p class=\"p1\"><span class=\"s1\">\u00a0<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\">00 00 00 00 00 00 31 9b<\/span><\/p>\n<\/td>\n<td style=\"width: 435px;\">Order reference number<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">14-17<\/td>\n<td style=\"width: 91px;\">4<\/td>\n<td style=\"width: 107px;\">\u00a000 00 00 64 (decimal: 100)<\/td>\n<td style=\"width: 435px;\">Number of shares executed.<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">\u00a018-25<\/td>\n<td style=\"width: 91px;\">8<\/td>\n<td style=\"width: 107px;\">\n<p class=\"p1\"><span class=\"s1\">00 00 00 00 00 00 03 b0<\/span><\/p>\n<\/td>\n<td style=\"width: 435px;\">\u00a0NASDAQ generated day-unique match number.<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">26<\/td>\n<td style=\"width: 91px;\">1<\/td>\n<td style=\"width: 107px;\">\n<p class=\"p1\"><span class=\"s1\">4e<\/span><\/p>\n<\/td>\n<td style=\"width: 435px;\">Printable<\/p>\n<ul class=\"ili-indent\">\n<li>&#8216;Y&#8217; &#8211; (0x59)<\/li>\n<li>&#8216;N&#8217; &#8211; (0x4e)<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 89px;\">27-30<\/td>\n<td style=\"width: 91px;\">4<\/td>\n<td style=\"width: 107px;\">\n<p class=\"p1\"><span class=\"s1\">00 02 9c 5c<\/span><\/p>\n<\/td>\n<td style=\"width: 435px;\">Execution Price:\u00a017.1100, or $17.11<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>I think that I have enough information to get started with the FPGA portion of this. \u00a0So I will skip analyzing the Order Replace Message for now. \u00a0So&#8230;.<\/p>\n<p>We have analyzed the following message types:<\/p>\n<ul class=\"ili-indent\">\n<li><strong>Timestamp Message<\/strong><\/li>\n<li><strong>System Event Message<\/strong><\/li>\n<li><strong>Stock Directory Message<\/strong><\/li>\n<li><strong>Stock Trading Action\u00a0Message<\/strong><\/li>\n<li><strong>Add Order &#8211; No MPID Attribution (well, kind of)<\/strong><\/li>\n<li><strong>Add Order with MPID Attribution<\/strong><\/li>\n<li><strong>Order Delete Message<\/strong><\/li>\n<li><strong>Order Cancel\u00a0Message<\/strong><\/li>\n<li><strong>Order Executed\u00a0Message<\/strong><\/li>\n<li><strong>Order Executed\u00a0with Price Message<\/strong><\/li>\n<\/ul>\n<p>What is missing? I am looking at the ITCH 4.1 specification and I see the following message types:<\/p>\n<ul class=\"ili-indent\">\n<li>Order Replace<\/li>\n<li>Trade Message (Non-Cross)<\/li>\n<li>Cross Trade Message<\/li>\n<li>Broken Trade \/ Order Execution Message<\/li>\n<li>Net Order Imbalance Indicator (NOII) Message<\/li>\n<li>Retail Price Improvement Indicator (RPII)<\/li>\n<\/ul>\n<p>So I can go ahead and analyze the rest of the messages, but I want to get to some coding. \u00a0Coding as in drawing some LabVIEW program (no, not writing, but drawing). \u00a0Anyway, I will take a look at how to write (aka draw) some LabVIEW for FPGA code that can handle the parsing and interpreting of the necessary information to react or trade to a Trump Event. \u00a0Stay tuned.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>So I went to NASDAQs ftp site and downloaded the entire ITCH feed for November\u00a09th, 2013. \u00a0The file was large &#8211; 319MB compressed, and you can download it yourself from here: ftp:\/\/emi.nasdaq.com\/ITCH\/11092013.NASDAQ_ITCH41.gz. NASDAQ has a very simple document describing the specification here: http:\/\/nasdaqtrader.com\/content\/technicalsupport\/specifications\/dataproducts\/NQTV-ITCH-V4_1.pdf I skimmed over the specification to get an idea of how Market &#8230; <a title=\"Filter Market Data Messages in an FPGA &#8211; part 1\" class=\"read-more\" href=\"https:\/\/fpganow.com\/index.php\/2017\/01\/19\/filter-market-data-inside-an-fpga\/\" aria-label=\"Read more about Filter Market Data Messages in an FPGA &#8211; part 1\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","footnotes":""},"categories":[4,5],"tags":[],"class_list":["post-60","post","type-post","status-publish","format-standard","hentry","category-hft","category-market-data","masonry-post","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50"],"_links":{"self":[{"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/posts\/60","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/comments?post=60"}],"version-history":[{"count":16,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/posts\/60\/revisions"}],"predecessor-version":[{"id":122,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/posts\/60\/revisions\/122"}],"wp:attachment":[{"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/media?parent=60"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/categories?post=60"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fpganow.com\/index.php\/wp-json\/wp\/v2\/tags?post=60"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}