XDP Common Client Specification
XDP Common Client Specification
Version Date
2.0m October 4, 2016
© 2016 NYSE. All rights reserved. No part of this material may be copied, photocopied or duplicated in any form
by any means or redistributed without the prior written consent of NYSE. All third party trademarks are owned
by their respective owners and are used with permission. NYSE and its affiliates do not recommend or make any
representation as to possible benefits from any securities or investments, or third-party products or services.
Investors should undertake their own due diligence regarding securities and investment practices. This material
may contain forward-looking statements regarding NYSE and its affiliates that are based on the current beliefs
and expectations of management, are subject to significant risks and uncertainties, and which may differ from
actual results. NYSE does not guarantee that its products or services will result in any savings or specific
outcome. All data is as of October 4, 2016. NYSE disclaims any duty to update this information.
Global OTC is an Alternative Trading System (“ATS”) registered with the U.S. Securities and Exchange
Commission (“SEC”) and operated by Archipelago Trading Services, Inc. (“ATSI”), a broker-dealer registered
with the SEC and a member of the Financial Industry Regulatory Authority (“FINRA”). Although ATSI is a
wholly-owned subsidiary of NYSE Group, Inc., Global OTC is not a stock exchange or self-regulatory
organization. The OTC equity securities traded on Global OTC are not U.S. exchange listed securities.
ICE/NYSE XDP COMMON CLIENT SPECIFICATION V2.0M
Preface
DOCUMENT HISTORY
The following table provides a description of recent changes to this document.
1.5 11/6/2013 Section 1.3.14 Symbol Index Mapping FTP – Update to Listed Markets, Section 2.13
Message Body – Update to Security Types
Section 2.18 MSG
Body – Updates to various fields
1.6 4/9/2014 Updated Security Status and Symbol Index messages to include notes reflecting
current implementation
Updated Security Status fields to have standalone SSRState, MarketState, and
SessionState fields
2.0 01/08/2015 Re-branded for ICE
Structure and explanatory text revised for clarity and simplicity
NYSE Security Status msg’s Transaction ID field changed to Reserved
Source Time Reference msg’s SymbolIndex field changed to System ID IBF 2.0
2.0a 02/16/2015 Noted shorter length of Symbol Index Mapping (msg type 3) and Security Status
(msg type 34) for Arca Integrated Feed only
Minor tweaks to language
2.0b 02/18/2015 Corrections in section 3.6.2 regarding Order ID’s
Corrected Packet Header DeliveryFlags for Seq Num Reset messages (4.3 and 9.2)
2.0c 03/10/2015 Corrected Security Status msg length to 46
Added detail about multicast priming in Section 9
Added detail about refresh formats in 5.1.3
Added detail about IP Table filtering in 8.4
2.0d 04/07/2015 Clarified descriptions and availability of last 8 fields in the Security Status msg
2.0e 05/12/2015 Modified Security Status msg, Security Status field to reflect Arca adoption of NYSE
SSR values
Clarified the description of Time field in Security status message and System ID field
in Order Acknowledgment message
2.0f 06/30/2015 Updated the Exchange Code field values in the Symbol Index Mapping Message to
include Global OTC primary symbols
Updated SSR Triggering Exchange ID field values in Security Status Message
Updated Listed Market values in the Symbol Index Mapping File format to include
the Listed Market for Global OTC primary symbols
2.0g 07/10/2015 Updated legal disclaimer for Global OTC on title page
2.0i Mar 18, 2016 Corrected fields in the symbol index ftp file
2.0k June 16, 2016 Restored guidance on handling the Session Change msg for users of Arca
Integrated and ArcaBook feeds. This was incorrectly removed from version 2.0.
2.0L Aug 18, 2016 Clarified trailing pipe characters in Symbol Index Mapping File (section 10)
2.0m Oct 4, 2016 Updated Security Status message to include V as the exchange ID for IEX.
Updated section 2.2 to reflect change of heartbeat frequency from 60 secs to 1 sec.
REFERENCE MATERIAL
The following lists the associated documents, which either should be read in conjunction with this document or which
provide other relevant information for the user:
SFTI US Technical Specification
SFTI US Customer Guide
NYSE Symbology
CONTACT INFORMATION
Service Desk (NYSE, NYSE MKT, NYSE Arca)
Australia Toll Free: +61.1800506227
China Toll Free: +86.4008822979
US Toll Free +1.866.873.7422
UK Toll Free 0808.101.3866
US Paid +1.212.656.3640
Email: service.desk@nyx.com
FURTHER INFORMATION
For additional information about market data products, visit http://www.nyxdata.com/Data-Products
For updated capacity figures, visit our capacity pages at: http://www.nyxdata.com/capacity
For details of IP addresses, visit our IP address pages at: http://www.nyxdata.com/ipaddresses
For a full glossary, visit: http://www.nyxdata.com/glossary/
Contents
Preface 2
Document History .............................................................................................................................................................. 2
Reference Material ............................................................................................................................................................ 3
Contact Information ........................................................................................................................................................... 3
Further Information ............................................................................................................................................................ 3
Contents ............................................................................................................................................................................. 4
1. Introduction .......................................................................................................................................................... 5
1.1 Receiving Real Time Market Data.......................................................................................................................... 5
1.2 Recovering from Errors .......................................................................................................................................... 5
2. Packets and Heartbeats ....................................................................................................................................... 6
2.1 Packet Header ....................................................................................................................................................... 6
2.2 Heartbeats ............................................................................................................................................................. 6
3. Message Field Content ........................................................................................................................................ 7
3.1 Message Header .................................................................................................................................................... 7
3.2 Date and Time Conventions ................................................................................................................................... 8
3.3 Sequence Numbers ............................................................................................................................................... 9
3.4 Symbol Sequence Numbers .................................................................................................................................. 9
3.5 Prices ..................................................................................................................................................................... 9
3.6 Order ID’s and Trade ID’s ...................................................................................................................................... 9
3.7 Symbol Indexes ................................................................................................................................................... 10
4. Messages Sent by the Publisher ...................................................................................................................... 11
4.1 Symbol Index Mapping Message (Msg Type 3) ................................................................................................... 11
4.2 Security Status Message (Msg Type 34) ............................................................................................................. 13
4.3 Sequence Number Reset Message (Msg Type 1) ............................................................................................... 16
4.4 Source Time Reference Message (Msg Type 2) .................................................................................................. 16
4.5 Symbol Clear Message (Msg Type 32) ................................................................................................................ 17
4.6 Trading Session Change Message (Msg Type 33) .............................................................................................. 17
5. Messages Sent by Refresh and Retrans Servers Only ................................................................................... 18
5.1 Refresh Header (Msg Type 35) ............................................................................................................................ 18
5.2 Message Unavailable Message (Msg Type 31) ................................................................................................... 20
6. Messages Sent by the Client to the Request Server ....................................................................................... 21
6.1 Retransmission Request Message (Msg Type 10)............................................................................................... 21
6.2 Refresh Request Message (Msg Type 15) ........................................................................................................... 22
6.3 Symbol Index Mapping Request Message (Msg Type 13) ................................................................................... 23
6.4 Heartbeat Response Message (Msg Type 12) ..................................................................................................... 23
7. Messages Sent by Request Server ................................................................................................................... 24
7.1 Request Response Message (Msg Type 11) ....................................................................................................... 24
8. Error Handling and the Request Server ........................................................................................................... 25
8.1 Handling Sequence Number Gaps ...................................................................................................................... 25
8.2 Recovering from Client Late Starts or Intraday Failures....................................................................................... 25
8.3 Refreshing Symbol Information ............................................................................................................................ 26
8.4 Request Server .................................................................................................................................................... 26
9. Operational Information .................................................................................................................................... 28
9.1 System Behavior on Start and Restart ................................................................................................................. 28
9.2 XDP Publisher Failover ........................................................................................................................................ 28
9.3 Disaster Recovery Site......................................................................................................................................... 28
9.4 XDP Production Hours ......................................................................................................................................... 28
9.5 XDP Test Hours ................................................................................................................................................... 28
10. Symbol Index Mapping File ............................................................................................................................... 29
1. Introduction
Retrans/
Real-Time Refresh Retransmission Refresh Service
Service Request Service
Service
Dual Dual Dual
Multicast TCP/IP Multicast Multicast
Channels Channels Channels
Client
In case of dropped multicast packets, the client can connect to a Request Server via TCP/IP to request
retransmissions of missed messages.
In case of client late start or intraday failure, the client can connect to the Request Server and request snapshot
refreshes of the state of the market.
At system startup, each channel publishes referential data about all symbols published on the channel. If a client
process misses this initial spin of symbol data, he can connect to the Request Server and request a refresh of some
or all of the missed data.
In response to these requests, retransmission and refresh data is published by the exchange over dedicated multicast
channels which correspond one-to-one with the real-time channels.
See Error Handling and the Request Server for complete information.
SIZE
FIELD NAME OFFSET FORMAT DESCRIPTION
(BYTES)
PktSize 0 2 Binary The size of the packet in bytes, including this 16
-byte packet header
SendTime 8 4 Binary The time when this packet was published to the
multicast channel, in seconds since Jan 1, 1970
00:00:00 UTC.
2.2 HEARTBEATS
To assist the client in confirming connection health, application heartbeats are sent once a minute by the Request
Server, and once a second by the real-time publishing servers (data, refresh and retransmissions channels).
A heartbeat consists of a packet containing a Packet Header and no messages. The Packet Header’s Delivery Flag is
set to 1 and Number Msgs is 0. Since a heartbeat packet contains no messages, a heartbeat does not increment the
next expected sequence number. See Sequence Numbers.
Heartbeats sent by the Request Server must be acknowledged by the client. See Request Server.
SIZE
FIELD NAME OFFSET FORMAT DESCRIPTION
(BYTES)
MsgSize 0 2 Binary The size of this message in bytes
SourceTimeNS 4 4 Binary
SymbolIndex 8 4 Binary
OrderID 12 4 Binary
Price 16 4 Binary
Market-specific
Volume 28 4 Binary Not published in Arca market
content
The variable message size can also insulate client code from future field additions that you may not need.
In example 2, an existing message type is 16 bytes long.
Example 2: Release N
SourceTimeNS 4 4 Binary
SymbolIndex 8 4 Binary
Price 12 4 Binary
In a future release, a four-byte volume field will be added, increasing the Msg Size to 20 bytes.
If the client wishes to delay upgrading his feed handler for the new content, no coding is needed at the time of the
release. Proper coding of the MsgSize field up front allows the client to handle the unforeseen 20-byte format. On his
own schedule, the client can upgrade his feed handler to process the new field.
Msg Size 0 2 Binary Size of the message: 20 bytes Look at the Msg
Size field to know
Msg Type 2 2 Binary The type of this message: where the next
999 – Price message example message starts.
SourceTimeNS 4 4 Binary
3.5 PRICES
All price fields are published as unsigned binary integers. To interpret a price correctly, the client must use the published
price value as a numerator along with the Price Scale Code in the symbol’s Symbol Index Mapping Message (Msg Type
3) as follows:
Numerator
Price
10 PriceScaleCode
For example, a price of $27.56 is represented as a published price field of 2756 and a PriceScaleCode of 2.
The table below shows the data structure required to combine the Order ID, Market ID, System ID, and GTC Indicator to
obtain the full UTP matching engine Order ID. The table assumes the client byte ordering is Little Endian. If the client
byte ordering is Big Endian, the byte order is reversed.
SymbolIndex 4 4 Binary The unique ID of this symbol for all products within this
market. This ID cannot be used to cross reference a
security between markets.
‘I’ – UNITS
‘L’ – INDEX LINKED NOTES
‘M’ - MISC/LIQUID TRUST
‘O’ – ORDINARY SHARES
‘P’ - PREFERRED STOCK
‘R’ – RIGHTS
‘S’ - SHARES OF BENEFICIARY INTEREST
‘T’ – TEST
‘U’ – UNITS
‘W’ – WARRANT
Type of Security used by NYSE and NYSE MKT:
‘A’ – COMMON STOCK
‘B’ – PREFERRED STOCK
‘C’ – WARRANT
‘D’ – RIGHT
‘E’ – CORPORATE BOND
‘F’ – TREASURY BOND
‘G’ – STRUCTURED PRODUCT
‘H’ – ADR COMMON
‘I’ – ADR PREFERRED
‘J’-ADR WARRANTS
‘K’ – ADR RIGHTS
‘L’ – ADR CORPORATE BOND
‘M’ – NY REGISTERED SHARE
‘N’ – GLOBAL REGISTERED SHARE
‘O’ – INDEX
‘P’ – FUND
‘Q’ – BASKET
‘R’ – UNIT
‘S’ – LIQUIDATING TRUST
‘U’ - UNKOWN
PrevClosePrice 28 4 Binary The previous day’s closing price for this security.
PrevCloseVolume 32 4 Binary The previous day’s closing volume for the security.
MPV 38 2 Binary Note: This field is available on NYSE and NYSE MKT
only, and is left as a future enhancement on Arca.
Clients will be notified upon availability.
Unit of Trade 40 2 Binary This field specifies the security Unit of Trade in shares.
Valid values are 1, 10, 50 and 100
Note: This field is available on NYSE and NYSE MKT
only, and is left as a future enhancement on Arca.
Clients will be notified upon availability.
SourceTime 4 4 Binary The time when this msg was generated in the order book,
in seconds since Jan 1, 1970 00:00:00 UTC.
SymbolIndex 12 4 Binary The unique ID of the symbol in the Symbol Index msg
Security Status 20 1 ASCII The the new status that this security is transitioning to.
The following are Halt Status Codes:
‘3’ - Opening Delay (NYSE/MKT only)
‘4’ - Trading Halt
‘5’ - Resume
‘6’ - No open/no resume (NYSE/MKT only)
The following are Short Sale Restriction Codes:
‘A’ – Short Sale Restriction Activated (Day 1)
‘C’ – Short Sale Restriction Continued (Day 2)
‘D’ - Short Sale Restriction Deactivated
NYSE Market State values :
‘O’ – Opened
‘P’ – Pre-opening
‘X’ -- Closed
The following values are the Price Indication values:
‘T’ – T - Time
‘I’ – Price Indication
‘G’ – Pre-Opening Price Indication
‘R’ – Rule 15 Indication.
SSR Triggering 34 1 ASCII This field is only populated when securityStatus = ‘A’, and
Exchange ID otherwise is defaulted to ‘0x20’.
Valid values are:
‘N’ – NYSE
‘P’ – NYSE Arca
‘Q’ – NASDAQ
‘A’ – NYSE MKT
‘B’ – NASDAQ OMX BX
‘C’ – NSX
‘D’ – FINRA
‘I’ – ISE
‘J’ – EDGA
‘K’ – EDGX
‘M’ – CHX
‘S’ – CTS
‘T’ – NASDAQ OMX
‘V’ – IEX
‘W’ – CBSX
‘X’ – NASDAQ OMX PSX
‘Y’ – BATS Y
‘Z’ – BATS
Note: In the Arca market, this field is not yet implemented
and is defaulted to 0.
and is defaulted to 0.
SSRState 43 1 ASCII The current SSR state, which this msg updates if the
Security Status field contains an SSR Code. Valid
values:
‘~’ – No Short Sale Restriction in Effect
‘E’ – Short Sale Restriction in Effect
Note: In the Arca market, this field is not yet implemented
and is defaulted to 0.
MarketState 44 1 ASCII The current Market State, which this msg updates if the
Security Status field contains a Market State Code. Valid
values:
‘O’ – Opened
‘P’ – Pre-Opening
‘X’ – Closed
Note: In the Arca market, this field is not yet implemented
and is defaulted to 0.
SIZE
FIELD NAME OFFSET FORMAT DESCRIPTIO
(BYTES)
MsgSize 0 2 Binary Size of the message: 14 Bytes
The time when this msg was generated in the order book,
SourceTime 4 4 Binary
in seconds since Jan 1, 1970 00:00:00 UTC.
The unique ID for this NYSE feed listed in the feed’s client
ProductID 12 1 Binary
specification.
SourceTime 4 4 Binary The time when this msg was generated in the order book, in
seconds since Jan 1, 1970 00:00:00 UTC.
SymbolIndex 12 4 Binary The unique ID of the symbol in the Symbol Index msg
NextSourceSeqNum 16 4 Binary The sequence number in the next message for this symbol
SourceTime 4 4 Binary The time when this msg was generated in the order book,
in seconds since Jan 1, 1970 00:00:00 UTC.
SymbolIndex 12 4 Binary The unique ID of the symbol in the Symbol Index msg
LastSeqNum 8 4 Binary The last sequence number sent on the channel for
any symbol. The refresh is the state of the order
book as of this sequence number.
LastSymbolSeqNum 12 4 Binary The last symbol sequence number sent for this
symbol. The refresh is the symbol state of this
symbol as of this symbol sequence number.
For a depth of book feed such as XDP Integrated or XDP ArcaBook, the sequence of refresh messages per symbol
consists of the following message types:
1. Symbol Index Mapping Message (Msg Type 3)
2. Imbalance Message (Msg Type 105), if there is a current imbalance
3. Security Status Message (Msg Type 34)
4. Trade Session Change (Msg Type 33), indicates the current trading session (applies to Arca market only)
5. Add Order Refresh (Msg Type 106), repeated as needed to specify the book state for this symbol
5.1.3.1 Refresh response to a request for all Symbol Index Mapping messages
There are no Refresh Header messages
First packet Delivery Flag =18 (START of refresh)
Intermediate packets Delivery Flag = 19 (PART of refresh)
Last packet Delivery Flag = 20 (END of refresh)
5.1.3.2 Refresh response to a request for a single Symbol Index Mapping message
There is no Refresh Header message.
One packet is sent Delivery Flag = 17 (ONE packet in the refresh)
ProductID 12 1 Binary The unique ID of the feed for which the retransmission was
requested (listed in the feed’s client specification).
ChannelID 13 1 Binary The ID of the multicast channel for which the retransmission
was requested.
ChannelID 23 1 Binary The ID of the multicast channel on which the gap occurred.
SIZE
FIELD NAME OFFSET FORMAT DESCRIPTION
(BYTES)
MsgSize 0 2 Binary Size of the message: 20 Bytes
SymbolIndex 4 4 Binary The ID (from the Symbol Index msg) of the symbol for
which a refresh is requested. To request a refresh for
all symbols in the channel, set this field to 0.
SourceID 8 10 ASCII The ID of the client requesting this refresh. This field is
up to 9 characters, null terminated.
ProductID 18 1 Binary The unique ID of the feed for which the refresh is
requested (listed in the feed’s client specification).
ChannelID 19 1 Binary The ID of the multicast channel for which the refresh is
requested.
SIZE
FIELD NAME OFFSET FORMAT DESCRIPTION
(BYTES)
MsgSize 0 2 Binary Size of the message: 21 Bytes
SymbolIndex 4 4 Binary The ID (from the Symbol Index msg) of the symbol for
which a refresh is requested.
To request a refresh for all symbols in the specified
channel, set this field to 0.
SourceID 8 10 ASCII The ID of the client requesting this symbol refresh. This
field is up to 9 characters, null terminated.
ProductID 18 1 Binary The unique ID of the feed for which the refresh is
requested (listed in the feed’s client specification).
ChannelID 19 1 Binary The ID of the multicast channel for which the refresh is
requested.
RetransmitMethod 20 1 Binary The delivery method for the requested symbol index
mapping information. Valid values:
0 – deliver via UDP
RequestSeqNum 4 4 Binary The sequence number of the request message sent by the
client. This can be used by the client to couple this
response with the original request message.
SourceID 16 10 ASCII The ID of the client making the request. This field is up to 9
characters, null terminated.
ProductID 26 1 Binary The unique ID of the feed for which the request was made
(listed in the feed’s client specification).
ChannelID 27 1 Binary The ID of the multicast channel for which the request was
made.
Status 28 1 ASCII The reason why the request was rejected. Valid values:
‘0’ – Message was accepted
‘1’ – Rejected due to an Invalid Source ID
‘2’ – Rejected due to invalid sequence range
‘3’ – Rejected due to maximum sequence range (see
threshold limits)
‘4’ – Rejected due to maximum request in a day
‘5’ – Rejected due to maximum number of refresh
requests in a day
‘6’ – Rejected. Request message SeqNum TTL (Time
to live) is too old. Use refresh to recover current state if
necessary.
‘7’ – Rejected due to an Invalid Channel ID
‘8’ – Rejected due to an Invalid Product ID
‘9’ – Rejected due to: 1) Invalid MsgType, or 2)
Mismatch between MsgType and MsgSize
Prevention of invalid Incoming requests from subscribers that are not N/A Request will not
subscribers in the enabled subscriber’s Source ID list will be processed.
not be honored.
XDP subscribers will need a Source ID, which is
a string that uniquely identifies the subscriber of
the retransmission requests. Please contact the
NYSE ARCA Service Desk to get a unique
Source ID.
Limitation of Only retransmission requests for 1000 1000 Request will not
Retransmission messages or less will be honored. be processed.
Message Requests
Limitation of Generic If the generic request on a message which is 75000 Request will not
Request Age in Seq not within this threshold, the request will not be be honored.
Nums honored.
Limitation of Generic Generic requests for messages not within the 500 Subsequent
Requests threshold number of requests per day will not be retransmissions
honored during that particular day. requests from that
subscriber will be
blocked.
Limitation of requests Up to 5000 refresh requests will be honored. x Request will not
for refresh messages be honored.
Limitation of Index Up to x Symbol Index Mapping requests will be x Request will not
Mapping Requests honored. be honored.
9. Operational Information
NYSE ARCA
Pipe-delimited: ftp://ftp.nyxdata.com/ARCASymbolMapping/ARCASymbolMapping.txt
XML format: ftp://ftp.nyxdata.com/ARCASymbolMapping/ARCASymbolMapping.XML
CQS Symbol ASCII The full symbol in CTS and UTP line format. See NYSE Symbology.
SymbolIndex Numeric The unique ID for this symbol. See Symbol Indexes. This ID is unique for
products within each market. It cannot be used to cross reference a
security between markets.
NYSE Market Character The market within the NYSE Group where this symbol is traded:
‘N’ – NYSE
‘P’ – NYSE Arca
‘A’ – NYSE MKT
PriceScaleCode Numeric A code used to place the decimal point in all price fields for this symbol.
See section 3.5 for details on price handling.
SystemID Numeric The ID of the matching engine instance that handles this symbol.
Bloomberg BSID Empty Reserved field. No character between the delimiting pipe characters.
Bloomberg Global ID Empty Reserved field. No character between the delimiting pipe characters.
NOTE: The pipe delmited (.txt) version of this file has an additional pipe character at the end of the Bloomberg Global ID
field, so every record in the file has three pipe characters after the System ID field.