BACK-END DATA ARCHITECTURE Smart Device Data Delivery This document will present how a smart device should structure the data it delivers. It will allow testers to reference legacy architecture in order to ensure that a new device delivers its data in the same manner, and it will give developers a reference when writing their write output. ---------------------------------------------------------------------------------------------------- 1 BATCH DATA STREAMING This section covers the structure for data delivered through Batch Data Streaming settings. The basic structure is XML with both a header piece and a report piece. Batch Data gets reported in a standardized way that most of the other formats emulate. 1.1 HEADER FORMAT: 00:b0:9d:b9:bb:b5 10.5.1.94 80 443 Cam-12172213 2300 -5 1 0 12172213 1.2 REPORT FORMAT: This report contains data collected from each zone that was present on the device during the aggregation time period. In the above example you can see that data was aggregated every 5 minutes for 30 minutes, and then delivered in a batch. Each tag contains several attributes that identify the given zone. All the attributes are editable on the device except the attribute, which defines the zone index and is assigned when the zone is created: - is a Count zone. - is a Queue zone. - is a Service Point zone. - is an Edge Detection zone. - is a Height Detection zone. The last attribute in each child tag -- i.e., -- is a debugging indicator. The number indicates certain device statuses that can be used to help isolate bug clues or understand a behavioral irregularity. An example with explanation follows: - : The device was online and functioning properly during the aggregation period. - : The device was offline during the aggregation period and is generating zeroes. - : This zone was created after the aggregation period but before the data was delivered. - : This zone was created after the aggregation period but before the data was delivered. - : A problem with the device's calibration or other quality constraints caused the Tracking Engine to disable. ---------------------------------------------------------------------------------------------------- 2 REAL TIME DATA STREAMING This section covers the structure for data delivered through Real Time Data Streaming settings. The basic structure is XML with both a header piece and a report piece. There are two delivery protocols available under Real Time Data Streaming -- raw XML and HTTP -- and they have some minor differences. The XML structure is similar to the Batch Data XML structure with slight changes to the schema. 2.1 VLI HEADER FORMAT: )♠ 00:b0:9d:b9:bb:b5 10.5.1.94 80 443 Cam-12172213 2300 -5 1 0 12172213 2.2 VLI REPORT FORMAT: 2.3 HTTP HEADER FORMAT: connected, reading data POST / HTTP/1.1 Host: 10.5.1.234:9889 Content-Length: 1294 Connection: Keep-Alive 00:b0:9d:b9:bb:b5 10.5.1.94 80 443 Cam-12172213 2300 -5 1 0 12172213 2.4 HTTP REPORT FORMAT: Disconnect This report contains data collected from each zone that was present on the device during the Frequency time period set on the device. The XML structure for Real Time Data delivery is quite a bit different than the other delivery formats. The tags are replaced with tags that contain the same attributes, but the big difference is the child tags. Real Time Data is intended to be received in real time and therefore does not have a setting for scheduling delivery. As a result the child tags do not contain timing information. They only contain attributes that provide counting-type metric data, such as Enters, Exits, TotalNumberServed, etc. Each tag contains several attributes that identify the given zone. All the attributes are editable on the device except the attribute, which defines the zone index and is assigned when the zone is created: - is a Count zone. - is a Queue zone. - is a Service Point zone. - is an Edge Detection zone. - is a Height Detection zone. ---------------------------------------------------------------------------------------------------- 3 ALERT DELIVERY DATA STREAMING This section covers the structure for data delivered through Alert Delivery Data Streaming settings. The data is delivered in real time and does not have an XML structure. There are four delivery protocols and each one is a bit different from the others. 3.1 VLI FORMAT (COUNTS OFF) < connected, reading data Lab-94-Enters Disconnect connected, reading data ♀ Lab-94-Exits Disconnect connected, reading data Lab-94-Enters Disconnect > 3.2 VLI FORMAT (COUNTS ON) < connected, reading data ü Lab-94-Exits,eventType=Exit,Name=CountZone01,SiteID=Site ID,DeviceID=Device ID,DT=2014-06-27T15:05:42,Enters=112,Exits=121 Disconnect connected, reading data â Lab-94-Enters,eventType=Enter,Name=CountZone01,SiteID=Site ID,DeviceID=Device ID,DT=2014-06-27T15:05:52,Enters=113,Exits=121 Disconnect connected, reading data â Lab-94-Enters,eventType=Enter,Name=CountZone01,SiteID=Site ID,DeviceID=Device ID,DT=2014-06-27T15:06:02,Enters=114,Exits=121 Disconnect connected, reading data ü Lab-94-Exits,eventType=Exit,Name=CountZone01,SiteID=Site ID,DeviceID=Device ID,DT=2014-06-27T15:06:12,Enters=114,Exits=122 Disconnect > 3.3 HTTP POST CONTENT FORMAT < connected, reading data POST / HTTP/1.1 Host: 10.5.1.234:9889 Content-Length: 133 Connection: Keep-Alive Lab-94-Enters,eventType=Enter,Name=CountZone01,SiteID=Site ID,DeviceID=Device ID,DT=2014-06-27T15:17:24,Enters=117,Exits=124 Disconnect connected, reading data POST / HTTP/1.1 Host: 10.5.1.234:9889 Content-Length: 131 Connection: Keep-Alive Lab-94-Exits,eventType=Exit,Name=CountZone01,SiteID=Site ID,DeviceID=Device ID,DT=2014-06-27T15:18:53,Enters=117,Exits=125 Disconnect > 3.4 TCP FORMAT < connected, reading data Lab-94-Exits,eventType=Exit,Name=CountZone01,SiteID=Site ID,DeviceID=Device ID,DT=2014-06-27T15:30:38,Enters=118,Exits=128 Disconnect connected, reading data Lab-94-Enters,eventType=Enter,Name=CountZone01,SiteID=Site ID,DeviceID=Device ID,DT=2014-06-27T15:32:59,Enters=119,Exits=128 Disconnect > 3.5 HTTP FORMAT < connected, reading data GET /Lab-94-Exits?eventType=Exit&Name=CountZone01&SiteID=Site ID&DeviceID=Device ID&DT=2014-06-27T15:34:27&Enters=119&Exits=129 HTTP/1.1 Host: 10.5.1.234:9889 Content-Length: 0 Connection: Keep-Alive > Alert deliveries are only designed to work with Count zones. In all formats the data is delimited by commas except for the standard HTTP format, which is delimited by ampersand symbols (&). Because there is a setting for Deliver Counts, which essentially turns on or off the inclusion of count metrics in the delivery, it is possible for the delivery to contain no useful data, as can be seen in the VLI (COUNTS OFF) format. ---------------------------------------------------------------------------------------------------- 4 EMAIL DELIVERY This section covers the structure for data delivered through Email Delivery settings. The basic structure is XML with both a header piece and a report piece. Email Delivery is reported a bit differently from the other formats, as it has no scheduling options and is always delivered once per day with a 24-hour block of metric data. The XML structure copies the Batch Data XML structure. 4.1 HEADER FORMAT: 00:b0:9d:b9:bb:b5 10.5.1.94 80 443 Cam-12172213 2300 -5 1 0 12172213 4.2 REPORT FORMAT: This report contains data collected from each zone that was present on the device during the aggregation time period. Email Delivery always delivers daily, so the XML for each zone should always contain a 24-hour block of data. But the aggregation level can be changed on the device, and in the above example you can see that data was aggregated every 1 hour. The child tags should always match the aggregation level set on the device, i.e., separated into chunks every 5 minutes, 15 minutes, etc. Each tag contains several attributes that identify the given zone. All the attributes are editable on the device except the attribute, which defines the zone index and is assigned when the zone is created: - is a Count zone. - is a Queue zone. - is a Service Point zone. - is an Edge Detection zone. - is a Height Detection zone. The last attribute in each child tag -- i.e., -- is a debugging indicator. The number indicates certain device statuses that can be used to help isolate bug clues or understand a behavioral irregularity. An example with explanation follows: - : The device was online and functioning properly during the aggregation period. - : The device was offline during the aggregation period and is generating zeroes. - : This zone was created after the aggregation period but before the data was delivered. - : This zone was created after the aggregation period but before the data was delivered. - : A problem with the device's calibration or other quality constraints caused the Tracking Engine to disable. ---------------------------------------------------------------------------------------------------- 5 FTP DELIVERY This section covers the structure for data delivered through FTP Delivery settings. The basic structure is XML with both a header piece and a report piece. FTP Delivery can be set to format the data in a standard XML structure that copies the Batch Data XML structure, or in a specialized pipe delimited format. 5.1 XML HEADER FORMAT: 00:b0:9d:b9:bb:b5 10.5.1.94 80 443 Cam-12172213 2300 -5 1 0 12172213 5.2 XML REPORT FORMAT: This report contains data collected from each zone that was present on the device during the aggregation time period. FTP Delivery can only deliver either hourly or daily, so the XML for each zone will always contain either a 24-hour block of data or an hourly block of data. The aggregation level can be set on the device, and in the above example you can see that data was aggregated every 5 minutes and then delivered on the hour. The child tags should always match the aggregation level set on the device, i.e., separated into chunks every 5 minutes, 15 minutes, etc. Each tag contains several attributes that identify the given zone. All the attributes are editable on the device except the attribute, which defines the zone index and is assigned when the zone is created: - is a Count zone. - is a Queue zone. - is a Service Point zone. - is an Edge Detection zone. - is a Height Detection zone. The last attribute in each child tag -- i.e., -- is a debugging indicator. The number indicates certain device statuses that can be used to help isolate bug clues or understand a behavioral irregularity. An example with explanation follows: - : The device was online and functioning properly during the aggregation period. - : The device was offline during the aggregation period and is generating zeroes. - : This zone was created after the aggregation period but before the data was delivered. - : This zone was created after the aggregation period but before the data was delivered. - : A problem with the device's calibration or other quality constraints caused the Tracking Engine to disable. 5.3 PIPE DELIMITED FORMAT: < 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:05:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:10:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:15:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:20:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:25:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:30:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:35:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:40:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:45:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:50:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 10:55:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|External ID|CountZone01|2|06/19/2014 11:00:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:05:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:10:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:15:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:20:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:25:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:30:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:35:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:40:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:45:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:50:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 10:55:00|0|0|0|0|10.5.1.94| 25|Division ID|Site ID|Ext ID|CountZone02|2|06/19/2014 11:00:00|0|0|0|0|10.5.1.94| > The pipe delimited format only supports Count zones, as this is a specific feature request by Tyco. The pipe delimited report formats the data with relevant information separated by the pipe line symbol (|). The elements are as follows: - Event Type: This value is always "25". - Division ID: A string set on the device's IP Settings page. - Site ID: A string set on the device's IP Settings page. - External ID: A string set on the device's Counting page. - Zone Name: A string set on the device's Counting page. - Door Type: This value is always "2". - Timestamp: The local date and time stored on the device at the moment of aggregation. - Enters: The number of Enters counted during the aggregation period. - Exits: The number of Exits counted during the aggregation period. - Total Enters: The total number of Enters counted since the zone was created or the last device reset. - Total Exits: The total number of Exits counted since the zone was created or the last device reset. - Device IP: A local IP address set on the device's IP Settings page. ----------------------------------------------------------------------------------------------------