Events and Metrics data

Storage of discrete events and metrics (schema)

The time-series data is stored in a Cosmos DB, with the schema as follows:

Events 

{
    // Partition Key: string
    "name": "<eventName>",
    // Row Key (ID): string
    // ISO 8601 format
    "id": "<enqueuedTimestampUtc>",
    "dimensions": {
        "<key1>": "<value1>",
        "<key2>": "<value2>",
        "<keyN>": "<valueN>"
    }
}

Metrics 

{
    // Partition Key: string
    "name": "<metricName>",
    // Row Key (ID): string
    // ISO 8601 format
    "id": "<enqueuedTimestampUtc>",
    // number
    "value": "<metricValue>",
    "dimensions": {
        "<key1>": "<value1>",
        "<key2>": "<value2>",
        "<keyN>": "<valueN>"
    }
}

All events will be in one container events, but in different logical partitions (with the name field (i.e. the event name) as the partition key, and the id being the UTC timestamp). This will facilitate fast queries, as well as creation of materialized views while avoiding cross-container joins.

The same applies for metrics too, with metrics as the container name.

Don’t worry, we’ll be well under the storage quota limits.

Aggregated 

{
    "metric": "<MetricName>", /* Partition Key*/
    "id": /* @TODO */
    "min": <minValue>, /* The minimum metric value (: number) in the specified time range */
    "max": <maxValue>, /* The maximum metric value (: number) in the specified time range */
    "sum": <sumOfAllValues>, /* The sum (: number) of all metric values in the specified time range */
    "avg": <avgOfAllValues>, /* The average (: number) of all metric values in the specified time range */
    "count": <countOfAllValues>, /* The total number (: number) of metric samples in the specified time range */
    "startDateTimeUtc": "<timestampUtc>", /* ISO 8601 format */
    "endDateTimeUtc": "<timestampUtc>", /* ISO 8601 format */
    "dimensions": {
        "<Key>": "<Value>",
        :
        :
    }
}

All aggregation will be in the aggregatedMetrics container.

  • A CosmosDB feed-processor will compute aggregations from discrete time-series data in the background.
  • Pre-aggregated data can be added via explicit API calls too (e.g. The Edge layer might choose to do this).

Ingestion of events and metrics

All microservices can inject their events and metrics into the Metrics microservice in one of two ways:

  1. REST API: See swagger page.

    When ingested, the event or metric will be queued for persistence into storage (CosmosDB).

    For events and metrics, the DateTimeUtc is optional and will default to the current UTC time if not specified.

    For metrics, the Unit field is optional. But it is recommended that you specify the unit of the metric (e.g. seconds, bytes, items, etc.) to facilitate querying and visualization later on.

    The Dimensions field is also optional and can be used to store additional metadata about the metric. For example, if you’re ingesting a metric for the number of orders processed, you can store the OrderId in the Dimensions field to be able to query the metric by OrderId later on. We’ll discuss this in detail in the next section on materialized views.

  2. Service bus queue: This is a good option if you’re generating a burst of metrics in a short duration and need to stream them in. The JSON to be queued has the exact same schema as used in the REST API’s request body.

Querying events and metrics

Use the REST API (swagger page) exposed by the Metrics microservice for extraction of the aggregated time-series data.

In the initial incarnation, we’ll not support server-side pagination, sorting/ordering, or filtering by dimensions. We’ll add these features as the need arises.

Reports

Materialized views will be used to generate reports (e.g. per-fulfillment stats). They’ll essentially join the various metrics on a dimension (e.g. OrderId or FulfillmentId).

Here is an example of a materialized view that gives us the per-fulfillment stats by joining all ingested events and metrics on the FulfillmentId dimension:

{
    "fulfillmentId": "",
    // aggregation of events (see full list of events below)
    "eventAggregations": [
        {
            "name": "orderFulfilledEvent",
            "count": "8",
            "dimensions": {
                "orderId": [
                    "O1234343434",
                    "O1234343435",
                    "O1234343436",
                    "O1234343437",
                    "O1234343438",
                    "O1234343439",
                    "O1234343440",
                    "O1234343441"
                ],
                "waveId": [
                    "wave0",
                    "wave1",
                    "wave2"
                ],
                "FulfillmentId": [
                    "F1234343434"
                ]
            }
        },
        {
            "name": "PickCompletedEvent",
            "count": "11",
            "dimensions": {
                "orderId": [
                    "O1234343434",
                    "O1234343435",
                    "O1234343436",
                    "O1234343437",
                    "O1234343438",
                    "O1234343439",
                    "O1234343440",
                    "O1234343441",
                    "O1234343442",
                    "O1234343443",
                    "O1234343444"
                ],
                "waveId": [
                    "wave0",
                    "wave1",
                    "wave2"
                ],
                "FulfillmentId": [
                    "F1234343434"
                ]
            }
        }
    ]
    // aggregation of metrics (see full list of metrics below)
    "metricAggregations": [
        {
            "name": "InternalSkusPicked",
            "min": "0",
            "max": "100",
            "sum": "1000",
            "avg": "50",
            "count": "20",
            "dimensions": {
                "orderId": [
                    "O1234343434",
                    "O1234343435",
                    "O1234343436",
                    "O1234343437",
                    "O1234343438",
                    "O1234343439",
                    "O1234343440",
                    "O1234343441",
                    "O1234343442",
                    "O1234343443",
                    "O1234343444"
                ],
                "waveId": [
                    "wave0",
                    "wave1",
                    "wave2"
                ],
                "FulfillmentId": [
                    "F1234343434"
                ]
            }
        },
        {
            "name": "ExternalSkusPicked",
            "min": "0",
            "max": "100",
            "sum": "1000",
            "avg": "50",
            "count": "20",
            "dimensions": {
                "orderId": [
                    "O1234343434",
                    "O1234343435",
                    "O1234343436",
                    "O1234343437",
                    "O1234343438",
                    "O1234343439",
                    "O1234343440",
                    "O1234343441",
                    "O1234343442",
                    "O1234343443",
                    "O1234343444"
                ],
                "waveId": [
                    "wave0",
                    "wave1",
                    "wave2"
                ],
                "FulfillmentId": [
                    "F1234343434"
                ]
            }
        }
    ]
}

List of Events

Events Publisher Notes
BinDecommissionedEvent CartePlus.Api  
BinPurgedEvent CartePlus.Api  
BinReleasedEvent CartePlus.Api  
DecommissionBinStartedEvent CartePlus.Api  
DefragCompletedEvent CartePlus.Api  
DefragStartedEvent CartePlus.Api  
FulfillmentCompletedEvent CartePlus.Api  
FulfillmentStartedEvent CartePlus.Api  
InwardCompletedEvent CartePlus.Api  
InwardStartedEvent CartePlus.Api  
ItemCreatedEvent CartePlus.Api  
ItemPurgedEvent CartePlus.Api  
OrderCreatedEvent CartePlus.Api  
OrderFulfilledEvent CartePlus.Api Not raised for “purely external” orders
PauseCompletedEvent CartePlus.Orchestrator  
PauseStartedEvent CartePlus.Orchestrator  
PickCompletedEvent CartePlus.Api  
PlcMessageAbortedEvent CartePlus.Api  
PlcMessageAcknowledgedEvent CartePlus.Api  
PlcMessageDelayedEvent CartePlus.Api  
PutCompletedEvent CartePlus.Api  
ReInductCompletedEvent CartePlus.Api  
ReInductStartedEvent CartePlus.Api  
ReleaseBinStartedEvent CartePlus.Api  
ReplenishCompletedEvent CartePlus.Api  
ReplenishStartedEvent CartePlus.Api  
StartPickEvent CartePlus.Api  
StartPutEvent CartePlus.Api  
WaveCompletedEvent CartePlus.Orchestrator  
WaveStartedEvent CartePlus.Orchestrator  

List of Metrics

Metrics Unit Publisher Notes
InternalSkusPicked N/A CartePlus.Orchestrator  
ExternalSkusPicked N/A CartePlus.Orchestrator  
TotalTimeForTopUps seconds CartePlus.Orchestrator  
TotalTimeForInbounds seconds CartePlus.Orchestrator  
TotalWaveChangeTime seconds CartePlus.Orchestrator  
TotalWaveTime seconds CartePlus.Orchestrator  
TotalFulfillmentTime seconds CartePlus.Orchestrator  

The PLC metrics are tracked separately in the next section.

List of PLC Metrics

PLC Metrics Unit Original PLC Tag Notes
  mm EMxx_Lift_Indexer_HMI.Lift_axis_1_Actual_Position  
  mm EMxx_Lift_Indexer_HMI.Lift_axis_2_Actual_Position  
  mm EMxx_Lift_Indexer_HMI.Indexer_Actual_Position  
  mm/s EMxx_Lift_Indexer_HMI.Lift_axis_1_Actual_Velocity  
  mm/s EMxx_Lift_Indexer_HMI.Lift_axis_2_Actual_Velocity  
  mm/s EMxx_Lift_Indexer_HMI.Indexer_Actual_Velocity  
  Amps EMxx_Bot_HMI.X_axis_actual_current  
  Amps EMxx_Bot_HMI.Y_axis_1_actual_current  
  Amps EMxx_Bot_HMI.Y_axis_2_actual_current  
  Amps EMxx_Bot_HMI.Z_axis_actual_current  
  °Celsius EMxx_Bot_HMI.X_axis_temp  
  °Celsius EMxx_Bot_HMI.Y_axis_1_temp  
  °Celsius EMxx_Bot_HMI.Y_axis_2_temp  
  °Celsius EMxx_Bot_HMI.Z_axis_temp  
  N-m EMxx_Bot_HMI.X_axis_actual_torque  
  N-m EMxx_Bot_HMI.Y_axis_1_actual_torque  
  N-m EMxx_Bot_HMI.Y_axis_2_actual_torque  
  N-m EMxx_Bot_HMI.Z_axis_actual_torque  
  mm EMxx_Bot_HMI.X_axis_actual_pos  
  mm EMxx_Bot_HMI.Y_axis_1_actual_pos  
  mm EMxx_Bot_HMI.Y_axis_2_actual_pos  
  mm EMxx_Bot_HMI.Z_axis_actual_pos  
  mm/s EMxx_Bot_HMI.X_axis_actual_velocity  
  mm/s EMxx_Bot_HMI.Y_axis_1_actual_velocity  
  mm/s EMxx_Bot_HMI.Y_axis_2_actual_velocity  
  mm/s EMxx_Bot_HMI.Z_axis_actual_velocity  
  mm EMxx_Bot_HMI.Y_axis_Single_deep_left_pos  
  mm EMxx_Bot_HMI.Y_axis_Double_deep_left_pos  
  mm EMxx_Bot_HMI.Y_axis_Single_deep_right_pos  
  mm EMxx_Bot_HMI.Y_axis_Double_deep_right_pos  
  N/A HMI_MoverID_Bot[xx]  
  N/A Program:UNxx_Bot_Axis.L_Main_Sequence  

Note: xx is variable, depends on Equipment module assignment number e.g. EMxx = EM01.