Sighthound Analytics

Analytics data sent by the Sighthound video/image analysis pipeline. This data is sent based on configuration when the number of detected objects or attributes of detected objects changes, the confidence of detected objects or their attributes improves, or a configurable timeout occurs.

Properties

apiVersion: Refer to sub-schemas/base.json#/definitions/apiVersion.
analyticsTimestamp (integer): Timestamp the frame corresponding to this analytics data was processed, in milliseconds since the epoch and GMT timezone. Minimum: 0.
sourceId (string): A global unique ID representing the media source, for instance a specific video stream from a camera sensor or RTSP feed.
frameId (string): An ID corresponding to this frame, which may be used to access the image corresponding to all box coordinates and object detections represented in this object, via the Media Service API.
frameDimensions (object): The dimensions (width and height) of the frame represented by frameId. Also used as the coordinate base for all bounding box coordinates.
- w (number): Width in pixels. Minimum: 0.
- h (number): Height in pixels. Minimum: 0.
metaClasses: Refer to sub-schemas/objectAnalytics.json#/definitions/metaClasses.
sensorEvents: Refer to sub-schemas/eventAnalytics.json#/definitions/sensorEvents.

Definitions
updated (boolean): This field is omitted when using image input, and can be assumed true. This field is true when this element was updated with the associated JSON packet in one of the following ways:
- Classification of the object has changed
- Classification score of the object has improved.

ALPR Example

A simple example mapping license plate to vehicle

example

{
  "apiVersion": {
    "major": 1,
    "minor": 1
  },
  "frameDimensions": {
    "w": 1024,
    "h": 768
  },
  "sourceId": "53658928-3cd3-4f24-b023-cb54228689ba",
  "frameId": "53658928-3cd3-4f24-b023-cb54228689ba-frame-1",
  "analyticsTimestamp": 1643922414525,
  "metaClasses": {
    "licensePlates": {
      "lpr-1-lp-1-1643922411122": {
        "attributes": {
          "lpString": {
            "value": "GYS2421",
            "attributeScore": 1,
            "detectionScore": 0.79,
            "updated": true
          },
          "lpRegion": {
            "value": "Pennsylvania",
            "attributeScore": 0.94,
            "detectionScore": 0.79,
            "updated": true
          }
        },
        "firstFrameTimestamp": 1643922414525,
        "class": "licenseplate",
        "bestDetectionTimestamp": 1643922414525,
        "box": {
          "height": 63,
          "width": 100,
          "x": 748,
          "y": 347
        },
        "detectionScore": 0.79,
        "updated": false,
        "links": [
          {
            "metaClass": "vehicles",
            "id": "lpr-1-car-1-1643922411137"
          }
        ]
      }
    },
    "vehicles": {
      "lpr-1-car-1-1643922411137": {
        "attributes": {
          "vehicleType": {
            "value": "audi a8 2002 2010",
            "attributeScore": 0.38,
            "detectionScore": 0.87,
            "updated": true
          },
          "color": {
            "value": "silver/grey",
            "attributeScore": 1,
            "detectionScore": 0.87,
            "updated": true
          }
        },
        "firstFrameTimestamp": 1643922414525,
        "class": "car",
        "bestDetectionTimestamp": 1643922414525,
        "box": {
          "height": 487,
          "width": 925,
          "x": 48,
          "y": 158
        },
        "detectionScore": 0.87,
        "updated": false,
        "links": [
          {
            "metaClass": "licensePlates",
            "id": "lpr-1-lp-1-1643922411122"
          }
        ]
      }
    }
  }
}

JSON Schema

Definitions

apiVersion (object)

Sighthound Event Analytics

Event data sent by the event processing microservice or pipeline. Based on pipeline configuration, may be sent with every frame, a subset of frames, or only frames when changes in presence state are detected.

Properties

sensorEvents: Refer to #/definitions/sensorEvents.

Definitions
objectsInRegionCount (integer): The total number of detected objects in a region, filtered by object types associated with a sensor. Minimum: 0.
direction (string): Visualizing the sensor line as an arm on a clock, with the first point as the pivot point, the object's trajectory direction (clockwise or counterclockwise). Minimum: 0.
startedAt (integer): The time in milliseconds since the epoch (GMT) when the event started, or when a link was established. Minimum: 0.
endedAt (integer): The time in milliseconds since the epoch (GMT) when the event ended. Minimum: 0.
sensorId (string): The globally unique sensor ID corresponding to this sensor. This ID is unique for a given sensor and stream combination, and tracks all historical events relative to this sensor. It also tracks configuration of the sensor, including the definition of lines or polygons and class filters defining the sensor.
updateCount (integer): The cumulative number of updates sent for this sensor, starting with 1 for the initial update and incremented once for each update sent for each unique sensor event ID. An update refers to a change in the state of the sensor due to a corresponding sensor event (entry, exit, crossing, ...). Minimum: 0.
links (array): Links between objects (see objectAnalytics schema) and events.
- Items (object)
presenceEvent (object): Describes an event where one or more objects are present in a region of interest. The event starts when the first object enters a region of interest. Updates are sent for each change in status, with updateCount incremented for each update. When the last object exits and the region is empty, the sensor event will become immutable and will track the total amount of time at least one object was present in the region of interest. Region definitons, object filtering and other items related to sensor definitions are tracked as a part of the sensorId associated with the event. Cannot contain additional properties.
countEvent (object): Describes an event where one object crosses a line. Cannot contain additional properties.
sensorEvents (object): A map of maps describing an event type.
- The top level map key is a name describing the event type. Supported types are presenceEvent, lineCrossingEvent, speedEvent.
- The sub level map key is a Unique ID representing the event, used to map additional object properties. This ID is guaranteed unique for each event for a given stream ID.

Sighthound Object Detection Analytics

Object detection related ananlytics data

Properties

metaClasses: Refer to #/definitions/metaClasses.

Definitions
detectionScore (number): The confidence score of object detection, in floating
point, ranging from 0.0 to 1.0 inclusive, where 1.0 is 100% confidence. When included in an attribute, this score represents the object Detection score for the parent object corresponding to the timestamp when the attribute value was determined. Minimum: 0. Maximum: 1.
attributeScore (number): The confidence score of attribute detection, in floating point, ranging from 0.0 to 1.0 inclusive, where 1.0 is 100% confidence. Minimum: 0. Maximum: 1.
updated (boolean)
metaClasses (object)

Presence Sensor Examples

An example sequence corresponding to presence region events.

Entry

A single vehicle entering a presence region

example

{
  "apiVersion": {
    "major": 1,
    "minor": 1
  },
  "sourceId": "53658928-3cd3-4f24-b023-cb54228689ba",
  "frameId": "53658928-3cd3-4f24-b023-cb54228689ba-frame-1",
  "frameDimensions": {"w": 1024, "h": 768},
  "analyticsTimestamp": 1631233271683,
  "metaClasses": {
    "vehicles": {
      "06138c16-ec45-45a9-bb57-31a08941e413": {
        "detectionScore": 0.9999,
        "bestDetectionTimestamp": 1628896071683,
        "firstFrameTimestamp": 1628896069676,
        "class" : "car",
        "box": {
          "height": 462,
          "width": 858,
          "x": 284,
          "y": 256
        },
        "updated": true,
        "attributes": {
          "vehicleType": {
            "detectionScore": 0.978,
            "attributeScore": 0.978,
            "updated": true,
            "value": "audi a8 2008"
          },
          "color": {
            "detectionScore": 0.9999,
            "attributeScore": 0.901,
            "updated": true,
            "value": "beige/cream"
          }
        }
      }
    },
    "people": {
      "65891069-0030-4355-8ab3-a493a6de122e": {
        "detectionScore": 0.9875,
        "bestDetectionTimestamp": 1631233287923,
        "firstFrameTimestamp": 1631233274642,
        "class" : "person",
        "box": {
          "height": 22,
          "width": 85,
          "x": 226,
          "y": 1824
        },
        "updated": false
      }
    }
  },
  "sensorEvents": {
    "presenceEvent": {
      "12345678-0030-4355-8ab3-a493a6de122e": {
        "sensorId": "87654321-0030-4355-8ab3-a493a6de122e",
        "objectsInRegionCount": 1,
        "startedAt": 1631233274642,
        "updateCount" : 1,
        "links": [
          {
            "metaClass": "vehicles",
            "id": "06138c16-ec45-45a9-bb57-31a08941e413",
            "startedAt": 1631233274642
          }
        ]
      }
    }
  }
}

Entry Person Enter

A person entering in addition to the vehicle

example

{
  "apiVersion": {
    "major": 1,
    "minor": 0
  },
  "sourceId": "53658928-3cd3-4f24-b023-cb54228689ba",
  "frameId": "53658928-3cd3-4f24-b023-cb54228689ba-frame-200",
  "analyticsTimestamp": 1631233275082,
  "frameDimensions": {"w": 1024, "h": 768},
  "metaClasses": {
    "vehicles": {
      "06138c16-ec45-45a9-bb57-31a08941e413": {
        "detectionScore": 0.9999,
        "bestDetectionTimestamp": 1628896071683,
        "firstFrameTimestamp": 1628896069676,
        "class" : "car",
        "box": {
          "height": 462,
          "width": 858,
          "x": 284,
          "y": 256
        },
        "updated": true,
        "attributes": {
          "vehicleType": {
            "detectionScore": 0.978,
            "attributeScore": 0.978,
            "updated": true,
            "value": "bus"
          },
          "color": {
            "detectionScore": 0.9999,
            "attributeScore": 0.901,
            "updated": true,
            "value": "beige/cream"
          }
        }
      }
    },
    "people": {
      "65891069-0030-4355-8ab3-a493a6de122e": {
        "detectionScore": 0.9875,
        "bestDetectionTimestamp": 1631233287923,
        "firstFrameTimestamp": 1631233274642,
        "class" : "person",
        "box": {
          "height": 22,
          "width": 85,
          "x": 226,
          "y": 245
        },
        "updated": false
      }
    }
  },
  "sensorEvents": {
    "presenceEvent": {
      "12345678-0030-4355-8ab3-a493a6de122e": {
        "sensorId": "87654321-0030-4355-8ab3-a493a6de122e",
        "objectsInRegionCount": 2,
        "startedAt": 1631233274642,
        "updateCount" : 2,
        "links": [
          {
            "metaClass": "vehicles",
            "id": "06138c16-ec45-45a9-bb57-31a08941e413",
            "startedAt": 1631233274642
          },
          {
            "metaClass": "people",
            "id": "65891069-0030-4355-8ab3-a493a6de122e",
            "startedAt": 1631233275082
          }
        ]
      }
    }
  }
}

Entry Person Leave

The person leaving the presence region with vehicle still present

example

{
  "apiVersion": {
    "major": 1,
    "minor": 0
  },
  "sourceId": "53658928-3cd3-4f24-b023-cb54228689ba",
  "frameId": "53658928-3cd3-4f24-b023-cb54228689ba-frame-200",
  "analyticsTimestamp": 1631233277000,
  "frameDimensions": {"w": 1024, "h": 768},
  "metaClasses": {
    "vehicles": {
      "06138c16-ec45-45a9-bb57-31a08941e413": {
        "detectionScore": 0.9999,
        "bestDetectionTimestamp": 1628896071683,
        "firstFrameTimestamp": 1628896069676,
        "class": "car",
        "box": {
          "height": 462,
          "width": 858,
          "x": 284,
          "y": 256
        },
        "updated": true,
        "attributes": {
          "vehicleType": {
            "detectionScore": 0.978,
            "attributeScore": 0.978,
            "updated": true,
            "value": "bus"
          },
          "color": {
            "detectionScore": 0.9999,
            "attributeScore": 0.901,
            "updated": true,
            "value": "beige/cream"
          }
        }
      }
    },
    "people": {
      "65891069-0030-4355-8ab3-a493a6de122e": {
        "detectionScore": 0.9875,
        "bestDetectionTimestamp": 1631233287923,
        "firstFrameTimestamp": 1631233274642,
        "class": "person",
        "box": {
          "height": 22,
          "width": 85,
          "x": 226,
          "y": 245
        },
        "updated": false
      }
    }
  },
  "sensorEvents": {
    "presenceEvent": {
      "12345678-0030-4355-8ab3-a493a6de122e": {
        "sensorId": "87654321-0030-4355-8ab3-a493a6de122e",
        "objectsInRegionCount": 1,
        "updateCount" : 3,
        "startedAt": 1631233274642,
        "links": [
          {
            "metaClass": "vehicles",
            "id": "06138c16-ec45-45a9-bb57-31a08941e413",
            "startedAt": 1631233274642
          }
        ]
      }
    }
  }
}

Entry Person Reenter

The person re-entering the presence region with vehicle still present

example

{
  "apiVersion": {
    "major": 1,
    "minor": 0
  },
  "sourceId": "53658928-3cd3-4f24-b023-cb54228689ba",
  "frameId": "53658928-3cd3-4f24-b023-cb54228689ba-frame-200",
  "analyticsTimestamp": 1631233277020,
  "frameDimensions": {"w": 1024, "h": 768},
  "metaClasses": {
    "vehicles": {
      "06138c16-ec45-45a9-bb57-31a08941e413": {
        "detectionScore": 0.9999,
        "bestDetectionTimestamp": 1628896071683,
        "firstFrameTimestamp": 1628896069676,
        "class": "car",
        "box": {
          "height": 462,
          "width": 858,
          "x": 284,
          "y": 256
        },
        "updated": true,
        "attributes": {
          "vehicleType": {
            "detectionScore": 0.978,
            "attributeScore": 0.978,
            "updated": true,
            "value": "bus"
          },
          "color": {
            "detectionScore": 0.9999,
            "attributeScore": 0.901,
            "updated": true,
            "value": "beige/cream"
          }
        }
      }
    },
    "people": {
      "65891069-0030-4355-8ab3-a493a6de122e": {
        "detectionScore": 0.9875,
        "bestDetectionTimestamp": 1631233287923,
        "firstFrameTimestamp": 1631233274642,
        "class": "person",
        "box": {
          "height": 22,
          "width": 85,
          "x": 226,
          "y": 245
        },
        "updated": false
      }
    }
  },
  "sensorEvents": {
    "presenceEvent": {
      "12345678-0030-4355-8ab3-a493a6de122e": {
        "sensorId": "87654321-0030-4355-8ab3-a493a6de122e",
        "objectsInRegionCount": 2,
        "updateCount" : 4,
        "startedAt": 1631233274642,
        "links": [
          {
            "metaClass": "vehicles",
            "id": "06138c16-ec45-45a9-bb57-31a08941e413",
            "startedAt": 1631233274642
          },
          {
            "metaClass": "people",
            "id": "65891069-0030-4355-8ab3-a493a6de122e",
            "startedAt": 1631233277020
          }
        ]
      }
    }
  }
}

Entry Complete

Both person and vehicle leaving the presence region and the event completing

example

{
  "apiVersion": {
    "major": 1,
    "minor": 0
  },
  "sourceId": "53658928-3cd3-4f24-b023-cb54228689ba",
  "frameId": "53658928-3cd3-4f24-b023-cb54228689ba-frame-999",
  "frameDimensions": {"w": 1024, "h": 768},
  "analyticsTimestamp": 1631233279123,
  "metaClasses": {
    "vehicles": {
      "06138c16-ec45-45a9-bb57-31a08941e413": {
        "detectionScore": 0.9999,
        "bestDetectionTimestamp": 1628896071683,
        "firstFrameTimestamp": 1628896069676,
        "class" : "car",
        "box": {
          "height": 462,
          "width": 858,
          "x": 284,
          "y": 256
        },
        "updated": true,
        "attributes": {
          "vehicleType": {
            "detectionScore": 0.978,
            "attributeScore": 0.978,
            "updated": true,
            "value": "bus"
          },
          "color": {
            "detectionScore": 0.9999,
            "attributeScore": 0.901,
            "updated": true,
            "value": "beige/cream"
          }
        }
      }
    },
    "people": {
      "65891069-0030-4355-8ab3-a493a6de122e": {
        "detectionScore": 0.9875,
        "bestDetectionTimestamp": 1631233287923,
        "firstFrameTimestamp": 1631233274642,
        "class" : "person",
        "box": {
          "height": 22,
          "width": 85,
          "x": 1024,
          "y": 856
        },
        "updated": false
      }
    }
  },
  "sensorEvents": {
    "presenceEvent": {
      "12345678-0030-4355-8ab3-a493a6de122e": {
        "sensorId": "87654321-0030-4355-8ab3-a493a6de122e",
        "objectsInRegionCount": 0,
        "updateCount" : 5,
        "startedAt": 1631233274642,
        "endedAt": 1631233279123,
        "links": []
      }
    }
  }
}