Monitoring a Deployment

Active Deployments can be monitored within Deeploy. Deeploy does this by intercepting logs (e.g., inferencing logs) and storing and showing key event information from the following categories:

• Deployment state and activity:
  • State:
    • Status of the deployment (e.g, ready, error) 
    • Age of the deployment
  • Activity: activity (# of calls over the last 24 hours)
• Inferencing graphs:
  • Activity: number of calls
  • Errors: number of unsuccessful calls
  • Performance: average request time
• Inferencing logs:
  • Predictions: a log of all past predictions. Predictions can include explanations
• Deployment events
  • Events (history): all events that impact deployments

Deployment state and activity

State

Every deployment has a state. The state reflects two different metrics:

• The status of the deployment itself
• The age of the deployment.

To ensure that deployments are effective and still fulfils its intended purpose, Deeploy can recommend updating the deployment. The age thresholds in the overview below can be customized in the Deployment Details tab. An overview of the default values for each state can be found in the table below. 

	Deploying	Waiting	Ready	Error	Archived
Age < 14 days	Waiting	Waiting	Healthy	Degraded	Archived
Age 14-27 days	Waiting	Waiting	Affected	Degraded	Archived
Age 28+ days	Waiting	Waiting	Degraded	Degraded	Archived

The state is represented by a coloured dot in the Deployment Overview (see figure below):

• Blue: Waiting
• Green: Healthy
• Yellow: Affected
• Red: Degraded
• Grey: Archived

Activity

Every Deployment in the Deployment Overview has an activity indicator. This yellow line graph shows the activity over the last 24 hours (see figure below).

The other Monitoring features are part of the Deployment Details. Navigate to the Deployment Details by clicking View (see figure below).

Inferencing graphs

The graphs show the following details over a predefined period:

• Activity: number of calls.
• Errors: number of unsuccessful calls
• Performance: average request time

Inferencing logs

The prediction and explanation calls that have been made with a deployment can be viewed from the Predictions tab from the Deployment Details.

• Request (Input): input tensor
• Response (Output): output tensor
• Response time: total time of request (ms)
• Timestamp: date and time of request showing the timezone of the current browser
• Explain (Currently only available for deployments that include a SHAP Kernel explainer): create a post-hoc explanation in the shape of a SHAP force plot for a selected prediction

Deployment events

The Events (History) tab from the Deployment Details shows a trace log of all events that impact the events that impact a Deployment, providing full accountability.

• Successful: indication whether the event was successful
• Event: the event type (e.g., Deployment updated)
• Data: detailed event data
• Executed by: member of the workspace that performed the event
• Timestamp: date and time of request showing the timezone of the current browser