You've learned about Temporal, checked out our samples, written a few Workflows, and now you're ready to productionize. In this article, we outline some techniques you can employ to ensure that your Workflows are ready for the future.
All the ways to run a Temporal Cluster
There are many ways to run a Temporal Cluster on your own. However, the right way for you depends entirely on your use case and where you plan to run it. This article aims to maintain a comprehensive list of all the ways we know of.
Temporal Platform limits sheet
Running into limits can cause unexpected failures. Knowing Temporal's limits can prevent that.
This page details many of the hard and soft limits that are coded into the Temporal Platform. Hard limits fail with an error; soft limits produce a warning log on the server side.
Python sandbox environment
The Temporal Python SDK enables you to run Workflow code in a sandbox environment to help prevent non-determinism errors in your application. The Temporal Workflow Sandbox for Python is not completely isolated, and some libraries can internally mutate state, which can result in breaking determinism.
How to explain Temporal
There are many ways to introduce and teach Temporal based on your background. Temporal doesn't have a monopoly on explaining Temporal.
Set up Grafana with Temporal Cloud observability to view metrics
Temporal Cloud and SDKs emit metrics that can be used to monitor performance and troubleshoot errors.
Temporal Cloud emits metrics through a Prometheus HTTP API endpoint which can be directly used as a Prometheus data source in Grafana or to query and export Cloud metrics to any observability platform.
Temporal Failures
Background: What is a Failure?
A Failure is Temporal's representation of various types of errors that occur in the system.
There are different types of Failures, and each has a different type in the SDKs and different information in the protobuf messages (which are used to communicate with the Temporal Cluster and appear in Event History).
Troubleshoot deadline-exceeded error
All requests made to the Temporal Cluster by the Client or Worker are gRPC requests.
Sometimes, when these frontend requests can't be completed, you'll see this particular error message: Context: deadline exceeded
.
Network interruptions, timeouts, server overload, and Query errors are some of the causes of this error.
The following sections discuss the nature of this error and how to troubleshoot it.
Set up Prometheus and Grafana to view metrics
The Temporal Cluster and SDKs emit metrics that can be used to monitor performance and troubleshoot issues. To collect and aggregate these metrics, you can use one of the following tools:
- Prometheus
- StatsD
- M3
After you enable your monitoring tool, you can relay these metrics to any monitoring and observability platform.
Non-determinism issues for Run Ids
The current Run Id is mutable and can change during a Workflow Retry. You should not rely on storing the current Run Id, or using it for any logical choices, because a Workflow Retry changes the Run Id and can lead to non-determinism issues.