Temporal production readiness - Develop

This guide explains what you need to develop to use Temporal in your production environment.

Data encryption

Temporal Server stores and persists the data handled in your Workflow Execution. Encrypting this data ensures that any sensitive application data is secure when handled by the Temporal Server.

For example, if you have sensitive information passed in the following objects that are persisted in the Workflow Execution Event History, use encryption to secure it:

• Inputs and outputs/results in your WorkflowWhat is a Workflow Execution?
  
  A Temporal Workflow Execution is a durable, scalable, reliable, and reactive function execution. It is the main unit of execution of a Temporal Application.
  
  Learn more, ActivityWhat is an Activity Execution?
  
  An Activity Execution is the full chain of Activity Task Executions.
  
  Learn more, and Child WorkflowWhat is a Child Workflow Execution?
  
  A Child Workflow Execution is a Workflow Execution that is spawned from within another Workflow.
  
  Learn more

• SignalWhat is a Signal?
  
  A Signal is an asynchronous request to a Workflow Execution.
  
  Learn more inputs
• MemoWhat is a Memo?
  
  A Memo is a non-indexed user-supplied set of Workflow Execution metadata that is displayed with Filtered List results.
  
  Learn more

• Headers (verify if applicable to your SDK)

• QueryWhat is a Query?
  
  A Query is a synchronous operation that is used to report the state of a Workflow Execution.
  
  Learn more inputs and results

• Results of Local ActivitiesWhat is a Local Activity?
  
  A Local Activity is an Activity Execution that executes in the same process as the Workflow Execution that spawns it.
  
  Learn more and Side EffectsWhat is a Side Effect?
  
  A Side Effect is a way to execute a short, non-deterministic code snippet, such as generating a UUID, that executes the provided function once and records its result into the Workflow Execution Event History.
  
  Learn more

• Application errors and failures

  Failure messages and stack traces are not encoded as codec-capable Payloads by default; you must explicitly enable encoding these common attributes on failures. For more details, see Failure ConverterWhat is a Failure Converter?
  
  A Failure Converter converts error objects to proto Failures and back.
  
  Learn more.

Using encryption ensures that your sensitive data exists unencrypted only on the Client and the Worker Process that is executing the Workflows and Activities, on hosts that you control.

By default, your data is serialized to a PayloadWhat is a Payload?

A Payload represents binary data such as input and output from Activities and Workflows.

Learn more by a Data ConverterWhat is a Data Converter?

A Data Converter is a Temporal SDK component that serializes and encodes data entering and exiting a Temporal Cluster.

Learn more. To encrypt your Payload, configure your custom encryption logic with a Payload CodecWhat is a Payload Codec?

A Payload Codec transforms an array of Payloads into another array of Payloads.

Learn more and set it with a custom Data ConverterWhat is a custom Data Converter?

A custom Data Converter extends the default Data Converter with custom logic for Payload conversion or Payload encryption.

Learn more.

A Payload Codec does byte-to-byte conversion to transform your Payload (for example, by implementing compression and/or encryption and decryption) and is an optional step that happens between the wire and the Payload ConverterWhat is a Payload Converter?

A Payload Converter serializes data, converting objects or values to bytes and back.

Learn more:

User code <--> Payload Converter <--> Payload Codec <--> Wire <--> Temporal Server

You can run your Payload Codec with a Codec ServerWhat is a Codec Server?

A Codec Server is an HTTP server that uses your custom Payload Codec to encode and decode your data remotely through endpoints.

Learn more and use the Codec Server endpoints in Web UI and tctl to decode your encrypted Payload locally.

However, if you plan to set up remote data encodingWhat is remote data encoding?

Remote data encding is using your custom Data Converter to decode (and encode) your Payloads remotely through endpoints.

Learn more for your data, ensure that you consider all security implications of running encryption remotely before implementing it.

In codec implementations, we recommend running the function (such as compression or encryption) on the entire input Payload and putting the result in the data field of a new Payload with a different encoding metadata field. Using this technique ensures that the input Payload's metadata is preserved. When the encoded Payload is sent to be decoded, you can verify the metadata field before applying the decryption. If your Payload is not encoded, we recommend passing the unencoded data to the decode function instead of failing the conversion.

Examples for implementing encryption:

• Go sample
• Java sample
• Python sample
• TypeScript sample

Examples for implementing compression:

• Go sample
• Java sample

Go

Create a custom Payload Codec

Create a custom PayloadCodec implementation and define your encryption/compression and decryption/decompression logic in the Encode and Decode functions.

The Payload Codec converts bytes to bytes. It must be used in an instance of CodecDataConverter that wraps a Data Converter to do the PayloadWhat is a Payload?

A Payload represents binary data such as input and output from Activities and Workflows.

Learn more conversions, and applies the custom encoding and decoding in PayloadCodec to the converted Payloads.

The following example from the Data Converter sample shows how to create a custom NewCodecDataConverter that wraps an instance of a Data Converter with a custom PayloadCodec.

// Create an instance of Data Converter with your codec.
var DataConverter = converter.NewCodecDataConverter(
    converter.GetDefaultDataConverter(),
    NewPayloadCodec(),
)
//...
// Create an instance of PaylodCodec.
func NewPayloadCodec() converter.PayloadCodec {
    return &Codec{}
}

Implement your encryption/compression logic in the Encode function and the decryption/decompression logic in the Decode function in your custom PayloadCodec, as shown in the following example.

// Codec implements converter.PayloadEncoder for snappy compression.
type Codec struct{}

// Encode implements converter.PayloadCodec.Encode.
func (Codec) Encode(payloads []*commonpb.Payload) ([]*commonpb.Payload, error) {
    result := make([]*commonpb.Payload, len(payloads))
    for i, p := range payloads {
        // Marshal proto
        origBytes, err := p.Marshal()
        if err != nil {
            return payloads, err
        }
        // Compress
        b := snappy.Encode(nil, origBytes)
        result[i] = &commonpb.Payload{
            Metadata: map[string][]byte{converter.MetadataEncoding: []byte("binary/snappy")},
            Data:     b,
        }
    }

    return result, nil
}

// Decode implements converter.PayloadCodec.Decode.
func (Codec) Decode(payloads []*commonpb.Payload) ([]*commonpb.Payload, error) {
    result := make([]*commonpb.Payload, len(payloads))
    for i, p := range payloads {
        // Decode only if it's our encoding
        if string(p.Metadata[converter.MetadataEncoding]) != "binary/snappy" {
            result[i] = p
            continue
        }
        // Uncompress
        b, err := snappy.Decode(nil, p.Data)
        if err != nil {
            return payloads, err
        }
        // Unmarshal proto
        result[i] = &commonpb.Payload{}
        err = result[i].Unmarshal(b)
        if err != nil {
            return payloads, err
        }
    }

    return result, nil
}

For remote data encoding/decoding, see Codec ServerWhat is a Codec Server?

A Codec Server is an HTTP server that uses your custom Payload Codec to encode and decode your data remotely through endpoints.

Learn more.

Set Data Converter to use custom Payload Codec

Set your custom PayloadCodec with an instance of DataConverter in your Dial client options that you use to create the client.

The following example shows how to set your custom Data Converter from a package called mycodecpackage.

//...
c, err := client.Dial(client.Options{
        // Set DataConverter here to ensure that Workflow inputs and results are
        // encoded as required.
        DataConverter: mycodecpackage.DataConverter,
    })
//...

For reference, see the following samples:

• Codec server
• Encryption

Java

Create a custom Payload Codec

Create a custom implementation of PayloadCodec and use it in CodecDataConverter to set a custom Data Converter.

The Payload Codec does byte-to-byte conversion and must be set with a Data Converter.

Define custom encryption/compression logic in your encode method and decryption/decompression logic in your decode method.

The following example from the Java encryption sample shows how to implement encryption and decryption logic on your payloads in your encode and decode methods.

class YourCustomPayloadCodec implements PayloadCodec {

  static final ByteString METADATA_ENCODING =
      ByteString.copyFrom("binary/encrypted", StandardCharsets.UTF_8);

  private static final String CIPHER = "AES/GCM/NoPadding";

  // Define constants that you can add to your encoded Payload to create a new Payload.
  static final String METADATA_ENCRYPTION_CIPHER_KEY = "encryption-cipher";

  static final ByteString METADATA_ENCRYPTION_CIPHER =
      ByteString.copyFrom(CIPHER, StandardCharsets.UTF_8);

  static final String METADATA_ENCRYPTION_KEY_ID_KEY = "encryption-key-id";

  private static final Charset UTF_8 = StandardCharsets.UTF_8;

  // See the linked sample for details on the methods called here.
  @NotNull
  @Override
  public List<Payload> encode(@NotNull List<Payload> payloads) {
    return payloads.stream().map(this::encodePayload).collect(Collectors.toList());
  }

  @NotNull
  @Override
  public List<Payload> decode(@NotNull List<Payload> payloads) {
    return payloads.stream().map(this::decodePayload).collect(Collectors.toList());
  }

  private Payload encodePayload(Payload payload) {
    String keyId = getKeyId();
    SecretKey key = getKey(keyId);

    byte[] encryptedData;
    try {
      encryptedData = encrypt(payload.toByteArray(), key); // The encrypt method contains your custom encryption logic.
    } catch (Throwable e) {
      throw new DataConverterException(e);
    }
    // Apply metadata to the encoded Payload that you can verify in your decode method before decoding.
    // See the sample for details on the metadata values set.
    return Payload.newBuilder()
        .putMetadata(EncodingKeys.METADATA_ENCODING_KEY, METADATA_ENCODING)
        .putMetadata(METADATA_ENCRYPTION_CIPHER_KEY, METADATA_ENCRYPTION_CIPHER)
        .putMetadata(METADATA_ENCRYPTION_KEY_ID_KEY, ByteString.copyFromUtf8(keyId))
        .setData(ByteString.copyFrom(encryptedData))
        .build();
  }

  private Payload decodePayload(Payload payload) {
    // Verify the incoming encoded Payload metadata before applying decryption.
    if (METADATA_ENCODING.equals(
        payload.getMetadataOrDefault(EncodingKeys.METADATA_ENCODING_KEY, null))) {
      String keyId;
      try {
        keyId = payload.getMetadataOrThrow(METADATA_ENCRYPTION_KEY_ID_KEY).toString(UTF_8);
      } catch (Exception e) {
        throw new PayloadCodecException(e);
      }
      SecretKey key = getKey(keyId);
      byte[] plainData;
      Payload decryptedPayload;

      try {
        plainData = decrypt(payload.getData().toByteArray(), key); // The decrypt method contains your custom decryption logic.
        decryptedPayload = Payload.parseFrom(plainData);
        return decryptedPayload;
      } catch (Throwable e) {
        throw new PayloadCodecException(e);
      }
    } else {
      return payload;
    }
  }

  private String getKeyId() {
    // Currently there is no context available to vary which key is used.
    // Use a fixed key for all payloads.
    // This still supports key rotation as the key ID is recorded on payloads allowing
    // decryption to use a previous key.

    return "test-key-test-key-test-key-test!";
  }

  private SecretKey getKey(String keyId) {
    // Key must be fetched from KMS or other secure storage.
    // Hard coded here only for example purposes.
    return new SecretKeySpec(keyId.getBytes(UTF_8), "AES");
  }

  //...
}

Set Data Converter to use custom Payload Codec

Use CodecDataConverter with an instance of a Data Converter and the custom PayloadCodec in the WorkflowClient options that you use in your Worker process and to start your Workflow Executions.

For example, to set a custom PayloadCodec implementation with DefaultDataConverter, use the following code:

WorkflowServiceStubs service = WorkflowServiceStubs.newLocalServiceStubs();
  // Client that can be used to start and signal Workflows
  WorkflowClient client =
      WorkflowClient.newInstance(
          service,
          WorkflowClientOptions.newBuilder()
              .setDataConverter(
                  new CodecDataConverter(
                      DefaultDataConverter.newDefaultInstance(),
                      Collections.singletonList(new YourCustomPayloadCodec()))) // Sets the custom Payload Codec created in the previous example with an instance of the default Data Converter.
              .build());

For example implementations, see the following samples:

• Encrypted Payloads
• Remote Data Encoder and Codec Server

PHP

Content is planned but not yet available.

The information you are looking for may be found in the legacy docs.

Python

Content is planned but not yet available.

The information you are looking for may be found in the legacy docs.

TypeScript

Content is planned but not yet available.

The information you are looking for may be found in the legacy docs.