Lex - Data Privacy Controls

Operata collector filters fields from JSON event payloads before further processing. Data Privacy Controls allow customers to remove sensitive data using a configurable Tier lists while a hardcoded mandatory list ensures fields required for system operation are never removed.

❗️
Note: This is for Amazon Connect Lex integration only.

How Data Privacy Controls work

Operata has a six-tier data redaction framework applied to the data Operata ingests - each JSON event goes through a deterministic, ordered filter pipeline.

The framework assigns every field in the event payload to exactly one of six tiers:

A Mandatory tier is defined and maintained by Operata; it supersedes all others.
Four customer-configurable tiers - Allow, Deny, Encrypt, and Transform these are mutually exclusive: a field may appear in only one.
A Default tier deals with any fields that are not part of the other six tiers.

Tier	What it does	Controlled by
Mandatory	Key and value transmitted unchanged. Cannot be denied, hashed, encrypted, or redacted.	Operata
Allow	Key and value transmitted unchanged. Customer declares which non-mandatory fields to receive.	Customer
Deny	Key and value removed entirely. Operata has no knowledge the field exists.	Customer
Encrypt	Key retained. Value replaced with a hash or ciphertext. Original value never transmitted.	Customer
Transform	Key retained. Value is transformed with a derived/calculated info.	Customer
Default	Key retained. All values are nullified.	Custom

Processing Architecture

When the collector processes an event, it applies filtering in four passes:

Pass 1 - Mandatory Override Pass

Check whether the marked field, or any of its descendants, appears in the mandatory list. If a mandatory field sits under a denied parent, retain the parent and prune all non-mandatory siblings. The mandatory list always takes precedence.

Pass 2 - Deny Pass

Evaluate every field path against the deny list. Mark fields for removal.*

Pass 3 - Any remaining items

The remaining filters (allow, encrypt, transform) execute

Pass 4 - Default Nullify Pass

All remaining field values in the payload outside of the lists will be have the key retained, but the values nullified.

❗️
Currently only Mandatory tier filtering will be possible - listed below

Path Notation

Both lists use dot-notation paths to identify fields.

A top-level key such as Parameters matches the entire object at that key.
A nested path such as Parameters.LexVersion matches only that specific subfield.

The processor applies rules at every depth level consistently, so deny-with-mandatory-preservation logic works the same at any nesting depth.

Parent Key Behaviour

When a deny list entry targets a parent key (for example, Parameters) and no mandatory list entry references any child of that parent, the processor removes the entire Parameters key. The result is not an empty object; the key is absent from the output entirely.

When a deny list entry targets a parent key and the mandatory/allow/encrypt/transform list references one or more children of that parent, the processor retains the parent containing only those mandatory children. Every other child is removed regardless of whether it also appears explicitly in the deny list.

Mandatory List for LEX v2

The following fields are hardcoded and the processor always preserves them regardless of deny/encrypt/transform list configuration.

errorLogSchema
errorLogSchemaType

timestamp
requestId
messageVersion
transcriptions[*].resolvedContext.intent
transcriptions[*].transcriptionConfidence
transcriptions[*].resolvedSlots.*.shape
fulfilledByAssistedNlu
requestAttributes.x-amz-lex:accept-content-types
requestAttributes.x-amz-lex:channels:platform
sessionId
inputMode
bargeIn
operationName
isTestWorkbenchTraffic
interpretations[*].intent.name
interpretations[*].intent.state
interpretations[*].intent.slots.*.shape
interpretations[*].intent.confirmationState
interpretations[*].interpretationSource
interpretations[*].nluConfidence
developerOverride
bot
sessionState.sessionAttributes.x-amz-lex:allow-interrupt:*:*
sessionState.sessionAttributes.x-amz-lex:audio:end-timeout-ms:*:*
sessionState.sessionAttributes.x-amz-lex:audio:start-timeout-ms:*:*
sessionState.sessionAttributes.x-amz-lex:barge-in-enabled:*:*
sessionState.sessionAttributes.x-amz-lex:connect-originating-request-id
sessionState.sessionAttributes.x-amz-lex:dtmf:deletion-character:*:*
sessionState.sessionAttributes.x-amz-lex:dtmf:end-timeout-ms:*:*
sessionState.sessionAttributes.bedrock_request_id
sessionState.sessionAttributes.confirm_bedrock_request_id
sessionState.sessionAttributes.confirm_llm
sessionState.sessionAttributes.confirm_llm_input_tokens
sessionState.sessionAttributes.confirm_llm_latency
sessionState.sessionAttributes.confirm_llm_model_id
sessionState.sessionAttributes.confirm_llm_output_tokens
sessionState.sessionAttributes.llm
sessionState.sessionAttributes.llm_input_tokens
sessionState.sessionAttributes.llm_latency
sessionState.sessionAttributes.llm_model_id
sessionState.sessionAttributes.llm_output_tokens
sessionState.dialogAction
sessionState.dialogAction.slotToElicit
sessionState.dialogAction.type
sessionState.intent.name
sessionState.intent.state
sessionState.intent.slots.*.shape
sessionState.intent.confirmationState
sessionState.originatingRequestId
missedUtterance
responseReason
utteranceContext