Kinesis Streams vs Kinesis Firehose

Why Kinesis Data Streams and not Kinesis Firehose?

Amazon Kinesis Data Firehose is an extract, transform, and load (ETL) service that reliably captures, transforms and delivers streaming data to data lakes, data stores, and analytics services.

It is commonly used by ingesting data from a Kinesis Data Stream, transforming it, then loading it into pre-configured destinations such as S3, Redshift, Splunk etc. There are many low/no-code use cases, such as taking Amazon Connect Contact Trace Records (CTR's) and sending them to S3.

Whilst this is a great use case, it can create some architectural complexities when multiple applications want to consume the same data - i.e. Operata consuming the CTR alongside a BI Reporting tool.

For this reason, it is recommended that if there is a likelihood more than more applications will be consuming CTR records, Kinesis Data Streams should be used as the destination for CTRs.

What changes are necessary to replace Kinesis Firehose with Kinesis Data Streams?

The good news is that Kinesis Data Streams can be used as a source for Kinesis Data Firehose, so you can have it both ways! This is the most straightforward approach and will mean that Operata - as well as a Kinesis Data Firehose - can consume CTR records from the same Kinesis Data Stream.

How to enable a Kinesis Stream and move the source for your Firehose

  1. Open the Amazon Connect console at https://console.aws.amazon.com/connect/

  2. On the instances page, choose the instance alias. The instance alias is also your instance name, which appears in your Amazon Connect URL.

  1. In the navigation pane, choose Data streaming.

  2. Choose Enable data streaming.

  3. Choose Kinesis Stream and select an existing stream, or choose Create a Kinesis stream to open the Kinesis console and create the stream.

  4. Take note of the Kinesis Stream Name

  5. In the Kinesis Console, create a new Kinesis Firehose by following this guide, using your new Kinesis Stream as a source, and your existing application choice as the destination. Kinesis Firehose.

  6. After testing, you are good to delete your old Kinesis Firehose.

Need Help?
We all do sometimes! Contact Operata support or your Operata account manager.