AWS Data Firehose and SigNoz Integration

Input and output integration overview

<p>This plugin listens for metrics sent via HTTP from AWS Data Firehose in supported data formats, providing real-time data ingestion capabilities.</p>

<p>This configuration turns any Telegraf agent into a Remote Write publisher for SigNoz, streaming rich metrics straight into the SigNoz backend with a single URL change.</p>

Integration details

AWS Data Firehose

<p>The AWS Data Firehose Telegraf plugin is designed to receive metrics from AWS Data Firehose via HTTP. This plugin listens for incoming data in various formats and processes it according to the request-response schema outlined in the official AWS documentation. Unlike standard input plugins that operate on a fixed interval, this service plugin initializes a listener that remains active, waiting for incoming metrics. This allows for real-time data ingestion from AWS Data Firehose, making it suitable for scenarios where immediate data processing is required. Key features include the ability to specify service addresses, paths, and support for TLS connections for secure data transmission. Additionally, the plugin accommodates optional authentication keys and custom tags, enhancing its flexibility in various use cases involving data streaming and processing.</p>

SigNoz

<p>SigNoz is an open source observability platform that stores metrics, traces, and logs. When you deploy SigNoz, its <strong>signoz-otel-collector-metrics</strong> service exposes a Prometheus Remote Write receiver (default <strong>:13133/api/v1/write</strong>). By configuring Telegraf’s Prometheus plugin to point at this endpoint, you can push any Telegraf collected metrics, SNMP counters, cloud services, or business KPIs—directly into SigNoz. The plugin natively serializes metrics in the Remote Write protobuf format, supports external labels, metadata export, retries, and TLS or bearer-token auth, so it fits zero-trust and multi-tenant SigNoz clusters. Inside SigNoz, the data lands in ClickHouse tables that back Metrics Explorer, alert rules, and unified dashboards. This approach lets organizations unify Prometheus and OTLP pipelines, enables long-term retention powered by ClickHouse compression, and avoids vendor lock-in while retaining PromQL-style queries.</p>

Configuration

AWS Data Firehose

SigNoz

Input and output integration examples

AWS Data Firehose

<ol> <li> <p><strong>Real-Time Data Analytics</strong>: Using the AWS Data Firehose plugin, organizations can stream data in real-time from various sources, such as application logs or IoT devices, directly into analytics platforms. This allows data teams to analyze incoming data as it is generated, enabling rapid insights and operational adjustments based on fresh metrics.</p> </li> <li> <p><strong>Profile Access Patterns for Optimization</strong>: By collecting data about how clients interact with applications through AWS Data Firehose, businesses can gain valuable insights into user behavior. This can drive content personalization strategies or optimize server architecture for better performance based on traffic patterns.</p> </li> <li> <p><strong>Automated Alerting Mechanism</strong>: Integrating AWS Data Firehose with alerting systems via this plugin allows teams to set up automated alerts based on specific metrics collected. For example, if a particular threshold is reached in the input data, alerts can trigger operations teams to investigate potential issues before they escalate.</p> </li> </ol>

SigNoz

<ol> <li> <p><strong>Multi-Cluster Federated Monitoring</strong>: Drop a Telegraf DaemonSet into each Kubernetes cluster, tag metrics with <code>cluster=&lt;name&gt;</code>, and Remote Write them to a central SigNoz instance. Ops teams get a single PromQL window across prod, staging, and edge clusters without running Thanos sidecars.</p> </li> <li> <p><strong>Factory-Floor Edge Gateway</strong>: A rugged Intel NUC on the shop floor runs Telegraf to scrape Modbus PLCs and environmental sensors. It batches readings every 5 seconds and pushes them over an intermittent 4G link to SigNoz SaaS. ClickHouse compression keeps costs low while AI-based outlier detection in SigNoz flags overheating motors before failure.</p> </li> <li> <p><strong>SaaS Usage Metering</strong>: Telegraf runs alongside each micro-service, exporting per-tenant counters (<code>api_calls</code>, <code>gigabytes_processed</code>). Remote Write streams the data to SigNoz where a scheduled ClickHouse materialized view aggregates usage for monthly billing—no separate metering stack required.</p> </li> <li> <p><strong>Autoscaling Feedback Loop</strong>: Combine Telegraf’s Kubernetes input with the Remote Write output to publish granular pod CPU and queue-length metrics into SigNoz. A custom SigNoz alert fires when P95 latency breaches 200 ms and a GitOps controller reads that alert to trigger a HorizontalPodAutoscaler tweak—closing the loop between observability and automation.</p> </li> </ol>

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.