Syslog and Thanos Integration

Input and output integration overview

<p>The Syslog plugin enables the collection of syslog messages from various sources using standard networking protocols. This functionality is critical for environments where systems need to be monitored and logged efficiently.</p>

<p>This plugin sends metrics from Telegraf to Thanos using the Prometheus remote write protocol over HTTP, allowing efficient and scalable ingestion into Thanos Receive components.</p>

Integration details

Syslog

<p>The Syslog plugin for Telegraf captures syslog messages transmitted over various protocols such as TCP, UDP, and TLS. It supports both RFC 5424 (the newer syslog protocol) and the older RFC 3164 (BSD syslog protocol). This plugin operates as a service input, effectively starting a service that listens for incoming syslog messages. Unlike traditional plugins, service inputs may not function with standard interval settings or CLI options like <code>--once</code>. It includes options for setting network configurations, socket permissions, message handling, and connection handling. Furthermore, the integration with Rsyslog allows forwarding of logging messages, making it a powerful tool for collecting and relaying system logs in real-time, thus seamlessly integrating into monitoring and logging systems.</p>

Thanos

<p>Telegraf’s HTTP plugin can send metrics directly to Thanos via its Remote Write-compatible Receive component. By setting the data format to <code>prometheusremotewrite</code>, Telegraf can serialize metrics into the same protobuf-based format used by native Prometheus clients. This setup enables high-throughput, low-latency metric ingestion into Thanos, facilitating centralized observability at scale. It is particularly useful in hybrid environments where Telegraf is collecting metrics from systems outside Prometheus’ native reach, such as SNMP devices, Windows hosts, or custom apps, and streams them directly to Thanos for long-term storage and global querying.</p>

Configuration

Syslog

Thanos

Input and output integration examples

Syslog

<ol> <li> <p><strong>Centralized Log Management</strong>: Use the Syslog plugin to aggregate log messages from multiple servers into a central logging system. This setup can help in monitoring overall system health, troubleshooting issues effectively, and maintaining audit trails by collecting syslog data from different sources.</p> </li> <li> <p><strong>Real-Time Alerting</strong>: Integrate the Syslog plugin with alerting tools to trigger real-time notifications when specific log patterns or errors are detected. For example, if a critical system error appears in the logs, an alert can be sent to the operations team, minimizing downtime and performing proactive maintenance.</p> </li> <li> <p><strong>Security Monitoring</strong>: Leverage the Syslog plugin for security monitoring by capturing logs from firewalls, intrusion detection systems, and other security devices. This logging capability enhances security visibility and helps in investigating potentially malicious activities by analyzing the captured syslog data.</p> </li> <li> <p><strong>Application Performance Tracking</strong>: Utilize the Syslog plugin to monitor application performance by collecting logs from various applications. This integration helps in analyzing the application’s behavior and performance trends, thus aiding in optimizing application processes and ensuring smoother operation.</p> </li> </ol>

Thanos

<ol> <li> <p><strong>Agentless Cloud Monitoring</strong>: Deploy Telegraf agents across cloud VMs to collect system and application metrics, then stream them directly into Thanos using Remote Write. This provides centralized observability without requiring Prometheus nodes at each location.</p> </li> <li> <p><strong>Scalable Windows Host Monitoring</strong>: Use Telegraf on Windows machines to collect OS-level metrics and send them via Remote Write to Thanos Receive. This enables observability across heterogeneous environments with native Prometheus support only on Linux.</p> </li> <li> <p><strong>Cross-Region Metrics Federation</strong>: Telegraf agents in multiple geographic regions can push data to region-local Thanos Receivers using this plugin. From there, Thanos can deduplicate and query metrics globally, reducing latency and network egress costs.</p> </li> <li> <p><strong>Integrating Third-Party Data into Thanos</strong>: Collect metrics from custom telemetry sources such as REST APIs or proprietary logs using Telegraf inputs and forward them to Thanos via Remote Write. This brings non-native data into a Prometheus-compatible, long-term analytics pipeline.</p> </li> </ol>

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.