All Integrations / NATS and PostgreSQL Integration

NATS and PostgreSQL Integration

Powerful performance with an easy integration, powered by Telegraf, the open source data connector built by InfluxData.

Get This Integration for Free
info

This is not the recommended configuration for real-time query at scale. For query and compression optimization, high-speed ingest, and high availability, you may want to consider <a href="/integrations/nats_consumer-influxdb/">NATS and InfluxDB</a>.

5B+

Telegraf downloads

#1

Time series database
Source: DB Engines

1B+

Downloads of InfluxDB

2,800+

Contributors

Table of Contents

Useful Links

Documentation

Telegraf Quickstart Training

Infrastructure Monitoring with Telegraf

Powerful Performance, Limitless Scale

Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.

See Ways to Get Started

Input and output integration overview

<p>The NATS Consumer Input Plugin enables real-time data consumption from NATS messaging subjects, integrating seamlessly into the Telegraf data pipeline for monitoring and metrics gathering.</p>

<p>The Telegraf PostgreSQL plugin allows you to efficiently write metrics to a PostgreSQL database while automatically managing the database schema.</p>

Integration details

NATS

<p>The NATS Consumer Plugin allows Telegraf to read metrics from specified NATS subjects and create metrics based on supported input data formats. Utilizing a Queue Group allows multiple instances of Telegraf to read from a NATS cluster in parallel, enhancing throughput and reliability. This plugin also supports various authentication methods, including username/password, NATS credentials files, and nkey seed files, ensuring secure communication with the NATS servers. It is particularly useful in environments where data persistence and message reliability are critical, thanks to features such as JetStream that facilitate the consumption of historical messages. Additionally, the ability to configure various operational parameters makes this plugin suitable for high-throughput scenarios while maintaining performance integrity.</p>

PostgreSQL

<p>The PostgreSQL plugin enables users to write metrics to a PostgreSQL database or a compatible database, providing robust support for schema management by automatically updating missing columns. The plugin is designed to facilitate integration with monitoring solutions, allowing users to efficiently store and manage time series data. It offers configurable options for connection settings, concurrency, and error handling, and supports advanced features such as JSONB storage for tags and fields, foreign key tagging, templated schema modifications, and support for unsigned integer data types through the pguint extension.</p>

Configuration

NATS

<pre><code class="language-toml">[[inputs.nats_consumer]] ## urls of NATS servers servers = ["nats://localhost:4222"] ## subject(s) to consume ## If you use jetstream you need to set the subjects ## in jetstream_subjects subjects = ["telegraf"] ## jetstream subjects ## jetstream is a streaming technology inside of nats. ## With jetstream the nats-server persists messages and ## a consumer can consume historical messages. This is ## useful when telegraf needs to restart it don't miss a ## message. You need to configure the nats-server. ## https://docs.nats.io/nats-concepts/jetstream. jetstream_subjects = ["js_telegraf"] ## name a queue group queue_group = "telegraf_consumers" ## Optional authentication with username and password credentials # username = "" # password = "" ## Optional authentication with NATS credentials file (NATS 2.0) # credentials = "/etc/telegraf/nats.creds" ## Optional authentication with nkey seed file (NATS 2.0) # nkey_seed = "/etc/telegraf/seed.txt" ## Use Transport Layer Security # secure = false ## Optional TLS Config # tls_ca = "/etc/telegraf/ca.pem" # tls_cert = "/etc/telegraf/cert.pem" # tls_key = "/etc/telegraf/key.pem" ## Use TLS but skip chain &amp; host verification # insecure_skip_verify = false ## Sets the limits for pending msgs and bytes for each subscription ## These shouldn't need to be adjusted except in very high throughput scenarios # pending_message_limit = 65536 # pending_bytes_limit = 67108864 ## Max undelivered messages ## This plugin uses tracking metrics, which ensure messages are read to ## outputs before acknowledging them to the original broker to ensure data ## is not lost. This option sets the maximum messages to read from the ## broker that have not been written by an output. ## ## This value needs to be picked with awareness of the agent's ## metric_batch_size value as well. Setting max undelivered messages too high ## can result in a constant stream of data batches to the output. While ## setting it too low may never flush the broker's messages. # max_undelivered_messages = 1000 ## Data format to consume. ## Each data format has its own unique set of configuration options, read ## more about them here: ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md data_format = "influx" </code></pre>

PostgreSQL

<pre><code class="language-toml"># Publishes metrics to a postgresql database [[outputs.postgresql]] ## Specify connection address via the standard libpq connection string: ## host=... user=... password=... sslmode=... dbname=... ## Or a URL: ## postgres://[user[:password]]@localhost[/dbname]?sslmode=[disable|verify-ca|verify-full] ## See https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING ## ## All connection parameters are optional. Environment vars are also supported. ## e.g. PGPASSWORD, PGHOST, PGUSER, PGDATABASE ## All supported vars can be found here: ## https://www.postgresql.org/docs/current/libpq-envars.html ## ## Non-standard parameters: ## pool_max_conns (default: 1) - Maximum size of connection pool for parallel (per-batch per-table) inserts. ## pool_min_conns (default: 0) - Minimum size of connection pool. ## pool_max_conn_lifetime (default: 0s) - Maximum age of a connection before closing. ## pool_max_conn_idle_time (default: 0s) - Maximum idle time of a connection before closing. ## pool_health_check_period (default: 0s) - Duration between health checks on idle connections. # connection = "" ## Postgres schema to use. # schema = "public" ## Store tags as foreign keys in the metrics table. Default is false. # tags_as_foreign_keys = false ## Suffix to append to table name (measurement name) for the foreign tag table. # tag_table_suffix = "_tag" ## Deny inserting metrics if the foreign tag can't be inserted. # foreign_tag_constraint = false ## Store all tags as a JSONB object in a single 'tags' column. # tags_as_jsonb = false ## Store all fields as a JSONB object in a single 'fields' column. # fields_as_jsonb = false ## Name of the timestamp column ## NOTE: Some tools (e.g. Grafana) require the default name so be careful! # timestamp_column_name = "time" ## Type of the timestamp column ## Currently, "timestamp without time zone" and "timestamp with time zone" ## are supported # timestamp_column_type = "timestamp without time zone" ## Templated statements to execute when creating a new table. # create_templates = [ # '''CREATE TABLE {{ .table }} ({{ .columns }})''', # ] ## Templated statements to execute when adding columns to a table. ## Set to an empty list to disable. Points containing tags for which there is no column will be skipped. Points ## containing fields for which there is no column will have the field omitted. # add_column_templates = [ # '''ALTER TABLE {{ .table }} ADD COLUMN IF NOT EXISTS {{ .columns|join ", ADD COLUMN IF NOT EXISTS " }}''', # ] ## Templated statements to execute when creating a new tag table. # tag_table_create_templates = [ # '''CREATE TABLE {{ .table }} ({{ .columns }}, PRIMARY KEY (tag_id))''', # ] ## Templated statements to execute when adding columns to a tag table. ## Set to an empty list to disable. Points containing tags for which there is no column will be skipped. # tag_table_add_column_templates = [ # '''ALTER TABLE {{ .table }} ADD COLUMN IF NOT EXISTS {{ .columns|join ", ADD COLUMN IF NOT EXISTS " }}''', # ] ## The postgres data type to use for storing unsigned 64-bit integer values (Postgres does not have a native ## unsigned 64-bit integer type). ## The value can be one of: ## numeric - Uses the PostgreSQL "numeric" data type. ## uint8 - Requires pguint extension (https://github.com/petere/pguint) # uint64_type = "numeric" ## When using pool_max_conns&gt;1, and a temporary error occurs, the query is retried with an incremental backoff. This ## controls the maximum backoff duration. # retry_max_backoff = "15s" ## Approximate number of tag IDs to store in in-memory cache (when using tags_as_foreign_keys). ## This is an optimization to skip inserting known tag IDs. ## Each entry consumes approximately 34 bytes of memory. # tag_cache_size = 100000 ## Enable &amp; set the log level for the Postgres driver. # log_level = "warn" # trace, debug, info, warn, error, none </code></pre>

Input and output integration examples

NATS

<ol> <li> <p><strong>Real-Time Analytics Dashboard</strong>: Utilize the NATS plugin to gather metrics from various NATS subjects in real time and feed them into a centralized analytics dashboard. This setup allows for immediate visibility into live application performance, enabling teams to react swiftly to operational issues or performance degradation.</p> </li> <li> <p><strong>Distributed System Monitoring</strong>: Deploy multiple instances of Telegraf configured with the NATS plugin across a distributed architecture. This approach allows teams to aggregate metrics from various microservices efficiently, providing a holistic view of system health and performance while ensuring no messages are dropped during transmission.</p> </li> <li> <p><strong>Historical Message Recovery</strong>: Leverage the capabilities of NATS JetStream along with this plugin to recover and process historical messages after Telegraf has been restarted. This feature is particularly beneficial for applications that require high reliability, ensuring that no critical metrics are lost even in case of service disruptions.</p> </li> <li> <p><strong>Dynamic Load Balancing</strong>: Implement a dynamic load balancing scenario where Telegraf instances consume messages from a NATS cluster based on load. Adjust the queue group settings to control the number of active consumers, allowing for better resource utilization and performance scaling as demand fluctuations occur.</p> </li> </ol>

PostgreSQL

<ol> <li> <p><strong>Real-Time Analytics with Complex Queries</strong>: Leverage the PostgreSQL plugin to store metrics from various sources in a PostgreSQL database, enabling real-time analytics using complex queries. This setup can help data scientists and analysts uncover patterns and trends, as they manipulate relational data across multiple tables while utilizing PostgreSQL’s robust query optimization features. Specifically, users can create sophisticated reports with JOIN operations across different metric tables, revealing insights that would typically remain hidden in embedded systems.</p> </li> <li> <p><strong>Integrating with TimescaleDB for Time-Series Data</strong>: Utilize the PostgreSQL plugin within a TimescaleDB instance to efficiently handle and analyze time-series data. By implementing hypertables, users can achieve greater performance and partitioning of topics over the time dimension. This integration allows users to run analytical queries over large amounts of time-series data while retaining the full power of PostgreSQL’s SQL queries, ensuring reliability and efficiency in metrics analysis.</p> </li> <li> <p><strong>Data Versioning and Historical Analysis</strong>: Implement a strategy using the PostgreSQL plugin to maintain different versions of metrics over time. Users can set up an immutable data table structure where older versions of tables are retained, enabling easy historical analysis. This approach not only provides insights into data evolution but also aids compliance with data retention policies, ensuring that the historical integrity of the datasets remains intact.</p> </li> <li> <p><strong>Dynamic Schema Management for Evolving Metrics</strong>: Use the plugin’s templating capabilities to create a dynamically changing schema that responds to metric variations. This use case allows organizations to adapt their data structure as metrics evolve, adding necessary fields and ensuring adherence to data integrity policies. By leveraging templated SQL commands, users can extend their database without manual intervention, facilitating agile data management practices.</p> </li> </ol>

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.

Useful Links

Documentation

Telegraf Quickstart Training

Infrastructure Monitoring with Telegraf

Powerful Performance, Limitless Scale

Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.

See Ways to Get Started

Related Integrations

HTTP and InfluxDB Integration

The HTTP plugin collects metrics from one or more HTTP(S) endpoints. It supports various authentication methods and configuration options for data formats.

View Integration

Kafka and InfluxDB Integration

This plugin reads messages from Kafka and allows the creation of metrics based on those messages. It supports various configurations including different Kafka settings and message processing options.

View Integration

Kinesis and InfluxDB Integration

The Kinesis plugin allows for reading metrics from AWS Kinesis streams. It supports multiple input data formats and offers checkpointing features with DynamoDB for reliable message processing.

View Integration

Start building now

Download Telegraf today or learn more about InfluxDB.

telegraf Get started with InfluxDB
Product & Solutions
Developers
Company

548 Market St, PMB 77953
San Francisco, California 94104

© 2025 InfluxData Inc. All Rights Reserved.

Legal Security Cookie Policy Comparison