Connect plugin: inputs

The 56 Redpanda Connect inputs the Connect plugin links, generated from the component specs of Redpanda Connect v4.110.0. Field descriptions are Redpanda Connect’s own, shortened to one sentence; follow Upstream documentation for the rest. Install and configuration forms are on the plugin page.

Use one as a route input:

input:
  custom:
    name: connect
    config:
      connector: mqtt          # the component name below
      urls: ["tcp://localhost:1883"]
      topics: ["orders"]

Category	Components
AWS	`aws_cloudwatch_logs`, `aws_kinesis`, `aws_s3`, `aws_sqs`
Azure	`azure_blob_storage`, `azure_cosmosdb`, `azure_queue_storage`, `azure_table_storage`
GCP	`gcp_bigquery_select`, `gcp_cloud_storage`, `gcp_pubsub`
Local	`csv`, `file`, `parquet`, `stdin`
Network	`http_client`, `http_server`, `nanomsg`, `sftp`, `socket`, `socket_server`, `websocket`
Services	`amqp_0_9`, `amqp_1`, `aws_cloudwatch_logs`, `aws_kinesis`, `aws_s3`, `aws_sqs`, `azure_blob_storage`, `azure_queue_storage`, `azure_table_storage`, `beanstalkd`, `cassandra`, `cockroachdb_changefeed`, `discord`, `gcp_bigquery_select`, `gcp_cloud_storage`, `gcp_pubsub`, `git`, `hdfs`, `mongodb`, `mqtt`, `nats`, `nats_jetstream`, `nats_kv`, `nats_stream`, `nsq`, `pulsar`, `redis_list`, `redis_pubsub`, `redis_scan`, `redis_streams`, `spicedb_watch`, `sql_raw`, `sql_select`, `timeplus`, `twitter_search`
Social	`discord`, `twitter_search`
SpiceDB	`spicedb_watch`
Utility	`batched`, `broker`, `dynamic`, `generate`, `inproc`, `read_until`, `resource`, `sequence`, `subprocess`

`amqp_0_9`

Connects to an AMQP (0.91) queue. AMQP is a messaging protocol used by various message brokers, including RabbitMQ.

connector: amqp_0_9 · URI connect+amqp-0-9://

Field	Type	Default	Description
`urls`	list of string	required	A list of URLs to connect to.
`queue`	string	required	An AMQP queue to consume from.
`consumer_tag`	string	`""`	A consumer tag.
`prefetch_count`	int	`10`	The maximum number of pending messages to have consumed at a time.

Advanced: queue_declare, bindings_declare, auto_ack, nack_reject_patterns, prefetch_size, tls.

Field	Type	Default	Description
`log_group_name`	string	required	The name of the CloudWatch Log Group to consume from.
`log_stream_names`	list of string		An optional list of log stream names to consume from.
`log_stream_prefix`	string		An optional log stream name prefix to filter streams.
`filter_pattern`	string		An optional CloudWatch Logs filter pattern to apply when querying log events.
`start_time`	string		The time to start consuming log events from.
`poll_interval`	string	`"5s"`	The interval at which to poll for new log events.
`auto_replay_nacks`	bool	`true`	Whether messages that are rejected (nacked) at the output level should be automatically replayed indefinitely, eventually resulting in back pressure if the cause of the rejections is persistent.

Field	Type	Default	Description
`streams`	list of string	required	One or more Kinesis data streams to consume from.
`dynamodb`	object		Determines the table used for storing and accessing the latest consumed sequence for shards, and for coordinating balanced consumers of streams.
`checkpoint_limit`	int	`1024`	The maximum gap between the in flight sequence versus the latest acknowledged sequence at a given time.
`auto_replay_nacks`	bool	`true`	Whether messages that are rejected (nacked) at the output level should be automatically replayed indefinitely, eventually resulting in back pressure if the cause of the rejections is persistent.
`commit_period`	string	`"5s"`	The period of time between each update to the checkpoint table.
`steal_grace_period`	string	`"2s"`	Determines how long beyond the next commit period a client will wait when stealing a shard for the current owner to store a checkpoint.
`start_from_oldest`	bool	`true`	Whether to consume from the oldest message when a sequence does not yet exist for the stream.
`batching`	object		Allows you to configure a batching policy.

Field	Type	Default	Description
`bucket`	string	`""`	The bucket to consume from.
`prefix`	string	`""`	An optional path prefix, if set only objects with the prefix are consumed when walking a bucket.
`scanner`	scanner	`{"to_the_end": {}}`	The scanner by which the stream of bytes consumed will be broken out into individual messages.
`sqs`	object		Consume SQS messages in order to trigger key downloads.

Field	Type	Default	Description
`storage_account`	string	`""`	The storage account to access.
`storage_access_key`	string	`""`	The storage account access key.
`storage_connection_string`	string	`""`	A storage account connection string.
`storage_sas_token`	string	`""`	The storage account SAS token.
`container`	string	required	The name of the container from which to download blobs.
`prefix`	string	`""`	An optional path prefix, if set only objects with the prefix are consumed.
`scanner`	scanner	`{"to_the_end": {}}`	The scanner by which the stream of bytes consumed will be broken out into individual messages.
`targets_input`	input		EXPERIMENTAL: An optional source of download targets, configured as a regular Redpanda Connect input.

Keyboard shortcuts

mq-bridge