Introduction

crossing streams

mq-bridge moves data reliably between message brokers, databases, files, and HTTP — with batching, retries, dead-letter queues, deduplication, and change data capture built in. One Rust engine, one config format, two ways to use it:

Almost everything in this book applies to both: connectors, middleware, delivery guarantees, and tuning share the same settings. A CLI URL’s query parameters are the same fields you write in a library’s YAML config.

Supported integrations include Kafka, RabbitMQ (AMQP), NATS, AWS SQS / SNS, MQTT, IBM MQ, HTTP, WebSocket, gRPC, ZeroMQ, MongoDB, Redis Streams, ClickHouse, Postgres CDC, SQLx (MySQL, MariaDB, PostgreSQL, SQLite), cloud object storage, and filesystem endpoints — plus Pulsar, Meilisearch, and Redpanda Connect through plugins.

A quick taste

In code — the same route runs in Rust, Python, or Node.js:

from mq_bridge import Route
Route.from_file("routes.yaml", "kafka_to_nats").run()

Without code — mqb copy moves data between databases, queues, and files in a single line of bash:

mqb copy \
  --from 'postgres://user:pass@localhost/db?table=src&sslmode=disable' \
  --to   'file://out.jsonl?format=raw' \
  --drain

The scheme selects the endpoint and query parameters configure it, so any source→sink pair is just one URL each. And it’s quick: in benchmarks a 1,000,000-row Postgres → JSONL job sustained 413,907 rows/s at ~55 MiB peak RSS (mq-bridge 0.4.20) — see Performance tuning for the full table and the conditions each row was measured under.

Philosophy

The project has one main bias: move data reliably without forcing the rest of the application to care too much about the transport. Kafka offsets, RabbitMQ nacks, HTTP responses, MongoDB polling, WebSocket frames, and file rows are all different in real life, but route code should still be able to receive a batch, process it, publish it, and commit it.

• Fast by default. Every endpoint is optimized around batch-shaped APIs, and the headless surfaces ship tuned for throughput: the copy CLI and the MCP server default to batch_size: 1024 and concurrency: 4. The library/config primitive defaults to batch_size: 512, concurrency: 1: batches fill opportunistically — a route takes whatever is already queued rather than waiting — so throughput comes for free while parallelism stays a deliberate per-route choice.
• Reliability is built in, not bolted on. Retries, dead-letter queues, deduplication, rate limiting, and cookie/session persistence wrap any endpoint. Ack/nack behaviour and retry/DLQ handling were designed to work with batching, including commit sequencing for cumulative-ack brokers.
• Not a framework. It is not a domain framework, an actor runtime, or a full stream processor. It cares about transport, routing, and delivery behaviour, not about prescribing your domain model.

Where to go next

• Writing code? Start with the library quick start, then Embed the library.
• Moving data without code? Start with Installation and the copy quick start.
• Want the end-to-end walkthroughs? See the Tutorials.
• Looking for a specific task? The Cookbook has short recipes.
• Need exact fields and defaults? The Reference is authoritative.
• Running it in production? See Operations, especially the Performance tuning page.
• Driving it from an AI agent? The same binary is an MCP server — the rows move without entering the model’s context.

Library vs. app

mq-bridge is the engine/library; mq-bridge-app is the application built on it. This book covers both; the engine’s Rust API reference lives on docs.rs, and the language bindings API covers Python and Node.js.