Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

PostgreSQL / MySQL / MariaDB / SQLite

Schemes: postgres://, postgresql://, mysql://, mariadb://, sqlite://

Query parameters recognised as config fields for this connector. The object-typed tls is set with a JSON literal, e.g. ?tls={...}. Any other ?key=value pair is passed through unchanged as a driver option on the connection URL.

NameTypeRequiredDefaultDescription
acquire_timeout_msintegernoTimeout for acquiring a connection from the pool in milliseconds. Defaults to 30000ms.
auto_create_tablebooleannofalse(Publisher only) If true, automatically create the table and indexes if they don’t exist. Defaults to false.
bulk_copybooleannofalse(Publisher only) PostgreSQL only. Bulk-load batches via COPY FROM STDIN (much faster than multi-row INSERT). Requires a token-based insert_query; no ON CONFLICT/RETURNING.
checkpoint_storestringno(Consumer only) Where to persist the resume cursor in cursor_column mode. A URL selects the backend; a bare name (or /name) reuses the source datastore with that table name: - absent → source datastore, table mqb_cursors_<source_table> (auto-unique) - /my_cursors → source datastore, table my_cursors - file:///var/lib/mqb/cursors.json → local JSON file (read-only / write-restricted sources) - postgres://user@host/db/table or mysql://host/db/table → external SQL table (table optional) - mongodb://host/db/collection → external MongoDB collection (collection optional) - s3://bucket/prefix (also gs://, az://, abfs://) → cloud object store; creds via env When no table/collection is named, it defaults to mqb_cursors_<source_table>. May embed connection credentials, so it is treated as a secret.
create_publicationbooleannofalse(Consumer only, CDC) When publication is set, create it if missing (default false). Needs table-owner privilege: it is auto-published FOR TABLE {table}.
cursor_columnstringno(Consumer only) Read an existing table non-destructively and resumably, paging by this monotonic column (SELECT * FROM {table} WHERE {cursor_column} > $last ORDER BY {cursor_column} ASC LIMIT n) and persisting the last read value under cursor_id. Does not delete/lock source rows. Mutually exclusive with delete_after_read.
cursor_idstringno(Consumer only) Cursor id used to key the persisted resume position. Recommended when cursor_column is set: without it, progress is not persisted and every restart re-copies from the beginning.
delete_after_readbooleannofalse(Consumer only) If true, delete messages after processing.
idle_timeout_msintegernoMaximum idle time for a connection in milliseconds. Defaults to 600000ms (10 minutes).
insert_querystringno(Publisher only) Optional. A custom SQL INSERT query. Use ? as a placeholder for the payload. If not provided, a default INSERT INTO {table} (payload) VALUES (?) is used. For multi-column inserts, embed explicit source tokens directly in the query: ${metadata:<key>} binds message.metadata["<key>"], and ${payload:<field>} binds the top-level JSON field <field> of the payload (types preserved: numbers/bools stay numeric/bool). There is no fallback between the two: an absent metadata key, non-JSON payload, or missing/non-scalar field binds SQL NULL. Example: INSERT INTO orders (customer_id, sku, qty) VALUES (${metadata:customer_id}, ${payload:sku}, ${payload:qty}). A query with no ${...} tokens behaves exactly as before (whole payload bound once). auto_create_table is not supported together with a token-based query. Tokens bind as text/number/bool; Postgres won’t implicitly cast text into a numeric/timestamptz column (these arrive as JSON strings from a sql source). Add an explicit cast next to the token — it is preserved verbatim in the SQL: VALUES (${payload:amount}::numeric, ${payload:created_at}::timestamptz).
max_connectionsintegernoMaximum number of connections in the pool. Defaults to 10.
max_lifetime_msintegernoMaximum lifetime of a connection in milliseconds. Defaults to 1800000ms (30 minutes).
max_polling_interval_msintegerno(Consumer only) If set, the poll interval backs off exponentially from polling_interval_ms up to this value while drained, resetting on new rows. Unset = constant interval.
min_connectionsintegernoMinimum number of connections to keep in the pool. Defaults to 0.
passwordstringnonullOptional password. Takes precedence over any credentials embedded in the url.
polling_interval_msintegerno(Consumer only) Polling interval in milliseconds. Defaults to 100ms.
publicationstringno(Consumer only, PostgreSQL) If set, consume via logical-replication CDC instead of cursor polling: streams inserts/updates/deletes from this publication. Requires the postgres-cdc feature and a Postgres URL. For full control use the dedicated postgres_cdc endpoint.
select_querystringno(Consumer only) Optional. A custom SQL SELECT query to fetch messages. This is only supported for PostgreSQL and Microsoft SQL Server. The query must include a placeholder for the batch size ($1 for PostgreSQL, @p1 for SQL Server). The bridge will bind the route’s batch_size to this placeholder.
sharedbooleannotrueShare one connection pool per connection (default: true); false forces a dedicated pool.
slot_namestringno(Consumer only, CDC) Replication slot name; created if missing. Defaults to mq_bridge_slot.
tablestringyesThe table to interact with.
tlsobjectnosee belowTLS configuration for the database connection.
urlstringyesDatabase connection URL. If it contains userinfo, it will be treated as a secret.
usernamestringnonullOptional username. Takes precedence over any credentials embedded in the url.

Struct-typed fields

tls

TLS configuration for secure connections.

Configures Transport Layer Security (TLS/SSL) for encrypted communication. Supports both client certificate (mutual TLS) and server certificate validation.

Examples

use mq_bridge::models::TlsConfig;

let tls = TlsConfig {
    required: true,
    ca_file: Some("/path/to/ca.pem".to_string()),
    cert_file: Some("/path/to/cert.pem".to_string()),
    key_file: Some("/path/to/key.pem".to_string()),
    ..Default::default()
};
NameTypeRequiredDefaultDescription
accept_invalid_certsbooleannofalseIf true, disable server certificate verification (insecure).
ca_filestringnoPath to the CA certificate file.
cert_filestringnoPath to the client certificate file (PEM).
cert_passwordstringnoPassword for the private key (if encrypted).
key_filestringnoPath to the client private key file (PEM).
requiredbooleannofalseIf true, enable TLS/SSL.