ClickHouse
Schemes: clickhouse://, clickhouses://
Query parameters recognised as config fields for this connector. The object-typed columns is set with a JSON literal, e.g. ?columns={...}. Any other ?key=value pair is passed through unchanged as a driver option on the connection URL.
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
async_insert | boolean | no | false | (Publisher only) If true, set the ClickHouse async_insert=1 server setting so inserts are buffered server-side. Defaults to false. |
checkpoint_store | string | no | — | (Consumer only) Where to persist the resume cursor. Because ClickHouse is unsuited to per-row cursor upserts, a durable checkpoint requires an external store URL: - file:///var/lib/mqb/cursors.json → local JSON file - postgres://user@host/db/table / mysql://host/db/table → external SQL table (table optional) - mongodb://host/db/collection → external MongoDB collection (collection optional) - s3://bucket/prefix (also gs://, az://, abfs://) → cloud object store; creds via env May embed connection credentials, so it is treated as a secret. |
columns | object | no | — | (Publisher only) Optional per-column mapping. Each entry maps a target column name to a value token: ${payload:<field>} takes the top-level JSON field <field> of the payload (JSON type preserved), ${metadata:<key>} takes message.metadata["<key>"] (as a string), and any other value is inserted literally. When omitted, the whole payload JSON object is inserted as one row. |
compression | none | gzip | lz4 | zstd | no | gzip | HTTP body compression for inserts and cursor reads (none, gzip, lz4, zstd). Applied as Content-Encoding on the request body and negotiated on the response via Accept-Encoding. lz4/zstd are faster than gzip; all are understood natively by ClickHouse. Defaults to gzip. |
connect_timeout_ms | integer | no | — | Connection (TCP + TLS handshake) timeout in milliseconds. Defaults to 10000ms. |
cursor_column | string | no | — | (Consumer only) Read an existing table non-destructively and resumably, paging by this monotonic column (SELECT … WHERE {cursor_column} > {last} ORDER BY {cursor_column} ASC LIMIT n) and persisting the last read value under cursor_id. |
cursor_id | string | no | — | (Consumer only) Cursor id used to key the persisted resume position. Without it, progress is not persisted and every restart re-copies from the beginning. |
database | string | no | — | Database name. Defaults to default. |
max_polling_interval_ms | integer | no | — | (Consumer only) If set, the poll interval backs off exponentially from polling_interval_ms up to this value while drained, resetting on new rows. Unset = constant interval. |
password | string | no | null | Optional password. Takes precedence over any credentials embedded in the url. |
polling_interval_ms | integer | no | — | (Consumer only) Polling interval in milliseconds when the table is drained. Defaults to 100ms. |
request_timeout_ms | integer | no | — | Request timeout in milliseconds for ClickHouse HTTP calls (inserts, cursor reads, status). Unset = no timeout (wait indefinitely), which suits very large batch inserts. |
select_columns | string | no | — | (Consumer only) Columns to select in cursor_column mode. Defaults to *. |
table | string | yes | — | The table to read from / write to. May be schema-qualified (db.table). |
tls | object | no | see below | TLS configuration for https:// connections. |
url | string | yes | — | ClickHouse HTTP endpoint URL, e.g. http://localhost:8123 (or https://…). If it contains userinfo, it will be treated as a secret. |
username | string | no | null | Optional username. Takes precedence over any credentials embedded in the url. Defaults to default. |
wait_for_async_insert | boolean | no | null | (Publisher only) With async_insert, wait for the server to flush before acking. Defaults to true (durable). False = fire-and-forget: faster, but a crash before flush can drop the batch. |
Struct-typed fields
tls
TLS configuration for secure connections.
Configures Transport Layer Security (TLS/SSL) for encrypted communication. Supports both client certificate (mutual TLS) and server certificate validation.
Examples
use mq_bridge::models::TlsConfig;
let tls = TlsConfig {
required: true,
ca_file: Some("/path/to/ca.pem".to_string()),
cert_file: Some("/path/to/cert.pem".to_string()),
key_file: Some("/path/to/key.pem".to_string()),
..Default::default()
};| Name | Type | Required | Default | Description |
|---|---|---|---|---|
accept_invalid_certs | boolean | no | false | If true, disable server certificate verification (insecure). |
ca_file | string | no | — | Path to the CA certificate file. |
cert_file | string | no | — | Path to the client certificate file (PEM). |
cert_password | string | no | — | Password for the private key (if encrypted). |
key_file | string | no | — | Path to the client private key file (PEM). |
required | boolean | no | false | If true, enable TLS/SSL. |