TiDB Cloud Ingestion Guide
Prerequisites
Category | Description |
---|---|
Version | Version XXXXXX. |
Settings | Enable Changefeed and configure XXXXXX. |
Permissions | The source database account requires the following minimum permissions: SELECT, SHOW DATABASES. Reference command: GRANT SELECT, SHOW DATABASES ON *.* TO 'username'@'%'; |
Tables | - Tables must have a primary key. - For wide tables, the base types of the associated fields in child tables must be consistent. |
Synchronization Details
- The source database
{db}
corresponds to the data warehouse databaseinput_{db}
. The warehouse database is automatically created after the ingestion task is set up. - The source table
{tableName}
retains the same name in the data warehouse. The table is automatically created after the ingestion task is set up. - For wide tables, the corresponding database and table names in the data warehouse are specified by the user.
- The synchronization impact of source database operations is as follows:
Source Operation | Auto-synced to Warehouse? | Impact on Warehouse | Risk Level |
---|---|---|---|
DML data changes | ✅ Yes | Data is synchronized | Low (Physical deletes are directly synchronized as deletes) |
CREATE TABLE | ❌ No | No change | Low (Requires manually modifying the ingestion task. The new table will undergo a one-time full synchronization.) |
ADD COLUMN | ✅ Yes | The new column is synchronized | Low (Default values for historical data in the column are not synchronized) |
DROP COLUMN | ❌ No | No change | Low |
RENAME COLUMN | ⚠️ Not directly synchronized | A new column is added to receive subsequent data | |
MODIFY COLUMN (Increase length) | ✅ Yes | Column length is changed | Low |
MODIFY COLUMN (Decrease length) | ❌ No | No change | Low |
MODIFY COLUMN_TYPE | ⚠️ Not directly synchronized | The data ingestion task will report an error. The platform handles this through a scheduled schema synchronization task. | |
DROP TABLE | ❌ No | No change | Low |
TRUNCATE TABLE | ❌ No | No change | Low |