Data Integration Introduction

Overview

Data Integration supports real-time data synchronization from various data sources for multiple scenarios. It synchronizes multiple databases and tables to the platform through a one-time full import followed by real-time incremental updates, offering the following features:

• Real-time: Supports data synchronization at the second level.
• Reliability: Ensures data consistency and accuracy through various mechanisms such as error recovery and automatic retries.
• Versatility:
  1. Diverse Data Sources: Supports a variety of data sources.
  2. Flexible Scenarios: Supports integrated full and incremental synchronization, database/table sharding, and wide table consolidation.
• Maintainability: Supports job monitoring and log viewing for easy troubleshooting.
• Ease of Use: Features a user-friendly interface where users only need to configure essential information.

Feature Details

Use Cases

Scenario	Description
One-to-One	Replicates the source table's structure and data to a target table on a one-to-one basis, ensuring both are synchronized.
Table Merging	Automatically merges multiple sharded source tables into a single standard target table, suitable for sharding scenarios like those using ShardingSphere.
Wide Table Consolidation	Merges multiple source tables into a single wide table in the target data warehouse using join keys, facilitating unified queries and efficient analysis.

Supported Data Sources

Category	Data Source	Connection Guide
Relational Database	MYSQL	👉MySQL Connection Guide
Relational Database	TiDB Cloud	👉TiDB Cloud Connection Guide

Data Type Mapping

👉Data Type Mapping

FAQ

👉FAQ