合迁移 MySQL 数据到 OceanBase 集群

2022年1月11日

0 252 2

Tags：数据迁移 OceanBase

👉 本文共约2760个字，系统预计阅读时间或需11分钟。

说明
练习目的
练习条件
练习内容
使用 mysqldump 将 mysql的表结构和数据同步到 OceanBase 的MySQL 租户中
使用sysbench给mysql环境生成数据
使用mysqldump导出
使用source命令导入数据
使用datax (离线)从 MySQL 同步表数据到 OceanBase
部署DATAX软件
修改同步作业文件
启动同步作业
datax 实现 OceanBase 以 csv 格式的导入导出
OceanBase 以 csv 格式导出数据
OceanBase 以 csv 格式导入数据
Canal 实现 mysql 与 OceanBase 的增量同步
mysql源端开启二进制日志
部署 canal admin
启动canal
部署canal deployer
启动 canal-deployer 服务
通过 canal-admin 的 web 服务配置 canal server 和 canal instance
创建 canal instance 并启动
部署 canal adapter
启动 canal adapter 服务
验证增量数据的同步情况
检查验证目标端数据同步情况

OceanBase Docker安装体验：https://www.dbaup.com/oceanbase-dockeranzhuangtiyan.html
手动部署 OceanBase 单副本集群：https://www.dbaup.com/shoudongbushu-oceanbase-danfubenjiqun.html
手动部署 OceanBase 三副本集群（在同一个节点）：https://www.dbaup.com/shoudongbushu-oceanbase-sanfubenjiqunzaitongyigejiedian.html
使用OBD部署一个OceanBase 三副本集群并使用Promethues监控OB（在不同节点）：https://www.dbaup.com/shiyongobdbushuyigeoceanbase-sanfubenjiqunbingshiyongpromethuesjiankongobzaibutongjiedian.html

说明

练习目的

本次练习目的掌握从 MySQL 向 OceanBase 迁移数据的基本方法：mysqldump、datax 、canal 等。

练习条件

有服务器，内存资源至少 12G*1台，部署有 OceanBase 集群。
有 mysql 数据库，有少量业务表和数据（数据量不要求很多）。没有业务表和数据的就推荐使用 sysbench、 tpcc 或者 tpch 的业务数据。

练习内容

请记录并分享下列内容：

（必选）使用 mysqldump 将 mysql的表结构和数据同步到 OceanBase 的MySQL 租户中。
（必选）使用 datax 配置至少一个表的 MySQL 到 OceanBase 的 MySQL 租户的离线同步。
（可选）使用 datax 配置至少一个表的 OceanBase 到 CSV 以及 CSV 到 OceanBase 的离线同步。
（可选）使用 canal 配置 MySQL 的增量同步到 OceanBase 的 MySQL 租户。

使用 mysqldump 将 mysql的表结构和数据同步到 OceanBase 的MySQL 租户中

源库：MySQL 5.7.36，IP地址：172.17.0.3，数据库：sbtest，10张表，每张表10万行数据

目标库：OB 3.1.1，IP地址：172.17.0.2，数据库：sbtest，租户：obmysql

使用sysbench给mysql环境生成数据

[root@docker35 ~]# mysql -uroot -plhr -h172.17.0.3
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.36-log MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 
mysql> create database sbtest;
Query OK, 1 row affected (0.06 sec)

Bye

[root@docker35 ~]# sysbench /usr/share/sysbench/oltp_common.lua --time=300 --mysql-host=172.17.0.3 --mysql-port=3306 --mysql-user=root --mysql-password=lhr --mysql-db=sbtest --table-size=100000 --tables=10 --threads=16 --events=999999999   prepare
sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)

Initializing worker threads...

Creating table 'sbtest8'...
Creating table 'sbtest1'...
Creating table 'sbtest7'...
Creating table 'sbtest6'...
Creating table 'sbtest5'...
Creating table 'sbtest3'...
Creating table 'sbtest9'...
Creating table 'sbtest4'...
Creating table 'sbtest10'...
Creating table 'sbtest2'...
Inserting 100000 records into 'sbtest7'
Inserting 100000 records into 'sbtest2'
Inserting 100000 records into 'sbtest3'
Inserting 100000 records into 'sbtest9'
Inserting 100000 records into 'sbtest5'
Inserting 100000 records into 'sbtest6'
Inserting 100000 records into 'sbtest10'
Inserting 100000 records into 'sbtest1'
Inserting 100000 records into 'sbtest4'
Inserting 100000 records into 'sbtest8'
Creating a secondary index on 'sbtest2'...
Creating a secondary index on 'sbtest7'...
Creating a secondary index on 'sbtest3'...
Creating a secondary index on 'sbtest9'...
Creating a secondary index on 'sbtest10'...
Creating a secondary index on 'sbtest5'...
Creating a secondary index on 'sbtest6'...
Creating a secondary index on 'sbtest8'...
Creating a secondary index on 'sbtest1'...
Creating a secondary index on 'sbtest4'...

[root@docker35 ~]# mysql -uroot -plhr -h172.17.0.3
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 36
Server version: 5.7.36-log MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use sbtest;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest10         |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
| sbtest6          |
| sbtest7          |
| sbtest8          |
| sbtest9          |
+------------------+
10 rows in set (0.00 sec)

mysql> select count(*) from sbtest1;
+----------+
| count(*) |
+----------+
|   100000 |
+----------+
1 row in set (0.04 sec)

mysql> SELECT TABLE_NAME,DATA_LENGTH,INDEX_LENGTH,(DATA_LENGTH+INDEX_LENGTH) as length,TABLE_ROWS,concat(round((DATA_LENGTH+INDEX_LENGTH)/1024/1024,2), 'MB') as total_size 
    -> FROM information_schema.TABLES 
    -> WHERE TABLE_SCHEMA='sbtest' 
    -> order by length desc
    -> ;
+------------+-------------+--------------+----------+------------+------------+
| TABLE_NAME | DATA_LENGTH | INDEX_LENGTH | length   | TABLE_ROWS | total_size |
+------------+-------------+--------------+----------+------------+------------+
| sbtest6    |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
| sbtest5    |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
| sbtest4    |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
| sbtest3    |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
| sbtest2    |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
| sbtest10   |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
| sbtest1    |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
| sbtest9    |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
| sbtest8    |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
| sbtest7    |    22593536 |      1589248 | 24182784 |      98712 | 23.06MB    |
+------------+-------------+--------------+----------+------------+------------+
10 rows in set (0.01 sec)

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

[root@docker35 ~]# mysql -uroot -plhr -h172.17.0.3

mysql: [Warning] Using a password on the command line interface can be insecure.

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 2

Server version: 5.7.36-log MySQL Community Server (GPL)

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

mysql> create database sbtest;

Query OK, 1 row affected (0.06 sec)

Bye

[root@docker35 ~]# sysbench /usr/share/sysbench/oltp_common.lua --time=300 --mysql-host=172.17.0.3 --mysql-port=3306 --mysql-user=root --mysql-password=lhr --mysql-db=sbtest --table-size=100000 --tables=10 --threads=16 --events=999999999 prepare

sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)

Initializing worker threads...

Creating table 'sbtest8'...

Creating table 'sbtest1'...

Creating table 'sbtest7'...

Creating table 'sbtest6'...

Creating table 'sbtest5'...

Creating table 'sbtest3'...

Creating table 'sbtest9'...

Creating table 'sbtest4'...

Creating table 'sbtest10'...

Creating table 'sbtest2'...

Inserting 100000 records into 'sbtest7'

Inserting 100000 records into 'sbtest2'

Inserting 100000 records into 'sbtest3'

Inserting 100000 records into 'sbtest9'

Inserting 100000 records into 'sbtest5'

Inserting 100000 records into 'sbtest6'

Inserting 100000 records into 'sbtest10'

Inserting 100000 records into 'sbtest1'

Inserting 100000 records into 'sbtest4'

Inserting 100000 records into 'sbtest8'

Creating a secondary index on 'sbtest2'...

Creating a secondary index on 'sbtest7'...

Creating a secondary index on 'sbtest3'...

Creating a secondary index on 'sbtest9'...

Creating a secondary index on 'sbtest10'...

Creating a secondary index on 'sbtest5'...

Creating a secondary index on 'sbtest6'...

Creating a secondary index on 'sbtest8'...

Creating a secondary index on 'sbtest1'...

Creating a secondary index on 'sbtest4'...

[root@docker35 ~]# mysql -uroot -plhr -h172.17.0.3

mysql: [Warning] Using a password on the command line interface can be insecure.

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 36

Server version: 5.7.36-log MySQL Community Server (GPL)

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use sbtest;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

Database changed

mysql> show tables;

+------------------+

| Tables_in_sbtest |

+------------------+

| sbtest1 |

| sbtest10 |

| sbtest2 |

| sbtest3 |

| sbtest4 |

| sbtest5 |

| sbtest6 |

| sbtest7 |

| sbtest8 |

| sbtest9 |

+------------------+

10 rows in set (0.00 sec)

mysql> select count(*) from sbtest1;

+----------+

| count(*) |

+----------+

| 100000 |

+----------+

1 row in set (0.04 sec)

mysql> SELECT TABLE_NAME,DATA_LENGTH,INDEX_LENGTH,(DATA_LENGTH+INDEX_LENGTH) as length,TABLE_ROWS,concat(round((DATA_LENGTH+INDEX_LENGTH)/1024/1024,2), 'MB') as total_size

-> FROM information_schema.TABLES

-> WHERE TABLE_SCHEMA='sbtest'

-> order by length desc

-> ;

+------------+-------------+--------------+----------+------------+------------+

+------------+-------------+--------------+----------+------------+------------+

| sbtest6 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

| sbtest5 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

| sbtest4 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

| sbtest3 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

| sbtest2 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

| sbtest10 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

| sbtest1 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

| sbtest9 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

| sbtest8 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

| sbtest7 | 22593536 | 1589248 | 24182784 | 98712 | 23.06MB |

+------------+-------------+--------------+----------+------------+------------+

10 rows in set (0.01 sec)

使用mysqldump导出

[root@docker35 soft]# mysqldump -uroot -plhr -h172.17.0.3 -P3306 --single-transaction --hex-blob --routines --events --triggers --master-data=2 --set-gtid-purged=OFF --databases sbtest --default-character-set=utf8  --max_allowed_packet=512M > sbtest.sql
mysqldump: [Warning] Using a password on the command line interface can be insecure.
[root@docker35 soft]# ll -h
total 315M
-rw-r--r-- 1 root root 7.9M Nov  5 09:10 obproxy-3.2.0-1.el7.x86_64.rpm
-rw-r--r-- 1 root root  47M Dec 31 17:57 oceanbase-ce-3.1.2-10000392021123010.el7.x86_64.rpm
-rw-r--r-- 1 root root 156K Dec 31 19:40 oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm
-rw-r--r-- 1 root root  72M Dec 18 06:25 prometheus-2.32.1.linux-amd64.tar.gz
-rw-r--r-- 1 root root 189M Jan 10 15:38 sbtest.sql

[root@docker35 soft]# mysqldump -uroot -plhr -h172.17.0.3 -P3306 --single-transaction --hex-blob --routines --events --triggers --master-data=2 --set-gtid-purged=OFF --databases sbtest --default-character-set=utf8 --max_allowed_packet=512M > sbtest.sql

mysqldump: [Warning] Using a password on the command line interface can be insecure.

[root@docker35 soft]# ll -h

total 315M

-rw-r--r-- 1 root root 7.9M Nov 5 09:10 obproxy-3.2.0-1.el7.x86_64.rpm

-rw-r--r-- 1 root root 47M Dec 31 17:57 oceanbase-ce-3.1.2-10000392021123010.el7.x86_64.rpm

-rw-r--r-- 1 root root 156K Dec 31 19:40 oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm

-rw-r--r-- 1 root root 72M Dec 18 06:25 prometheus-2.32.1.linux-amd64.tar.gz

-rw-r--r-- 1 root root 189M Jan 10 15:38 sbtest.sql

使用source命令导入数据

OB的环境参考：https://www.dbaup.com/oceanbase-dockeranzhuangtiyan.html

[admin@0209306f10bf tmp]$  obclient -h 127.1 -uroot@obmysql#obce-single -P2883 -p -c -A test
Enter password: 
Welcome to the OceanBase.  Commands end with ; or \g.
Your MySQL connection id is 10
Server version: 5.6.25 OceanBase 3.1.1 (r4-8c615943cbd25a6f7b8bdfd8677a13a21709a05e) (Built Oct 21 2021 10:52:05)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [test]> show databases;
+--------------------+
| Database           |
+--------------------+
| oceanbase          |
| information_schema |
| mysql              |
| test               |
| lhrdb              |
+--------------------+
5 rows in set (0.005 sec)

MySQL [test]> source /tmp/sbtest.sql
Query OK, 0 rows affected (0.001 sec)

ERROR 1064 (42000) at line 2 in file: '/tmp/sbtest.sql': You have an error in your SQL syntax; check the manual that corresponds to your OceanBase version for the right syntax to use near '--' at line 1
Query OK, 0 rows affected (0.001 sec)

Query OK, 5258 rows affected (0.478 sec)
Records: 5258  Duplicates: 0  Warnings: 0
.....
Query OK, 0 rows affected (0.001 sec)

Query OK, 5258 rows affected (0.456 sec)
Records: 5258  Duplicates: 0  Warnings: 0

Query OK, 5258 rows affected (0.478 sec)
Records: 5258  Duplicates: 0  Warnings: 0

MySQL [sbtest]> 
MySQL [sbtest]> 
MySQL [sbtest]> show databases;
+--------------------+
| Database           |
+--------------------+
| oceanbase          |
| information_schema |
| mysql              |
| test               |
| lhrdb              |
| sbtest             |
+--------------------+
6 rows in set (0.006 sec)

MySQL [sbtest]> use sbtest;
Database changed
MySQL [sbtest]> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest10         |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
| sbtest6          |
| sbtest7          |
| sbtest8          |
| sbtest9          |
+------------------+
10 rows in set (0.004 sec)

MySQL [sbtest]> select count(1) from sbtest1;
+----------+
| count(1) |
+----------+
|   100000 |
+----------+
1 row in set (0.375 sec)

[admin@0209306f10bf tmp]$ obclient -h 127.1 -uroot@obmysql#obce-single -P2883 -p -c -A test

Enter password:

Welcome to the OceanBase. Commands end with ; or \g.

Your MySQL connection id is 10

Server version: 5.6.25 OceanBase 3.1.1 (r4-8c615943cbd25a6f7b8bdfd8677a13a21709a05e) (Built Oct 21 2021 10:52:05)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [test]> show databases;

+--------------------+

| Database |

+--------------------+

| oceanbase |

| information_schema |

| mysql |

| test |

| lhrdb |

+--------------------+

5 rows in set (0.005 sec)

MySQL [test]> source /tmp/sbtest.sql

Query OK, 0 rows affected (0.001 sec)

ERROR 1064 (42000) at line 2 in file: '/tmp/sbtest.sql': You have an error in your SQL syntax; check the manual that corresponds to your OceanBase version for the right syntax to use near '--' at line 1

Query OK, 0 rows affected (0.001 sec)

Query OK, 5258 rows affected (0.478 sec)

Records: 5258 Duplicates: 0 Warnings: 0

.....

Query OK, 0 rows affected (0.001 sec)

Query OK, 5258 rows affected (0.456 sec)

Records: 5258 Duplicates: 0 Warnings: 0

Query OK, 5258 rows affected (0.478 sec)

Records: 5258 Duplicates: 0 Warnings: 0

MySQL [sbtest]>

MySQL [sbtest]> show databases;

+--------------------+

| Database |

+--------------------+

| oceanbase |

| information_schema |

| mysql |

| test |

| lhrdb |

| sbtest |

+--------------------+

6 rows in set (0.006 sec)

MySQL [sbtest]> use sbtest;

Database changed

MySQL [sbtest]> show tables;

+------------------+

| Tables_in_sbtest |

+------------------+

| sbtest1 |

| sbtest10 |

| sbtest2 |

| sbtest3 |

| sbtest4 |

| sbtest5 |

| sbtest6 |

| sbtest7 |

| sbtest8 |

| sbtest9 |

+------------------+

10 rows in set (0.004 sec)

MySQL [sbtest]> select count(1) from sbtest1;

+----------+

| count(1) |

+----------+

| 100000 |

+----------+

1 row in set (0.375 sec)

使用datax (离线)从 MySQL 同步表数据到 OceanBase

使用 datax 配置至少一个表的 MySQL 到 OceanBase 的 MySQL 租户的离线同步。

部署DATAX软件

有关dataX更多内容请参考：https://www.dbaup.com/alikaiyuanetlgongjuzhidataxhedatax-webjieshao.html

-- 大小：820MB
wget http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz

tar -xf datax.tar.gz -C /usr/local/
cd /usr/local/datax

# 删除datax中的隐藏文件
find /usr/local/datax/plugin -name ".*" | xargs rm -f

yum install -y python2 java

-- 生成模板文件
python2 /usr/local/datax/bin/datax.py -r mysqlreader -w oceanbasev10writer > /usr/local/datax/job/my2ob.json

# 根据实际环境修改
vi /usr/local/datax/job/my2ob.json

-- 大小：820MB

wget http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz

tar -xf datax.tar.gz -C /usr/local/

cd /usr/local/datax

# 删除datax中的隐藏文件

find /usr/local/datax/plugin -name ".*" | xargs rm -f

yum install -y python2 java

-- 生成模板文件

python2 /usr/local/datax/bin/datax.py -r mysqlreader -w oceanbasev10writer > /usr/local/datax/job/my2ob.json

# 根据实际环境修改

vi /usr/local/datax/job/my2ob.json

修改同步作业文件

cat > /usr/local/datax/job/my2ob.json <<"EOF"
{    
"job": {
        "setting": {
            "speed": {
                "channel": 2,
            },
            "errorLimit": {
                "record": 10
            }
        },
        "content": [
            {
                "reader": {
                    "name": "mysqlreader",
                    "parameter": {
                        "username": "root",
                        "password": "lhr",
                        "splitPk": "id",
                        "column": ["*"],
                        "connection": [
                            {
                                "jdbcUrl": [
                                    "jdbc:mysql://172.17.0.3:3306/sbtest"
                                ],
                                "table": [
                                    "sbtest1"
                                ]
                            }
                        ]
                    }
                },
                "writer": {
                    "name": "oceanbasev10writer",
                    "parameter": {
                        "writeMode": "insert",
                        "username": "root",
                        "password": "lhr",
                        "writerThreadCount": 5,
                        "column": [
                            "*"
                        ],
                        "connection": [
                           {
                                "jdbcUrl": "||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8",
                                "table": [
                                    "sbtest1"
                                ]
                            }
                        ]
                    }
                }
            }
        ]
    }
}
EOF

cat > /usr/local/datax/job/my2ob.json <<"EOF"

{

"job": {

"setting": {

"speed": {

"channel": 2,

"errorLimit": {

"record": 10

}

"content": [

{

"reader": {

"name": "mysqlreader",

"parameter": {

"username": "root",

"password": "lhr",

"splitPk": "id",

"column": ["*"],

"connection": [

{

"jdbcUrl": [

"jdbc:mysql://172.17.0.3:3306/sbtest"

"table": [

"sbtest1"

]

}

]

}

"writer": {

"name": "oceanbasev10writer",

"parameter": {

"writeMode": "insert",

"username": "root",

"password": "lhr",

"writerThreadCount": 5,

"column": [

"*"

"connection": [

{

"jdbcUrl": "||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8",

"table": [

"sbtest1"

]

}

]

}

]

}

EOF

启动同步作业

python2 /usr/local/datax/bin/datax.py /usr/local/datax/job/my2ob.json

1	python2 /usr/local/datax/bin/datax.py /usr/local/datax/job/my2ob.json

日志：

[admin@0209306f10bf ~]$ python2 /usr/local/datax/bin/datax.py /usr/local/datax/job/my2ob.json

DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.

2022-01-10 09:04:59.884 [main] INFO  VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2022-01-10 09:04:59.898 [main] INFO  Engine - the machine info  => 

        osInfo: Red Hat, Inc. 1.8 25.312-b07
        jvmInfo:        Linux amd64 3.10.0-1127.10.1.el7.x86_64
        cpu num:        16

        totalPhysicalMemory:    -0.00G
        freePhysicalMemory:     -0.00G
        maxFileDescriptorCount: -1
        currentOpenFileDescriptorCount: -1

        GC Names        [PS MarkSweep, PS Scavenge]

        MEMORY_NAME                    | allocation_size                | init_size                      
        PS Eden Space                  | 256.00MB                       | 256.00MB                       
        Code Cache                     | 240.00MB                       | 2.44MB                         
        Compressed Class Space         | 1,024.00MB                     | 0.00MB                         
        PS Survivor Space              | 42.50MB                        | 42.50MB                        
        PS Old Gen                     | 683.00MB                       | 683.00MB                       
        Metaspace                      | -0.00MB                        | 0.00MB                         

2022-01-10 09:04:59.935 [main] INFO  Engine - 
{
        "content":[
                {
                        "reader":{
                                "name":"mysqlreader",
                                "parameter":{
                                        "column":[
                                                "*"
                                        ],
                                        "connection":[
                                                {
                                                        "jdbcUrl":[
                                                                "jdbc:mysql://172.17.0.3:3306/sbtest"
                                                        ],
                                                        "table":[
                                                                "sbtest1"
                                                        ]
                                                }
                                        ],
                                        "password":"***",
                                        "splitPk":"id",
                                        "username":"root"
                                }
                        },
                        "writer":{
                                "name":"oceanbasev10writer",
                                "parameter":{
                                        "column":[
                                                "*"
                                        ],
                                        "connection":[
                                                {
                                                        "jdbcUrl":"||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8",
                                                        "table":[
                                                                "sbtest1"
                                                        ]
                                                }
                                        ],
                                        "password":"***",
                                        "username":"root",
                                        "writeMode":"insert",
                                        "writerThreadCount":5
                                }
                        }
                }
        ],
        "setting":{
                "errorLimit":{
                        "record":10
                },
                "speed":{
                        "channel":2
                }
        }
}

2022-01-10 09:04:59.963 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2022-01-10 09:04:59.969 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2022-01-10 09:04:59.994 [main] INFO  JobContainer - DataX jobContainer starts job.
2022-01-10 09:04:59.999 [main] INFO  JobContainer - Set jobId = 0
2022-01-10 09:05:00.632 [job-0] INFO  OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2022-01-10 09:05:00.634 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2022-01-10 09:05:00.659 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:00.686 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8
2022-01-10 09:05:01.073 [job-0] INFO  DbUtils - value for query [SHOW VARIABLES LIKE 'ob_compatibility_mode'] is [MYSQL]
2022-01-10 09:05:01.086 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:01.117 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:01.136 [job-0] INFO  OriginalConfPretreatmentUtil - table:[sbtest1] all columns:[
id,k,c,pad
].
2022-01-10 09:05:01.164 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2022-01-10 09:05:01.167 [job-0] INFO  OriginalConfPretreatmentUtil - Write data [
insert INTO %s (id,k,c,pad) VALUES(?,?,?,?)
], which jdbcUrl like:[||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]
2022-01-10 09:05:01.168 [job-0] INFO  JobContainer - jobContainer starts to do prepare ...
2022-01-10 09:05:01.169 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2022-01-10 09:05:01.169 [job-0] INFO  JobContainer - DataX Writer.Job [oceanbasev10writer] do prepare work .
2022-01-10 09:05:01.171 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:01.171 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:01.191 [job-0] INFO  DbUtils - value for query [show variables like 'version'] is [3.1.1]
2022-01-10 09:05:01.206 [job-0] INFO  JobContainer - jobContainer starts to do split ...
2022-01-10 09:05:01.206 [job-0] INFO  JobContainer - Job set Channel-Number to 2 channels.
2022-01-10 09:05:01.226 [job-0] INFO  SingleTableSplitUtil - split pk [sql=SELECT MIN(id),MAX(id) FROM sbtest1] is running... 
2022-01-10 09:05:01.256 [job-0] INFO  SingleTableSplitUtil - After split(), allQuerySql=[
select * from sbtest1  where  (1 <= id AND id < 10001) 
select * from sbtest1  where  (10001 <= id AND id < 20001) 
select * from sbtest1  where  (20001 <= id AND id < 30001) 
select * from sbtest1  where  (30001 <= id AND id < 40001) 
select * from sbtest1  where  (40001 <= id AND id < 50001) 
select * from sbtest1  where  (50001 <= id AND id < 60001) 
select * from sbtest1  where  (60001 <= id AND id < 70001) 
select * from sbtest1  where  (70001 <= id AND id < 80001) 
select * from sbtest1  where  (80001 <= id AND id < 90001) 
select * from sbtest1  where  (90001 <= id AND id <= 100000) 
select * from sbtest1  where  id IS NULL
].
2022-01-10 09:05:01.256 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] splits to [11] tasks.
2022-01-10 09:05:01.260 [job-0] INFO  JobContainer - DataX Writer.Job [oceanbasev10writer] splits to [11] tasks.
2022-01-10 09:05:01.300 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2022-01-10 09:05:01.315 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2022-01-10 09:05:01.320 [job-0] INFO  JobContainer - Running by standalone Mode.
2022-01-10 09:05:01.337 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [2] channels for [11] tasks.
2022-01-10 09:05:01.358 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2022-01-10 09:05:01.358 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2022-01-10 09:05:01.373 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2022-01-10 09:05:01.378 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started
2022-01-10 09:05:01.379 [0-0-0-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:01.379 [0-0-1-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:01.380 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (1 <= id AND id < 10001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:01.380 [0-0-1-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (10001 <= id AND id < 20001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:01.382 [0-0-1-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:01.382 [0-0-0-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:01.403 [0-0-1-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:01.431 [0-0-0-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:01.449 [0-0-1-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:01.455 [0-0-1-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:01.456 [0-0-1-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:01.464 [0-0-0-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:01.464 [0-0-1-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:01.464 [0-0-1-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:01.469 [0-0-0-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:01.470 [0-0-0-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:01.482 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:01.482 [0-0-1-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:01.483 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:01.483 [0-0-1-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:01.500 [0-0-0-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:01.502 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:01.512 [0-0-1-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:01.527 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:01.543 [0-0-1-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:01.562 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:01.575 [0-0-1-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:01.589 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:01.615 [0-0-1-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:01.627 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:01.659 [0-0-1-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:01.659 [0-0-1-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:01.673 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:01.673 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:01.677 [0-0-1-writer] INFO  ColumnMetaCache - fetch columnMeta of table sbtest1 success
2022-01-10 09:05:01.773 [0-0-1-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:01.813 [0-0-0-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:02.043 [0-0-1-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (10001 <= id AND id < 20001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:02.131 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (1 <= id AND id < 10001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:02.186 [0-0-0-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:02.260 [0-0-1-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:02.781 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[1409]ms
2022-01-10 09:05:02.782 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[1] is successed, used[1405]ms
2022-01-10 09:05:02.784 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[2] attemptCount[1] is started
2022-01-10 09:05:02.784 [0-0-2-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:02.785 [0-0-2-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (20001 <= id AND id < 30001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:02.785 [0-0-2-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:02.786 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[3] attemptCount[1] is started
2022-01-10 09:05:02.812 [0-0-3-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:02.812 [0-0-3-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (30001 <= id AND id < 40001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:02.813 [0-0-3-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:02.820 [0-0-2-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:02.837 [0-0-3-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:02.845 [0-0-2-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:02.849 [0-0-2-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:02.850 [0-0-2-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:02.854 [0-0-3-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:02.854 [0-0-2-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:02.854 [0-0-2-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:02.857 [0-0-3-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:02.858 [0-0-3-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:02.863 [0-0-3-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:02.863 [0-0-2-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:02.863 [0-0-3-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:02.863 [0-0-2-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:02.874 [0-0-3-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:02.874 [0-0-3-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:02.884 [0-0-2-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:02.893 [0-0-3-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:02.917 [0-0-2-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:02.925 [0-0-3-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:02.933 [0-0-2-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:02.941 [0-0-3-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:02.949 [0-0-2-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:02.957 [0-0-3-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:02.972 [0-0-2-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:02.973 [0-0-2-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:02.980 [0-0-3-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:02.980 [0-0-3-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:03.000 [0-0-2-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:03.011 [0-0-3-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:03.091 [0-0-2-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (20001 <= id AND id < 30001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:03.099 [0-0-3-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (30001 <= id AND id < 40001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:03.117 [0-0-2-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:03.126 [0-0-3-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:03.589 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[2] is successed, used[805]ms
2022-01-10 09:05:03.592 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[4] attemptCount[1] is started
2022-01-10 09:05:03.592 [0-0-4-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:03.593 [0-0-4-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (40001 <= id AND id < 50001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:03.627 [0-0-4-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:03.639 [0-0-4-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:03.647 [0-0-4-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:03.651 [0-0-4-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:03.652 [0-0-4-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:03.653 [0-0-4-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:03.653 [0-0-4-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:03.661 [0-0-4-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:03.661 [0-0-4-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:03.669 [0-0-4-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:03.690 [0-0-4-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:03.699 [0-0-4-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:03.706 [0-0-4-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:03.725 [0-0-4-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:03.726 [0-0-4-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:03.726 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[3] is successed, used[940]ms
2022-01-10 09:05:03.729 [0-0-5-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:03.729 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[5] attemptCount[1] is started
2022-01-10 09:05:03.729 [0-0-5-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (50001 <= id AND id < 60001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:03.730 [0-0-5-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:03.753 [0-0-5-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:03.764 [0-0-4-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:03.768 [0-0-5-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:03.772 [0-0-5-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:03.772 [0-0-5-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:03.773 [0-0-5-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:03.774 [0-0-5-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:03.790 [0-0-5-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:03.790 [0-0-5-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:03.810 [0-0-5-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:03.839 [0-0-4-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (40001 <= id AND id < 50001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:03.848 [0-0-5-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:03.857 [0-0-5-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:03.866 [0-0-5-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:03.875 [0-0-5-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:03.876 [0-0-5-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:03.906 [0-0-5-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:04.034 [0-0-5-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (50001 <= id AND id < 60001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:04.051 [0-0-5-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:04.061 [0-0-4-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:04.531 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[4] is successed, used[939]ms
2022-01-10 09:05:04.531 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[5] is successed, used[803]ms
2022-01-10 09:05:04.533 [0-0-6-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:04.533 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[6] attemptCount[1] is started
2022-01-10 09:05:04.534 [0-0-6-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (60001 <= id AND id < 70001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:04.534 [0-0-6-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:04.537 [0-0-7-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:04.537 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[7] attemptCount[1] is started
2022-01-10 09:05:04.537 [0-0-7-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (70001 <= id AND id < 80001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:04.537 [0-0-7-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:04.545 [0-0-6-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:04.554 [0-0-7-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:04.566 [0-0-6-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:04.571 [0-0-6-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:04.572 [0-0-6-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:04.576 [0-0-7-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:04.576 [0-0-6-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:04.576 [0-0-6-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:04.580 [0-0-7-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:04.580 [0-0-7-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:04.585 [0-0-7-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:04.585 [0-0-6-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:04.585 [0-0-6-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:04.585 [0-0-7-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:04.595 [0-0-7-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:04.595 [0-0-7-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:04.603 [0-0-6-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:04.612 [0-0-7-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:04.620 [0-0-6-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:04.628 [0-0-7-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:04.636 [0-0-6-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:04.643 [0-0-7-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:04.651 [0-0-6-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:04.659 [0-0-7-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:04.673 [0-0-6-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:04.673 [0-0-6-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:04.683 [0-0-7-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:04.683 [0-0-7-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:04.711 [0-0-6-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:04.719 [0-0-7-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:04.811 [0-0-6-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (60001 <= id AND id < 70001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:04.822 [0-0-7-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (70001 <= id AND id < 80001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:04.834 [0-0-6-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:04.846 [0-0-7-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:05.339 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[7] is successed, used[803]ms
2022-01-10 09:05:05.343 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[8] attemptCount[1] is started
2022-01-10 09:05:05.343 [0-0-8-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (80001 <= id AND id < 90001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:05.343 [0-0-8-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:05.344 [0-0-8-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:05.352 [0-0-8-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:05.360 [0-0-8-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:05.364 [0-0-8-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:05.365 [0-0-8-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:05.370 [0-0-8-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:05.370 [0-0-8-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:05.381 [0-0-8-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:05.381 [0-0-8-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:05.389 [0-0-8-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:05.398 [0-0-8-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:05.408 [0-0-8-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:05.416 [0-0-8-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:05.423 [0-0-8-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:05.423 [0-0-8-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:05.444 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[6] is successed, used[910]ms
2022-01-10 09:05:05.446 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[9] attemptCount[1] is started
2022-01-10 09:05:05.446 [0-0-9-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:05.446 [0-0-9-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  (90001 <= id AND id <= 100000) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:05.447 [0-0-9-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:05.452 [0-0-8-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:05.456 [0-0-9-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:05.502 [0-0-9-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:05.506 [0-0-9-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:05.506 [0-0-9-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:05.510 [0-0-9-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:05.510 [0-0-9-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:05.524 [0-0-9-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:05.524 [0-0-9-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:05.540 [0-0-9-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:05.552 [0-0-8-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (80001 <= id AND id < 90001) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:05.560 [0-0-9-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:05.568 [0-0-8-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:05.579 [0-0-9-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:05.601 [0-0-9-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:05.608 [0-0-9-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:05.608 [0-0-9-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:05.837 [0-0-9-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-01-10 09:05:06.462 [0-0-9-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  (90001 <= id AND id <= 100000) 
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:06.483 [0-0-9-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 5, finished = 0
2022-01-10 09:05:06.827 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[8] is successed, used[1485]ms
2022-01-10 09:05:06.831 [0-0-10-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-01-10 09:05:06.831 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[10] attemptCount[1] is started
2022-01-10 09:05:06.831 [0-0-10-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1  where  id IS NULL
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:06.832 [0-0-10-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-01-10 09:05:06.928 [0-0-10-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1  where  id IS NULL
] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-01-10 09:05:06.928 [0-0-10-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-01-10 09:05:07.015 [0-0-10-writer] INFO  CommonRdbmsWriter$Task - write mode: update
2022-01-10 09:05:07.038 [0-0-10-writer] INFO  CommonRdbmsWriter$Task - Skip columns: ID,
2022-01-10 09:05:07.038 [0-0-10-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)
2022-01-10 09:05:07.039 [0-0-10-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:07.040 [0-0-10-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:07.139 [0-0-10-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-01-10 09:05:07.139 [0-0-10-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-01-10 09:05:07.216 [0-0-10-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-01-10 09:05:07.247 [0-0-10-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-01-10 09:05:07.317 [0-0-10-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-01-10 09:05:07.356 [0-0-10-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-01-10 09:05:07.366 [0-0-10-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-01-10 09:05:07.366 [0-0-10-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-01-10 09:05:07.375 [0-0-10-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 0, finished = 0
2022-01-10 09:05:07.469 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[10] is successed, used[639]ms
2022-01-10 09:05:07.573 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[9] is successed, used[2127]ms
2022-01-10 09:05:07.574 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] completed it's tasks.
2022-01-10 09:05:11.356 [job-0] INFO  StandAloneJobContainerCommunicator - Total 100000 records, 18788895 bytes | Speed 1.79MB/s, 10000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 3.153s |  All Task WaitReaderTime 0.719s | Percentage 100.00%
2022-01-10 09:05:11.356 [job-0] INFO  AbstractScheduler - Scheduler accomplished all tasks.
2022-01-10 09:05:11.356 [job-0] INFO  JobContainer - DataX Writer.Job [oceanbasev10writer] do post work.
2022-01-10 09:05:11.356 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do post work.
2022-01-10 09:05:11.356 [job-0] INFO  JobContainer - DataX jobId [0] completed successfully.
2022-01-10 09:05:11.357 [job-0] INFO  HookInvoker - No hook invoked, because base dir not exists or is a file: /usr/local/datax/hook
2022-01-10 09:05:11.358 [job-0] INFO  JobContainer - 
         [total cpu info] => 
                averageCpu                     | maxDeltaCpu                    | minDeltaCpu                    
                -1.00%                         | -1.00%                         | -1.00%

         [total gc info] => 
                 NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime     
                 PS MarkSweep         | 0                  | 0                  | 0                  | 0.000s             | 0.000s             | 0.000s             
                 PS Scavenge          | 3                  | 3                  | 3                  | 0.237s             | 0.237s             | 0.237s             

2022-01-10 09:05:11.358 [job-0] INFO  JobContainer - PerfTrace not enable!
2022-01-10 09:05:11.359 [job-0] INFO  StandAloneJobContainerCommunicator - Total 100000 records, 18788895 bytes | Speed 1.79MB/s, 10000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 3.153s |  All Task WaitReaderTime 0.719s | Percentage 100.00%
2022-01-10 09:05:11.360 [job-0] INFO  JobContainer - 
任务启动时刻                    : 2022-01-10 09:04:59
任务结束时刻                    : 2022-01-10 09:05:11
任务总计耗时                    :                 11s
任务平均流量                    :            1.79MB/s
记录写入速度                    :          10000rec/s
读出记录总数                    :              100000
读写失败总数                    :                   0

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

[admin@0209306f10bf ~]$ python2 /usr/local/datax/bin/datax.py /usr/local/datax/job/my2ob.json

DataX (DATAX-OPENSOURCE-3.0), From Alibaba !

2022-01-10 09:04:59.884 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl

2022-01-10 09:04:59.898 [main] INFO Engine - the machine info =>

osInfo: Red Hat, Inc. 1.8 25.312-b07

jvmInfo: Linux amd64 3.10.0-1127.10.1.el7.x86_64

cpu num: 16

totalPhysicalMemory: -0.00G

freePhysicalMemory: -0.00G

maxFileDescriptorCount: -1

currentOpenFileDescriptorCount: -1

GC Names [PS MarkSweep, PS Scavenge]

MEMORY_NAME | allocation_size | init_size

PS Eden Space | 256.00MB | 256.00MB

Code Cache | 240.00MB | 2.44MB

Compressed Class Space | 1,024.00MB | 0.00MB

PS Survivor Space | 42.50MB | 42.50MB

PS Old Gen | 683.00MB | 683.00MB

Metaspace | -0.00MB | 0.00MB

2022-01-10 09:04:59.935 [main] INFO Engine -

{

"content":[

{

"reader":{

"name":"mysqlreader",

"parameter":{

"column":[

"*"

"connection":[

{

"jdbcUrl":[

"jdbc:mysql://172.17.0.3:3306/sbtest"

"table":[

"sbtest1"

]

}

"password":"***",

"splitPk":"id",

"username":"root"

}

"writer":{

"name":"oceanbasev10writer",

"parameter":{

"column":[

"*"

"connection":[

{

"jdbcUrl":"||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8",

"table":[

"sbtest1"

]

}

"password":"***",

"username":"root",

"writeMode":"insert",

"writerThreadCount":5

}

"setting":{

"errorLimit":{

"record":10

"speed":{

"channel":2

}

2022-01-10 09:04:59.963 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null

2022-01-10 09:04:59.969 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0

2022-01-10 09:04:59.994 [main] INFO JobContainer - DataX jobContainer starts job.

2022-01-10 09:04:59.999 [main] INFO JobContainer - Set jobId = 0

2022-01-10 09:05:00.632 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.

2022-01-10 09:05:00.634 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.

2022-01-10 09:05:00.659 [job-0] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:00.686 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8

2022-01-10 09:05:01.073 [job-0] INFO DbUtils - value for query [SHOW VARIABLES LIKE 'ob_compatibility_mode'] is [MYSQL]

2022-01-10 09:05:01.086 [job-0] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:01.117 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:01.136 [job-0] INFO OriginalConfPretreatmentUtil - table:[sbtest1] all columns:[

id,k,c,pad

2022-01-10 09:05:01.164 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.

2022-01-10 09:05:01.167 [job-0] INFO OriginalConfPretreatmentUtil - Write data [

insert INTO %s (id,k,c,pad) VALUES(?,?,?,?)

], which jdbcUrl like:[||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]

2022-01-10 09:05:01.168 [job-0] INFO JobContainer - jobContainer starts to do prepare ...

2022-01-10 09:05:01.169 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work .

2022-01-10 09:05:01.169 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] do prepare work .

2022-01-10 09:05:01.171 [job-0] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:01.171 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:01.191 [job-0] INFO DbUtils - value for query [show variables like 'version'] is [3.1.1]

2022-01-10 09:05:01.206 [job-0] INFO JobContainer - jobContainer starts to do split ...

2022-01-10 09:05:01.206 [job-0] INFO JobContainer - Job set Channel-Number to 2 channels.

2022-01-10 09:05:01.226 [job-0] INFO SingleTableSplitUtil - split pk [sql=SELECT MIN(id),MAX(id) FROM sbtest1] is running...

2022-01-10 09:05:01.256 [job-0] INFO SingleTableSplitUtil - After split(), allQuerySql=[

select * from sbtest1 where (1 <= id AND id < 10001)

select * from sbtest1 where (10001 <= id AND id < 20001)

select * from sbtest1 where (20001 <= id AND id < 30001)

select * from sbtest1 where (30001 <= id AND id < 40001)

select * from sbtest1 where (40001 <= id AND id < 50001)

select * from sbtest1 where (50001 <= id AND id < 60001)

select * from sbtest1 where (60001 <= id AND id < 70001)

select * from sbtest1 where (70001 <= id AND id < 80001)

select * from sbtest1 where (80001 <= id AND id < 90001)

select * from sbtest1 where (90001 <= id AND id <= 100000)

select * from sbtest1 where id IS NULL

2022-01-10 09:05:01.256 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] splits to [11] tasks.

2022-01-10 09:05:01.260 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] splits to [11] tasks.

2022-01-10 09:05:01.300 [job-0] INFO JobContainer - jobContainer starts to do schedule ...

2022-01-10 09:05:01.315 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.

2022-01-10 09:05:01.320 [job-0] INFO JobContainer - Running by standalone Mode.

2022-01-10 09:05:01.337 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [2] channels for [11] tasks.

2022-01-10 09:05:01.358 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.

2022-01-10 09:05:01.358 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.

2022-01-10 09:05:01.373 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started

2022-01-10 09:05:01.378 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started

2022-01-10 09:05:01.379 [0-0-0-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:01.379 [0-0-1-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:01.380 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (1 <= id AND id < 10001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:01.380 [0-0-1-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (10001 <= id AND id < 20001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:01.382 [0-0-1-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:01.382 [0-0-0-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:01.403 [0-0-1-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:01.431 [0-0-0-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:01.449 [0-0-1-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:01.455 [0-0-1-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:01.456 [0-0-1-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:01.464 [0-0-0-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:01.464 [0-0-1-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:01.464 [0-0-1-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:01.469 [0-0-0-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:01.470 [0-0-0-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:01.482 [0-0-0-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:01.482 [0-0-1-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:01.483 [0-0-0-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:01.483 [0-0-1-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:01.500 [0-0-0-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:01.502 [0-0-0-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:01.512 [0-0-1-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:01.527 [0-0-0-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:01.543 [0-0-1-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:01.562 [0-0-0-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:01.575 [0-0-1-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:01.589 [0-0-0-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:01.615 [0-0-1-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:01.627 [0-0-0-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:01.659 [0-0-1-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:01.659 [0-0-1-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:01.673 [0-0-0-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:01.673 [0-0-0-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:01.677 [0-0-1-writer] INFO ColumnMetaCache - fetch columnMeta of table sbtest1 success

2022-01-10 09:05:01.773 [0-0-1-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:01.813 [0-0-0-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:02.043 [0-0-1-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (10001 <= id AND id < 20001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:02.131 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (1 <= id AND id < 10001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:02.186 [0-0-0-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:02.260 [0-0-1-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:02.781 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[1409]ms

2022-01-10 09:05:02.782 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] is successed, used[1405]ms

2022-01-10 09:05:02.784 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[2] attemptCount[1] is started

2022-01-10 09:05:02.784 [0-0-2-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:02.785 [0-0-2-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (20001 <= id AND id < 30001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:02.785 [0-0-2-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:02.786 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[3] attemptCount[1] is started

2022-01-10 09:05:02.812 [0-0-3-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:02.812 [0-0-3-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (30001 <= id AND id < 40001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:02.813 [0-0-3-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:02.820 [0-0-2-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:02.837 [0-0-3-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:02.845 [0-0-2-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:02.849 [0-0-2-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:02.850 [0-0-2-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:02.854 [0-0-3-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:02.854 [0-0-2-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:02.854 [0-0-2-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:02.857 [0-0-3-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:02.858 [0-0-3-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:02.863 [0-0-3-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:02.863 [0-0-2-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:02.863 [0-0-3-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:02.863 [0-0-2-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:02.874 [0-0-3-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:02.874 [0-0-3-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:02.884 [0-0-2-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:02.893 [0-0-3-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:02.917 [0-0-2-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:02.925 [0-0-3-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:02.933 [0-0-2-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:02.941 [0-0-3-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:02.949 [0-0-2-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:02.957 [0-0-3-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:02.972 [0-0-2-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:02.973 [0-0-2-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:02.980 [0-0-3-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:02.980 [0-0-3-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:03.000 [0-0-2-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:03.011 [0-0-3-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:03.091 [0-0-2-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (20001 <= id AND id < 30001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:03.099 [0-0-3-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (30001 <= id AND id < 40001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:03.117 [0-0-2-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:03.126 [0-0-3-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:03.589 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[2] is successed, used[805]ms

2022-01-10 09:05:03.592 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[4] attemptCount[1] is started

2022-01-10 09:05:03.592 [0-0-4-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:03.593 [0-0-4-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (40001 <= id AND id < 50001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:03.627 [0-0-4-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:03.639 [0-0-4-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:03.647 [0-0-4-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:03.651 [0-0-4-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:03.652 [0-0-4-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:03.653 [0-0-4-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:03.653 [0-0-4-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:03.661 [0-0-4-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:03.661 [0-0-4-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:03.669 [0-0-4-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:03.690 [0-0-4-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:03.699 [0-0-4-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:03.706 [0-0-4-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:03.725 [0-0-4-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:03.726 [0-0-4-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:03.726 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[3] is successed, used[940]ms

2022-01-10 09:05:03.729 [0-0-5-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:03.729 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[5] attemptCount[1] is started

2022-01-10 09:05:03.729 [0-0-5-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (50001 <= id AND id < 60001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:03.730 [0-0-5-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:03.753 [0-0-5-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:03.764 [0-0-4-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:03.768 [0-0-5-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:03.772 [0-0-5-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:03.772 [0-0-5-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:03.773 [0-0-5-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:03.774 [0-0-5-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:03.790 [0-0-5-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:03.790 [0-0-5-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:03.810 [0-0-5-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:03.839 [0-0-4-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (40001 <= id AND id < 50001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:03.848 [0-0-5-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:03.857 [0-0-5-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:03.866 [0-0-5-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:03.875 [0-0-5-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:03.876 [0-0-5-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:03.906 [0-0-5-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:04.034 [0-0-5-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (50001 <= id AND id < 60001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:04.051 [0-0-5-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:04.061 [0-0-4-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:04.531 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[4] is successed, used[939]ms

2022-01-10 09:05:04.531 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[5] is successed, used[803]ms

2022-01-10 09:05:04.533 [0-0-6-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:04.533 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[6] attemptCount[1] is started

2022-01-10 09:05:04.534 [0-0-6-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (60001 <= id AND id < 70001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:04.534 [0-0-6-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:04.537 [0-0-7-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:04.537 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[7] attemptCount[1] is started

2022-01-10 09:05:04.537 [0-0-7-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (70001 <= id AND id < 80001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:04.537 [0-0-7-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:04.545 [0-0-6-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:04.554 [0-0-7-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:04.566 [0-0-6-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:04.571 [0-0-6-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:04.572 [0-0-6-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:04.576 [0-0-7-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:04.576 [0-0-6-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:04.576 [0-0-6-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:04.580 [0-0-7-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:04.580 [0-0-7-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:04.585 [0-0-7-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:04.585 [0-0-6-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:04.585 [0-0-6-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:04.585 [0-0-7-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:04.595 [0-0-7-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:04.595 [0-0-7-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:04.603 [0-0-6-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:04.612 [0-0-7-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:04.620 [0-0-6-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:04.628 [0-0-7-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:04.636 [0-0-6-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:04.643 [0-0-7-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:04.651 [0-0-6-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:04.659 [0-0-7-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:04.673 [0-0-6-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:04.673 [0-0-6-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:04.683 [0-0-7-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:04.683 [0-0-7-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:04.711 [0-0-6-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:04.719 [0-0-7-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:04.811 [0-0-6-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (60001 <= id AND id < 70001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:04.822 [0-0-7-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (70001 <= id AND id < 80001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:04.834 [0-0-6-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:04.846 [0-0-7-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:05.339 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[7] is successed, used[803]ms

2022-01-10 09:05:05.343 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[8] attemptCount[1] is started

2022-01-10 09:05:05.343 [0-0-8-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (80001 <= id AND id < 90001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:05.343 [0-0-8-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:05.344 [0-0-8-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:05.352 [0-0-8-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:05.360 [0-0-8-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:05.364 [0-0-8-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:05.365 [0-0-8-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:05.370 [0-0-8-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:05.370 [0-0-8-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:05.381 [0-0-8-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:05.381 [0-0-8-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:05.389 [0-0-8-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:05.398 [0-0-8-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:05.408 [0-0-8-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:05.416 [0-0-8-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:05.423 [0-0-8-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:05.423 [0-0-8-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:05.444 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[6] is successed, used[910]ms

2022-01-10 09:05:05.446 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[9] attemptCount[1] is started

2022-01-10 09:05:05.446 [0-0-9-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:05.446 [0-0-9-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where (90001 <= id AND id <= 100000)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:05.447 [0-0-9-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:05.452 [0-0-8-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:05.456 [0-0-9-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:05.502 [0-0-9-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:05.506 [0-0-9-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:05.506 [0-0-9-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:05.510 [0-0-9-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:05.510 [0-0-9-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:05.524 [0-0-9-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:05.524 [0-0-9-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:05.540 [0-0-9-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:05.552 [0-0-8-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (80001 <= id AND id < 90001)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:05.560 [0-0-9-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:05.568 [0-0-8-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:05.579 [0-0-9-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:05.601 [0-0-9-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:05.608 [0-0-9-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:05.608 [0-0-9-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:05.837 [0-0-9-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false

2022-01-10 09:05:06.462 [0-0-9-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where (90001 <= id AND id <= 100000)

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:06.483 [0-0-9-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 5, finished = 0

2022-01-10 09:05:06.827 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[8] is successed, used[1485]ms

2022-01-10 09:05:06.831 [0-0-10-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask

2022-01-10 09:05:06.831 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[10] attemptCount[1] is started

2022-01-10 09:05:06.831 [0-0-10-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from sbtest1 where id IS NULL

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:06.832 [0-0-10-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.

2022-01-10 09:05:06.928 [0-0-10-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from sbtest1 where id IS NULL

] jdbcUrl:[jdbc:mysql://172.17.0.3:3306/sbtest?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].

2022-01-10 09:05:06.928 [0-0-10-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.

2022-01-10 09:05:07.015 [0-0-10-writer] INFO CommonRdbmsWriter$Task - write mode: update

2022-01-10 09:05:07.038 [0-0-10-writer] INFO CommonRdbmsWriter$Task - Skip columns: ID,

2022-01-10 09:05:07.038 [0-0-10-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO sbtest1 (id,k,c,pad) VALUES(?,?,?,?) ON DUPLICATE KEY UPDATE k=VALUES(k),c=VALUES(c),pad=VALUES(pad)

2022-01-10 09:05:07.039 [0-0-10-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:07.040 [0-0-10-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:07.139 [0-0-10-writer] ERROR ConcurrentTableWriterTask - partCalculator is null

2022-01-10 09:05:07.139 [0-0-10-writer] INFO ConcurrentTableWriterTask - start 1 insert task.

2022-01-10 09:05:07.216 [0-0-10-writer] INFO ConcurrentTableWriterTask - start 2 insert task.

2022-01-10 09:05:07.247 [0-0-10-writer] INFO ConcurrentTableWriterTask - start 3 insert task.

2022-01-10 09:05:07.317 [0-0-10-writer] INFO ConcurrentTableWriterTask - start 4 insert task.

2022-01-10 09:05:07.356 [0-0-10-writer] INFO ConcurrentTableWriterTask - start 5 insert task.

2022-01-10 09:05:07.366 [0-0-10-writer] INFO DBUtil - this is ob1_0 jdbc url.

2022-01-10 09:05:07.366 [0-0-10-writer] INFO DBUtil - this is ob1_0 jdbc url. user=obce-single:obmysql:root :url=jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true

2022-01-10 09:05:07.375 [0-0-10-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 0, finished = 0

2022-01-10 09:05:07.469 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[10] is successed, used[639]ms

2022-01-10 09:05:07.573 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[9] is successed, used[2127]ms

2022-01-10 09:05:07.574 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks.

2022-01-10 09:05:11.356 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.

2022-01-10 09:05:11.356 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] do post work.

2022-01-10 09:05:11.356 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do post work.

2022-01-10 09:05:11.356 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.

2022-01-10 09:05:11.357 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /usr/local/datax/hook

2022-01-10 09:05:11.358 [job-0] INFO JobContainer -

[total cpu info] =>

averageCpu | maxDeltaCpu | minDeltaCpu

-1.00% | -1.00% | -1.00%

[total gc info] =>

PS MarkSweep | 0 | 0 | 0 | 0.000s | 0.000s | 0.000s

PS Scavenge | 3 | 3 | 3 | 0.237s | 0.237s | 0.237s

2022-01-10 09:05:11.358 [job-0] INFO JobContainer - PerfTrace not enable!

2022-01-10 09:05:11.360 [job-0] INFO JobContainer -

任务启动时刻 : 2022-01-10 09:04:59

任务结束时刻 : 2022-01-10 09:05:11

任务总计耗时 : 11s

任务平均流量 : 1.79MB/s

记录写入速度 : 10000rec/s

读出记录总数 : 100000

读写失败总数 : 0

datax 实现 OceanBase 以 csv 格式的导入导出

OceanBase 以 csv 格式导出数据

生成 datax 配置 json 模板

python2 /usr/local/datax/bin/datax.py -r oceanbasev10reader -w txtfilewriter > /usr/local/datax/job/oceanbase2csv.json

1	python2 /usr/local/datax/bin/datax.py -r oceanbasev10reader -w txtfilewriter > /usr/local/datax/job/oceanbase2csv.json

修改 json 文件：

cat > /usr/local/datax/job/oceanbase2csv.json <<"EOF"
{
   "job": {
       "setting": {
           "speed": {
               "channel": 4
           },
           "errorLimit": {
               "record": 0,
               "percentage": 0.1
           }
       },
       "content": [
           {
               "reader": {
                   "name": "oceanbasev10reader",
                   "parameter": {
                       "column": ["*"],
                       "connection": [
                           {
                               "jdbcUrl": ["||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8"],
                                "table": ["sbtest1"]
                           }
                       ],
                       "password": "lhr",
                       "username": "root",
                       "where": ""
                   }
               },
               "writer": {
                   "name": "txtfilewriter",
                   "parameter": {
                       "dateFormat": "yyyy-MM-dd hh:mm:ss",
                       "fieldDelimiter": ",",
                       "fileName": "ob2csv",
                       "path": "/tmp/",
                       "writeMode": "truncate"
                   }
               }
           }
       ]
   }
}
EOF

cat > /usr/local/datax/job/oceanbase2csv.json <<"EOF"

{

"job": {

"setting": {

"speed": {

"channel": 4

"errorLimit": {

"record": 0,

"percentage": 0.1

}

"content": [

{

"reader": {

"name": "oceanbasev10reader",

"parameter": {

"column": ["*"],

"connection": [

{

"jdbcUrl": ["||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8"],

"table": ["sbtest1"]

}

"password": "lhr",

"username": "root",

"where": ""

}

"writer": {

"name": "txtfilewriter",

"parameter": {

"dateFormat": "yyyy-MM-dd hh:mm:ss",

"fieldDelimiter": ",",

"fileName": "ob2csv",

"path": "/tmp/",

"writeMode": "truncate"

}

]

}

EOF

执行任务：

python2 /usr/local/datax/bin/datax.py  /usr/local/datax/job/oceanbase2csv.json

1	python2 /usr/local/datax/bin/datax.py /usr/local/datax/job/oceanbase2csv.json

查看结果：

[admin@0209306f10bf tmp]$ ll
total 19M
drwxr-xr-x 2 admin admin 4.0K Jan 10 18:07 hsperfdata_admin
drwxr-xr-x 2 root  root  4.0K Jan 10 16:22 hsperfdata_root
-rw-rw-r-- 1 admin admin  19M Jan 10 18:07 ob2csv__c0bee24e_73cc_4aeb_904d_f337836e95f1
[admin@0209306f10bf tmp]$ more ob2csv__c0bee24e_73cc_4aeb_904d_f337836e95f1 | wc -l
100000
[admin@0209306f10bf tmp]$ head -n 2 ob2csv__c0bee24e_73cc_4aeb_904d_f337836e95f1 
1,50147,68487932199-96439406143-93774651418-41631865787-96406072701-20604855487-25459966574-28203206787-41238978918-19503783441,22195207048-70116052123-74140395089-76317954521-98694025897
2,50248,13241531885-45658403807-79170748828-69419634012-13605813761-77983377181-01582588137-21344716829-87370944992-02457486289,28733802923-10548894641-11867531929-71265603657-36546888392
[admin@0209306f10bf tmp]$

[admin@0209306f10bf tmp]$ ll

total 19M

drwxr-xr-x 2 admin admin 4.0K Jan 10 18:07 hsperfdata_admin

drwxr-xr-x 2 root root 4.0K Jan 10 16:22 hsperfdata_root

-rw-rw-r-- 1 admin admin 19M Jan 10 18:07 ob2csv__c0bee24e_73cc_4aeb_904d_f337836e95f1

[admin@0209306f10bf tmp]$ more ob2csv__c0bee24e_73cc_4aeb_904d_f337836e95f1 | wc -l

100000

[admin@0209306f10bf tmp]$ head -n 2 ob2csv__c0bee24e_73cc_4aeb_904d_f337836e95f1

1,50147,68487932199-96439406143-93774651418-41631865787-96406072701-20604855487-25459966574-28203206787-41238978918-19503783441,22195207048-70116052123-74140395089-76317954521-98694025897

2,50248,13241531885-45658403807-79170748828-69419634012-13605813761-77983377181-01582588137-21344716829-87370944992-02457486289,28733802923-10548894641-11867531929-71265603657-36546888392

[admin@0209306f10bf tmp]$

OceanBase 以 csv 格式导入数据

生成 datax 配置 json 模板

python2 /usr/local/datax/bin/datax.py -r oceanbasev10reader -w txtfilewriter > /usr/local/datax/job/csv2oceanbase.json

1	python2 /usr/local/datax/bin/datax.py -r oceanbasev10reader -w txtfilewriter > /usr/local/datax/job/csv2oceanbase.json

修改json文件：

cat > /usr/local/datax/job/csv2oceanbase.json <<"EOF"
{
   "job": {
       "setting": {
           "speed": {
               "channel": 4
           },
           "errorLimit": {
               "record": 0,
               "percentage": 0.1
           }
       },
       "content": [
           {
               "reader": {
                   "name": "txtfilereader",
                   "parameter": {
                       "path": ["/tmp/ob/"],
                       "fileName": "ob2csv",
                       "encoding": "UTF-8",
                       "column": ["*"],
                       "dateFormat": "yyyy-MM-dd hh:mm:ss" ,
                       "nullFormat": "\\N" ,
                       "fieldDelimiter": ","
                   }
               },
               "writer": {
                   "name": "oceanbasev10writer",
                   "parameter": {
                       "obWriteMode": "insert",
                       "column": ["*"],
                       "preSql": [
                           "truncate table sbtest11"
                       ],
                       "connection": [
                           {
                               "jdbcUrl": "||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8",
                                "table": [
                                   "sbtest11"
                               ]
                           }
                       ],
                       "username": "root",
                       "password": "lhr",
                       "writerThreadCount": 10,
                       "batchSize": 100,
                       "memstoreThreshold": "0.9"
                   }
               }
           }
       ]
   }
}
EOF

cat > /usr/local/datax/job/csv2oceanbase.json <<"EOF"

{

"job": {

"setting": {

"speed": {

"channel": 4

"errorLimit": {

"record": 0,

"percentage": 0.1

}

"content": [

{

"reader": {

"name": "txtfilereader",

"parameter": {

"path": ["/tmp/ob/"],

"fileName": "ob2csv",

"encoding": "UTF-8",

"column": ["*"],

"dateFormat": "yyyy-MM-dd hh:mm:ss" ,

"nullFormat": "\\N" ,

"fieldDelimiter": ","

}

"writer": {

"name": "oceanbasev10writer",

"parameter": {

"obWriteMode": "insert",

"column": ["*"],

"preSql": [

"truncate table sbtest11"

"connection": [

{

"jdbcUrl": "||_dsc_ob10_dsc_||obce-single:obmysql||_dsc_ob10_dsc_||jdbc:oceanbase://127.0.0.1:2883/sbtest?useUnicode=true&characterEncoding=utf-8",

"table": [

"sbtest11"

]

}

"username": "root",

"password": "lhr",

"writerThreadCount": 10,

"batchSize": 100,

"memstoreThreshold": "0.9"

}

]

}

EOF

执行导入操作：

MySQL [sbtest]> create table sbtest11 like sbtest1;
Query OK, 0 rows affected (0.58 sec)

MySQL [sbtest]> desc sbtest11;
+-------+-----------+------+-----+---------+----------------+
| Field | Type      | Null | Key | Default | Extra          |
+-------+-----------+------+-----+---------+----------------+
| id    | int(11)   | NO   | PRI | NULL    | auto_increment |
| k     | int(11)   | NO   | MUL | 0       |                |
| c     | char(120) | NO   |     |         |                |
| pad   | char(60)  | NO   |     |         |                |
+-------+-----------+------+-----+---------+----------------+
4 rows in set (0.06 sec)

MySQL [sbtest]> select count(*) from sbtest11;
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.05 sec)

-- 导入
python2 /usr/local/datax/bin/datax.py  /usr/local/datax/job/csv2oceanbase.json

MySQL [sbtest]> create table sbtest11 like sbtest1;

Query OK, 0 rows affected (0.58 sec)

MySQL [sbtest]> desc sbtest11;

+-------+-----------+------+-----+---------+----------------+

+-------+-----------+------+-----+---------+----------------+

| id | int(11) | NO | PRI | NULL | auto_increment |

| k | int(11) | NO | MUL | 0 | |

| c | char(120) | NO | | | |

| pad | char(60) | NO | | | |

+-------+-----------+------+-----+---------+----------------+

4 rows in set (0.06 sec)

MySQL [sbtest]> select count(*) from sbtest11;

+----------+

| count(*) |

+----------+

| 0 |

+----------+

1 row in set (0.05 sec)

-- 导入

python2 /usr/local/datax/bin/datax.py /usr/local/datax/job/csv2oceanbase.json

结果：

OB数据库查询：

MySQL [sbtest]> select count(*) from sbtest11;
+----------+
| count(*) |
+----------+
|   100000 |
+----------+
1 row in set (0.32 sec)

MySQL [sbtest]> select count(*) from sbtest11;

+----------+

| count(*) |

+----------+

| 100000 |

+----------+

1 row in set (0.32 sec)

注意：

Path：需要给定文件的路径，该路径下不要有其他文件，执行会报错。

本人提供Oracle(OCP、OCM)、MySQL(OCP)、PostgreSQL(PGCA、PGCE、PGCM)等数据库的培训和考证业务，私聊QQ646634621或微信dbaup66，谢谢！

后续精彩内容已被站长无情隐藏，请输入验证码解锁本文！

获取验证码：请先关注本站微信公众号，然后回复“验证码”，获取验证码。在微信里搜索“AiDBA”或者“dbaup6”或者微信扫描右侧二维码都可以关注本站微信公众号。

打赏赞(2)

标签：数据迁移 OceanBase

小麦苗

学习或考证，均可联系麦老师，请加微信db_bao或QQ646634621

发表回复取消回复

要发表评论，您必须先登录。

合迁移 MySQL 数据到 OceanBase 集群

说明

练习目的

练习条件

练习内容

使用 mysqldump 将 mysql的表结构和数据同步到 OceanBase 的MySQL 租户中

使用sysbench给mysql环境生成数据

使用mysqldump导出

使用source命令导入数据

使用datax (离线)从 MySQL 同步表数据到 OceanBase

部署DATAX软件

修改同步作业文件

启动同步作业

datax 实现 OceanBase 以 csv 格式的导入导出

OceanBase 以 csv 格式导出数据

OceanBase 以 csv 格式导入数据

相关文章

您可能还喜欢...

发表回复取消回复

网站公告

网站寄语

本站其它工具

搜索本网站

标签云☁

网站日历

网站归档

网站分类

2024 年 11 月
一	二	三	四	五	六	日
				1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30

合 迁移 MySQL 数据到 OceanBase 集群

说明

练习目的

练习条件

练习内容

使用 mysqldump 将 mysql的表结构和数据同步到 OceanBase 的MySQL 租户中

使用sysbench给mysql环境生成数据

使用mysqldump导出

使用source命令导入数据

使用datax (离线)从 MySQL 同步表数据到 OceanBase

部署DATAX软件

修改同步作业文件

启动同步作业

datax 实现 OceanBase 以 csv 格式的导入导出

OceanBase 以 csv 格式导出数据

OceanBase 以 csv 格式导入数据

相关文章

您可能还喜欢...

手工迁移数据库Oracle 11.2.0.4到新机器并升级到12.2.0.1版本

GreenPlum备份恢复工具之gpbackup和gprestore

OceanBase 表分组和复制表特性简介

发表回复 取消回复

网站公告

网站寄语

本站其它工具

搜索本网站

标签云☁

网站日历

网站归档

网站分类

合迁移 MySQL 数据到 OceanBase 集群

发表回复取消回复