原 使用gpbackup在欧拉openEuler 22.03上备份GP6时报错open no such file or directory 其实内部报错 rsync symbol lookup error rsync undefined symbol ZSTD_compressStream2
Tags： 原创故障处理openEulerZSTD_compressStream2
👉 本文共约552个字，系统预计阅读时间或需3分钟。
简介
使用gpbackup在欧拉openEuler 22.03上备份GP6时，会报错“exit status 1: 20241030:14:05:47 gpbackup_s3_plugin:gpadmin:openeuler22:2235183-[ERROR]:-open /tmp/20241030140546_s3_config.yaml: no such file or directory”
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | cat > /home/gpadmin/s3_config.yaml <<"EOF" executablepath: $GPHOME/bin/gpbackup_s3_plugin options: region: endpoint: http://192.16.7.162:9001 aws_access_key_id: 12345678 aws_secret_access_key: 12345678 bucket: test folder: test EOF gpbackup --dbname postgres --jobs=4 --plugin-config /home/gpadmin/s3_config.yaml [gpadmin@openeuler22 ~]$ gpbackup --dbname postgres --jobs=4 --plugin-config /home/gpadmin/s3_config.yaml 20241023:15:56:55 gpbackup:gpadmin:openeuler22:1671724-[INFO]:-gpbackup version = 1.30.7 20241023:15:56:55 gpbackup:gpadmin:openeuler22:1671724-[INFO]:-Greenplum Database Version = 6.28.0 build commit:dc1f64048a6b086d2a139a5b610ea2b04ec7b6c4 20241023:15:56:55 gpbackup:gpadmin:openeuler22:1671724-[INFO]:-Starting backup of database postgres 20241023:15:56:55 gpbackup:gpadmin:openeuler22:1671724-[INFO]:-Reading Plugin Config /home/gpadmin/s3_config.yaml 20241023:15:56:55 gpbackup:gpadmin:openeuler22:1671724-[CRITICAL]:-exit status 1: 20241023:15:56:55 gpbackup_s3_plugin:gpadmin:openeuler22:1671825-[ERROR]:-open /tmp/20241023155655_s3_config.yaml: no such file or directory 20241023:15:56:56 gpbackup:gpadmin:openeuler22:1671724-[INFO]:-Found neither /usr/local/greenplum-db-6.28.0/bin/gp_email_contacts.yaml nor /home/gpadmin/gp_email_contacts.yaml 20241023:15:56:56 gpbackup:gpadmin:openeuler22:1671724-[INFO]:-Email containing gpbackup report /opt/greenplum/data/master/gpseg-1/backups/20241023/20241023155655/gpbackup_20241023155655_report will not be sent 20241023:15:56:56 gpbackup:gpadmin:openeuler22:1671724-[ERROR]:-ERROR: Plugin failed to process /opt/greenplum/data/master/gpseg-1/backups/20241023/20241023155655/gpbackup_20241023155655_config.yaml. 20241023:15:56:56 gpbackup_s3_plugin:gpadmin:openeuler22:1671835-[ERROR]:-open /tmp/20241023155655_s3_config.yaml: no such file or directory 20241023:15:56:56 gpbackup:gpadmin:openeuler22:1671724-[INFO]:-Beginning cleanup 20241023:15:56:56 gpbackup:gpadmin:openeuler22:1671724-[INFO]:-Cleanup complete [gpadmin@openeuler22 ~]$ gpbackup --dbname postgres --jobs=4 --plugin-config /home/gpadmin/s3_config.yaml --debug 20241030:14:05:46 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Backup Command: [gpbackup --dbname postgres --jobs=4 --plugin-config /home/gpadmin/s3_config.yaml --debug] 20241030:14:05:46 gpbackup:gpadmin:openeuler22:2235079-[INFO]:-gpbackup version = 1.30.7 20241030:14:05:46 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Initializing 5 database connections 20241030:14:05:46 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Exported synchronized snapshot 000002ED-1 20241030:14:05:46 gpbackup:gpadmin:openeuler22:2235079-[INFO]:-Greenplum Database Version = 6.28.0 build commit:dc1f64048a6b086d2a139a5b610ea2b04ec7b6c4 20241030:14:05:46 gpbackup:gpadmin:openeuler22:2235079-[INFO]:-Starting backup of database postgres 20241030:14:05:46 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Validating Tables and Schemas exist in Database 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Creating backup directories 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[INFO]:-Reading Plugin Config /home/gpadmin/s3_config.yaml 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Plugin config path: /tmp/20241030140546_s3_config.yaml 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Getting database size 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Checking plugin api version on all hosts 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-source /usr/local/greenplum-db-6.28.0/greenplum_path.sh && /usr/local/greenplum-db-6.28.0/bin/gpbackup_s3_plugin plugin_api_version 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Checking plugin version on all hosts 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-source /usr/local/greenplum-db-6.28.0/greenplum_path.sh && /usr/local/greenplum-db-6.28.0/bin/gpbackup_s3_plugin --version 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Copying plugin config to all hosts 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Wrote 272 bytes to plugin config /tmp/20241030140546_s3_config.yaml_1730268347054519688_-1 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Wrote 272 bytes to plugin config /tmp/20241030140546_s3_config.yaml_1730268347054744380_0 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-Wrote 272 bytes to plugin config /tmp/20241030140546_s3_config.yaml_1730268347054811255_1 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-rsync -e ssh /tmp/20241030140546_s3_config.yaml_1730268347054811255_1 openeuler22:/tmp/20241030140546_s3_config.yaml; rm /tmp/20241030140546_s3_config.yaml_1730268347054811255_1 20241030:14:05:47 gpbackup:gpadmin:openeuler22:2235079-[CRITICAL]:-exit status 1: 20241030:14:05:47 gpbackup_s3_plugin:gpadmin:openeuler22:2235183-[ERROR]:-open /tmp/20241030140546_s3_config.yaml: no such file or directory github.com/greenplum-db/gpbackup/utils.(*PluginConfig).executeHook /tmp/build/16b62eb3/go/src/github.com/greenplum-db/gpbackup/utils/plugin.go:251 github.com/greenplum-db/gpbackup/utils.(*PluginConfig).SetupPluginForBackup /tmp/build/16b62eb3/go/src/github.com/greenplum-db/gpbackup/utils/plugin.go:216 github.com/greenplum-db/gpbackup/backup.DoSetup /tmp/build/16b62eb3/go/src/github.com/greenplum-db/gpbackup/backup/backup.go:104 main.main.func1 /tmp/build/16b62eb3/go/src/github.com/greenplum-db/gpbackup/gpbackup.go:22 github.com/spf13/cobra.(*Command).execute /tmp/build/16b62eb3/go/src/github.com/greenplum-db/gpbackup/vendor/github.com/spf13/cobra/command.go:920 github.com/spf13/cobra.(*Command).ExecuteC /tmp/build/16b62eb3/go/src/github.com/greenplum-db/gpbackup/vendor/github.com/spf13/cobra/command.go:1044 github.com/spf13/cobra.(*Command).Execute /tmp/build/16b62eb3/go/src/github.com/greenplum-db/gpbackup/vendor/github.com/spf13/cobra/command.go:968 main.main /tmp/build/16b62eb3/go/src/github.com/greenplum-db/gpbackup/gpbackup.go:27 runtime.main /usr/local/go/src/runtime/proc.go:267 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:1650 20241030:14:05:48 gpbackup:gpadmin:openeuler22:2235079-[INFO]:-Found neither /usr/local/greenplum-db-6.28.0/bin/gp_email_contacts.yaml nor /home/gpadmin/gp_email_contacts.yaml 20241030:14:05:48 gpbackup:gpadmin:openeuler22:2235079-[INFO]:-Email containing gpbackup report /opt/greenplum/data/master/gpseg-1/backups/20241030/20241030140546/gpbackup_20241030140546_report will not be sent 20241030:14:05:48 gpbackup:gpadmin:openeuler22:2235079-[DEBUG]:-/usr/local/greenplum-db-6.28.0/bin/gpbackup_s3_plugin backup_file /tmp/20241030140546_s3_config.yaml /opt/greenplum/data/master/gpseg-1/backups/20241030/20241030140546/gpbackup_20241030140546_config.yaml 20241030:14:05:48 gpbackup:gpadmin:openeuler22:2235079-[ERROR]:-ERROR: Plugin failed to process /opt/greenplum/data/master/gpseg-1/backups/20241030/20241030140546/gpbackup_20241030140546_config.yaml. 20241030:14:05:48 gpbackup_s3_plugin:gpadmin:openeuler22:2235193-[ERROR]:-open /tmp/20241030140546_s3_config.yaml: no such file or directory 20241030:14:05:48 gpbackup:gpadmin:openeuler22:2235079-[INFO]:-Beginning cleanup 20241030:14:05:48 gpbackup:gpadmin:openeuler22:2235079-[INFO]:-Cleanup complete |
分析
主要问题是文件/usr/local/greenplum-db/lib/libzstd.so.1版本低，替换为本机自带版本即可，解决办法：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | [gpadmin@openeuler22 ~]$ rsync -e ssh /tmp/20241030140546_s3_config.yaml_1730268347054811255_1 openeuler22:/tmp/20241030140546_s3_config.yaml; rsync: symbol lookup error: rsync: undefined symbol: ZSTD_compressStream2 [gpadmin@openeuler22 ~]$ [gpadmin@openeuler22 ~]$ ldd $(which rsync) | grep zstd libzstd.so.1 => /usr/local/greenplum-db-6.28.0/lib/libzstd.so.1 (0x00007f42836b0000) [gpadmin@openeuler22 ~]$ ln -sf /usr/lib64/libzstd.so.1.5.0 /usr/local/greenplum-db/lib/libzstd.so.1 [gpadmin@openeuler22 ~]$ gpbackup --dbname postgres --jobs=4 --plugin-config /home/gpadmin/s3_config.yaml 20241030:14:17:14 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-gpbackup version = 1.30.7 20241030:14:17:14 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Greenplum Database Version = 6.28.0 build commit:dc1f64048a6b086d2a139a5b610ea2b04ec7b6c4 20241030:14:17:14 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Starting backup of database postgres 20241030:14:17:14 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Reading Plugin Config /home/gpadmin/s3_config.yaml 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Backup Timestamp = 20241030141714 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Backup Database = postgres 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Gathering table state information 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Acquiring ACCESS SHARE locks on tables 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Gathering additional table metadata 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Getting partition definitions 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Getting storage information 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Getting child partitions with altered schema 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[WARNING]:-No tables in backup set contain data. Performing metadata-only backup instead. 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Metadata will be written to /opt/greenplum/data/master/gpseg-1/backups/20241030/20241030141714/gpbackup_20241030141714_metadata.sql 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Writing global database metadata 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Global database metadata backup complete 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Writing pre-data metadata 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Pre-data metadata metadata backup complete 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Writing post-data metadata 20241030:14:17:15 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Post-data metadata backup complete 20241030:14:17:16 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Found neither /usr/local/greenplum-db-6.28.0/bin/gp_email_contacts.yaml nor /home/gpadmin/gp_email_contacts.yaml 20241030:14:17:16 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Email containing gpbackup report /opt/greenplum/data/master/gpseg-1/backups/20241030/20241030141714/gpbackup_20241030141714_report will not be sent 20241030:14:17:17 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Beginning cleanup 20241030:14:17:17 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Cleanup complete 20241030:14:17:17 gpbackup:gpadmin:openeuler22:2251723-[INFO]:-Backup completed successfully [gpadmin@openeuler22 ~]$ |
解决
1 | ln -sf /usr/lib64/libzstd.so.1.5.0 /usr/local/greenplum-db/lib/libzstd.so.1 |
类似故障
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | [gpadmin@openeuler22 ~]$ cd /soft [gpadmin@openeuler22 soft]$ ll total 177460 -rw-r--r-- 1 root root 19401617 Sep 29 11:06 greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg -rw-r--r-- 1 root root 162312556 Aug 12 15:40 greenplum-db-6.28.0-rhel7-x86_64.rpm [gpadmin@openeuler22 soft]$ gppkg -i greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg 20241023:15:29:54:1668796 gppkg:openeuler22:gpadmin-[INFO]:-Starting gppkg with args: -i greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg 20241023:15:29:54:1668796 gppkg:openeuler22:gpadmin-[INFO]:-Installing package greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg 20241023:15:29:54:1668796 gppkg:openeuler22:gpadmin-[INFO]:-Validating rpm installation cmdStr='rpm --test -i /usr/local/greenplum-db-6.28.0/.tmp/gpbackup_tools_RHEL7-1.30.7-1.x86_64.rpm --dbpath /usr/local/greenplum-db-6.28.0/share/packages/database --prefix /usr/local/greenplum-db-6.28.0' 20241023:15:29:54:1668796 gppkg:openeuler22:gpadmin-[CRITICAL]:-Error occurred: non-zero rc: 127 Command was: 'rpm --test -i /usr/local/greenplum-db-6.28.0/.tmp/gpbackup_tools_RHEL7-1.30.7-1.x86_64.rpm --dbpath /usr/local/greenplum-db-6.28.0/share/packages/database --prefix /usr/local/greenplum-db-6.28.0' rc=127, stdout='', stderr='rpm: symbol lookup error: /usr/lib64/librpmio.so.9: undefined symbol: ZSTD_compressStream2 ' [gpadmin@openeuler22 soft]$ rpm --test -i /usr/local/greenplum-db-6.28.0/.tmp/gpbackup_tools_RHEL7-1.30.7-1.x86_64.rpm --dbpath /usr/local/greenplum-db-6.28.0/share/packages/database --prefix /usr/local/greenplum-db-6.28.0 rpm: symbol lookup error: /usr/lib64/librpmio.so.9: undefined symbol: ZSTD_compressStream2 [gpadmin@openeuler22 soft]$ rpm -qa | grep zstd rpm: symbol lookup error: /usr/lib64/librpmio.so.9: undefined symbol: ZSTD_compressStream2 [gpadmin@openeuler22 soft]$ export LD_LIBRARY_PATH='' [gpadmin@openeuler22 soft]$ rpm -qa | grep zstd zstd-1.5.0-4.oe2203.x86_64 zstd-devel-1.5.0-4.oe2203.x86_64 [gpadmin@openeuler22 soft]$ rpm --test -i /usr/local/greenplum-db-6.28.0/.tmp/gpbackup_tools_RHEL7-1.30.7-1.x86_64.rpm --dbpath /usr/local/greenplum-db-6.28.0/share/packages/database --prefix /usr/local/greenplum-db-6.28.0 [gpadmin@openeuler22 soft]$ [gpadmin@openeuler22 soft]$ [gpadmin@openeuler22 soft]$ gppkg -i greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg 20241023:15:33:05:1669006 gppkg:openeuler22:gpadmin-[INFO]:-Starting gppkg with args: -i greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg 20241023:15:33:05:1669006 gppkg:openeuler22:gpadmin-[INFO]:-Installing package greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg 20241023:15:33:05:1669006 gppkg:openeuler22:gpadmin-[INFO]:-Validating rpm installation cmdStr='rpm --test -i /usr/local/greenplum-db-6.28.0/.tmp/gpbackup_tools_RHEL7-1.30.7-1.x86_64.rpm --dbpath /usr/local/greenplum-db-6.28.0/share/packages/database --prefix /usr/local/greenplum-db-6.28.0' 20241023:15:33:05:1669006 gppkg:openeuler22:gpadmin-[INFO]:-Installing greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg locally 20241023:15:33:06:1669006 gppkg:openeuler22:gpadmin-[INFO]:-Validating rpm installation cmdStr='rpm --test -i /usr/local/greenplum-db-6.28.0/.tmp/gpbackup_tools_RHEL7-1.30.7-1.x86_64.rpm --dbpath /usr/local/greenplum-db-6.28.0/share/packages/database --prefix /usr/local/greenplum-db-6.28.0' 20241023:15:33:06:1669006 gppkg:openeuler22:gpadmin-[INFO]:-Installing rpms cmdStr='rpm -i --force /usr/local/greenplum-db-6.28.0/.tmp/gpbackup_tools_RHEL7-1.30.7-1.x86_64.rpm --dbpath /usr/local/greenplum-db-6.28.0/share/packages/database --prefix=/usr/local/greenplum-db-6.28.0' 20241023:15:33:08:1669006 gppkg:openeuler22:gpadmin-[INFO]:-Completed local installation of greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg. 20241023:15:33:08:1669006 gppkg:openeuler22:gpadmin-[INFO]:-gpbackup 1.30.7 successfully installed 20241023:15:33:08:1669006 gppkg:openeuler22:gpadmin-[INFO]:-greenplum_backup_restore-1.30.7-gp6-rhel7-x86_64.gppkg successfully installed. |
总结
1、类似问题可能在gpcc老版本会出现。
2、若是出现错误：undefined symbol: ZSTD_compressStream2 ， 则解决办法：
本人提供Oracle(OCP、OCM)、MySQL(OCP)、PostgreSQL(PGCA、PGCE、PGCM)等数据库的培训和考证业务，私聊QQ646634621或微信dbaup66，谢谢！
