合 GreenPlum删除了pg_xlog文件夹内容报错could not locate a valid checkpoint record
现象
直接删除GreenPlum下pg_xlog的目录文件内容:
1 2 | [gpadmin@gpdb6 pg_xlog]$ rm -rf /opt/greenplum/data/primary/gpseg1/pg_xlog/* [gpadmin@gpdb6 pg_xlog]$ rm -rf /opt/greenplum/data/primary/gpseg0/pg_xlog/* |
然后发现报错:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 | [gpadmin@gpdb6 pg_xlog]$ du -sh /opt/greenplum/data/primary/gpseg1/pg_xlog /opt/greenplum/data/primary/gpseg0/pg_xlog 0 /opt/greenplum/data/primary/gpseg1/pg_xlog 0 /opt/greenplum/data/primary/gpseg0/pg_xlog [gpadmin@gpdb6 pg_xlog]$ drop dropdb droplang dropuser [gpadmin@gpdb6 pg_xlog]$ dropdb db1 WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=505966) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=505967) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 1 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=505971) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=505972) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 2 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=505975) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=505976) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 3 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=505979) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=505980) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 4 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=505983) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=505984) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 5 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=505988) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=505989) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 6 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=505992) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=505993) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 7 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=505996) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=505997) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 8 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=506000) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=506001) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 9 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=506005) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=506004) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: the distributed transaction 'Commit Prepared' broadcast failed to one or more segments. Retrying ... try 10 DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared NOTICE: Releasing segworker group to retry broadcast. WARNING: primary failure, xlog record is invalid, failover requested (seg0 192.92.0.18:6000 pid=506009) HINT: run gprecoverseg to re-establish mirror connectivity WARNING: primary failure, xlog record is invalid, failover requested (seg1 192.92.0.18:6001 pid=506008) HINT: run gprecoverseg to re-establish mirror connectivity dropdb: database removal failed: PANIC: unable to complete 'Commit Prepared' broadcast (cdbtm.c:661) DETAIL: gid=1704254598-0000003761, state=Retry Commit Prepared server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. [gpadmin@gpdb6 pg_xlog]$ gpstart -a 20240103:12:06:44:506252 gpstart:gpdb6:gpadmin-[INFO]:-Starting gpstart with args: -a 20240103:12:06:44:506252 gpstart:gpdb6:gpadmin-[INFO]:-Gathering information and validating the environment... 20240103:12:06:44:506252 gpstart:gpdb6:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.23.1 build commit:2731a45ecb364317207c560730cf9e2cbf17d7e4 Open Source' 20240103:12:06:44:506252 gpstart:gpdb6:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232' 20240103:12:06:44:506252 gpstart:gpdb6:gpadmin-[WARNING]:-postmaster.pid file exists on Master, checking if recovery startup required 20240103:12:06:44:506252 gpstart:gpdb6:gpadmin-[INFO]:-Commencing recovery startup checks 20240103:12:06:44:506252 gpstart:gpdb6:gpadmin-[INFO]:-Have lock file /tmp/.s.PGSQL.5432 and a process running on port 5432 20240103:12:06:44:506252 gpstart:gpdb6:gpadmin-[ERROR]:-gpstart error: Master instance process running [gpadmin@gpdb6 pg_xlog]$ gpstop -M fast -ar 20240103:12:06:50:506571 gpstop:gpdb6:gpadmin-[INFO]:-Starting gpstop with args: -M fast -ar 20240103:12:06:50:506571 gpstop:gpdb6:gpadmin-[INFO]:-Gathering information and validating the environment... 20240103:12:06:50:506571 gpstop:gpdb6:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information 20240103:12:06:50:506571 gpstop:gpdb6:gpadmin-[INFO]:-Obtaining Segment details from master... 20240103:12:06:50:506571 gpstop:gpdb6:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.23.1 build commit:2731a45ecb364317207c560730cf9e2cbf17d7e4 Open Source' 20240103:12:06:50:506571 gpstop:gpdb6:gpadmin-[INFO]:-Commencing Master instance shutdown with mode='fast' 20240103:12:06:50:506571 gpstop:gpdb6:gpadmin-[INFO]:-Master segment instance directory=/opt/greenplum/data/master/gpseg-1 20240103:12:06:50:506571 gpstop:gpdb6:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process 20240103:12:06:50:506571 gpstop:gpdb6:gpadmin-[INFO]:-Terminating processes for segment /opt/greenplum/data/master/gpseg-1 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-Stopping master standby host gpdb6 mode=fast 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-Successfully shutdown standby process on gpdb6 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-Targeting dbid [2, 3] for shutdown 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-Commencing parallel segment instance shutdown, please wait... 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-0.00% of jobs completed 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-100.00% of jobs completed 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:- Segments stopped successfully = 2 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:- Segments with errors during stop = 0 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-Successfully shutdown 2 of 2 segment instances 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-Database successfully shutdown with no errors reported 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-Cleaning up leftover gpmmon process 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-No leftover gpmmon process found 20240103:12:06:51:506571 gpstop:gpdb6:gpadmin-[INFO]:-Cleaning up leftover gpsmon processes 20240103:12:06:52:506571 gpstop:gpdb6:gpadmin-[INFO]:-No leftover gpsmon processes on some hosts. not attempting forceful termination on these hosts 20240103:12:06:52:506571 gpstop:gpdb6:gpadmin-[INFO]:-Cleaning up leftover shared memory 20240103:12:06:52:506571 gpstop:gpdb6:gpadmin-[INFO]:-Restarting System... 20240103:12:06:54:506571 gpstop:gpdb6:gpadmin-[CRITICAL]:-Error occurred: non-zero rc: 2 Command was: '$GPHOME/bin/gpstart -a -d /opt/greenplum/data/master/gpseg-1' rc=2, stdout='20240103:12:06:52:506964 gpstart:gpdb6:gpadmin-[INFO]:-Starting gpstart with args: -a -d /opt/greenplum/data/master/gpseg-1 20240103:12:06:52:506964 gpstart:gpdb6:gpadmin-[INFO]:-Gathering information and validating the environment... 20240103:12:06:52:506964 gpstart:gpdb6:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.23.1 build commit:2731a45ecb364317207c560730cf9e2cbf17d7e4 Open Source' 20240103:12:06:52:506964 gpstart:gpdb6:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232' 20240103:12:06:52:506964 gpstart:gpdb6:gpadmin-[INFO]:-Starting Master instance in admin mode 20240103:12:06:52:506964 gpstart:gpdb6:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information 20240103:12:06:52:506964 gpstart:gpdb6:gpadmin-[INFO]:-Obtaining Segment details from master... 20240103:12:06:52:506964 gpstart:gpdb6:gpadmin-[INFO]:-Setting new master era 20240103:12:06:52:506964 gpstart:gpdb6:gpadmin-[INFO]:-Master Started... 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:-Shutting down master 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait... 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:-Process results... 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[ERROR]:-No segment started for content: 0. 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:-dumping success segments: [] 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:-DBID:3 FAILED host:'gpdb6' datadir:'/opt/greenplum/data/primary/gpseg1' with reason:'PG_CTL failed.' 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:-DBID:2 FAILED host:'gpdb6' datadir:'/opt/greenplum/data/primary/gpseg0' with reason:'PG_CTL failed.' 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:- Successful segment starts = 0 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[WARNING]:-Failed segment starts = 2 <<<<<<<< 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:- Skipped segment starts (segments are marked down in configuration) = 0 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:-Successfully started 0 of 2 segment instances <<<<<<<< 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[WARNING]:-Segment instance startup failures reported 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[WARNING]:-Failed start 2 of 2 segment instances <<<<<<<< 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[WARNING]:-Review /home/gpadmin/gpAdminLogs/gpstart_20240103.log 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:06:53:506964 gpstart:gpdb6:gpadmin-[INFO]:-Commencing parallel segment instance shutdown, please wait... 20240103:12:06:54:506964 gpstart:gpdb6:gpadmin-[ERROR]:-gpstart error: Do not have enough valid segments to start the array. ', stderr='' |
排查
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | [gpadmin@gpdb6 pg_log]$ gpstart -a 20240103:12:15:39:507603 gpstart:gpdb6:gpadmin-[INFO]:-Starting gpstart with args: -a 20240103:12:15:39:507603 gpstart:gpdb6:gpadmin-[INFO]:-Gathering information and validating the environment... 20240103:12:15:39:507603 gpstart:gpdb6:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.23.1 build commit:2731a45ecb364317207c560730cf9e2cbf17d7e4 Open Source' 20240103:12:15:39:507603 gpstart:gpdb6:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232' 20240103:12:15:40:507603 gpstart:gpdb6:gpadmin-[INFO]:-Starting Master instance in admin mode 20240103:12:15:40:507603 gpstart:gpdb6:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information 20240103:12:15:40:507603 gpstart:gpdb6:gpadmin-[INFO]:-Obtaining Segment details from master... 20240103:12:15:40:507603 gpstart:gpdb6:gpadmin-[INFO]:-Setting new master era 20240103:12:15:40:507603 gpstart:gpdb6:gpadmin-[INFO]:-Master Started... 20240103:12:15:40:507603 gpstart:gpdb6:gpadmin-[INFO]:-Shutting down master 20240103:12:15:41:507603 gpstart:gpdb6:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait... . 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:-Process results... 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[ERROR]:-No segment started for content: 0. 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:-dumping success segments: [] 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:-DBID:3 FAILED host:'gpdb6' datadir:'/opt/greenplum/data/primary/gpseg1' with reason:'PG_CTL failed.' 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:-DBID:2 FAILED host:'gpdb6' datadir:'/opt/greenplum/data/primary/gpseg0' with reason:'PG_CTL failed.' 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:- Successful segment starts = 0 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[WARNING]:-Failed segment starts = 2 <<<<<<<< 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:- Skipped segment starts (segments are marked down in configuration) = 0 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:-Successfully started 0 of 2 segment instances <<<<<<<< 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[WARNING]:-Segment instance startup failures reported 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[WARNING]:-Failed start 2 of 2 segment instances <<<<<<<< 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[WARNING]:-Review /home/gpadmin/gpAdminLogs/gpstart_20240103.log 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:----------------------------------------------------- 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[INFO]:-Commencing parallel segment instance shutdown, please wait... 20240103:12:15:42:507603 gpstart:gpdb6:gpadmin-[ERROR]:-gpstart error: Do not have enough valid segments to start the array. [gpadmin@gpdb6 pg_log]$ ll total 8 -rw------- 1 gpadmin gpadmin 1975 Jan 3 12:15 gpdb-2024-01-03_121542.csv -rw------- 1 gpadmin gpadmin 1020 Jan 3 12:15 startup.log [gpadmin@gpdb6 pg_log]$ more startup.log 2024-01-03 12:15:41.700932 CST,,,p507749,th-1510483840,,,,0,,,seg0,,,,,"LOG","00000","Metrics collector: query info collect hook initialized",,,,,,,,"_PG_init","metrics_collector_gp.c",145, 2024-01-03 12:15:41.701054 CST,,,p507749,th-1510483840,,,,0,,,seg0,,,,,"LOG","00000","Metrics collector: utility query info collect hook initialized",,,,,,,,"_PG_init","metrics_collector_gp.c",149, 2024-01-03 12:15:41.701107 CST,,,p507749,th-1510483840,,,,0,,,seg0,,,,,"LOG","00000","Metrics collector: objatin query access tables hook initialized",,,,,,,,"_PG_init","metrics_collector_gp.c",154, 2024-01-03 12:15:41.709050 CST,,,p507749,th-1510483840,,,,0,,,seg0,,,,,"WARNING","01000","Better set max_prepared_transactions greater than max_connections",,,,,,,,"tmShmemInit","cdbtm.c",1100, 2024-01-03 12:15:42.325375 CST,,,p507749,th-1510483840,,,,0,,,seg0,,,,,"LOG","00000","redirecting log output to logging collector process",,"Future log output will appear in directory ""pg_log"".",,,,,,"SysLogger_Start","syslogger.c",999, [gpadmin@gpdb6 pg_log]$ ll total 8 -rw------- 1 gpadmin gpadmin 1975 Jan 3 12:15 gpdb-2024-01-03_121542.csv -rw------- 1 gpadmin gpadmin 1020 Jan 3 12:15 startup.log [gpadmin@gpdb6 pg_log]$ more gpdb-2024-01-03_121542.csv 2024-01-03 12:15:42.343260 CST,,,p507755,th-1510483840,,,,0,,,seg0,,,,,"LOG","00000","database system was shut down at 2024-01-03 12:06:51 CST",,,,,,,0,,"xlog.c",6476, 2024-01-03 12:15:42.398648 CST,,,p507755,th-1510483840,,,,0,,,seg0,,,,,"LOG","00000","invalid primary checkpoint record at location 3/179D87B8",,,,,,,0,,"xlog.c",8249, 2024-01-03 12:15:42.398686 CST,,,p507755,th-1510483840,,,,0,,,seg0,,,,,"LOG","00000","invalid secondary checkpoint record at location 3/179D8538",,,,,,,0,,"xlog.c",8254, 2024-01-03 12:15:42.402639 CST,"gpadmin","postgres",p507758,th-1510483840,"[local]",,2024-01-03 12:15:42 CST,0,,,seg0,,,,,"FATAL","57P03","the database system is starting up","last replayed record at 0/0",,,,,,0,,"postmaster.c",2570, 2024-01-03 12:15:42.505843 CST,"gpadmin","postgres",p507760,th-1510483840,"[local]",,2024-01-03 12:15:42 CST,0,,,seg0,,,,,"FATAL","57P03","the database system is starting up","last replayed record at 0/0",,,,,,0,,"postmaster.c",2570, 2024-01-03 12:15:42.547434 CST,,,p507755,th-1510483840,,,,0,,,seg0,,,,,"PANIC","XX000","could not locate a valid checkpoint record",,,,,,,0,,"xlog.c",6745,"Stack trace: 1 0xbfcc9c postgres errstart (elog.c:557) 2 0x74e95b postgres StartupXLOG (xlog.c:6744) 3 0xa1b983 postgres StartupProcessMain (startup.c:248) 4 0x7899bf postgres AuxiliaryProcessMain (bootstrap.c:453) 5 0xa179dc postgres <symbol not found> (postmaster.c:5762) 6 0xa1b182 postgres PostmasterMain (postmaster.c:1509) 7 0x6b4dd1 postgres main (main.c:205) 8 0x7f81a2b5e555 libc.so.6 __libc_start_main + 0xf5 9 0x6c0c7c postgres <symbol not found> + 0x6c0c7c " 2024-01-03 12:15:42.563658 CST,,,p507749,th-1510483840,,,,0,,,seg0,,,,,"LOG","00000","startup process (PID 507755) was terminated by signal 6: Aborted",,,,,,,0,,"postmaster.c",4009, 2024-01-03 12:15:42.563707 CST,,,p507749,th-1510483840,,,,0,,,seg0,,,,,"LOG","00000","aborting startup due to startup process failure",,,,,,,0,,"postmaster.c",3249, |