合 GreenPlum中的镜像的分步策略介绍(group、spread和Block Mirroring)
Tags: GreenPlumBlock Mirroringgroupspread镜像分步策略
简介
在 Greenplum 数据库中,镜像(mirroring)是一种用于确保数据高可用性的机制。每个数据分片(segment)都有一个主节点(primary)和一个或多个镜像节点(mirror),当主节点出现故障时,镜像节点可以接管工作。
Greenplum 提供了两种标准的镜像分布策略:group 和 spread。
- Group mirroring — 每台主机镜像另外机器的主segment。这是 gpinitsystem和gpaddmirrors的默认选项。
- Spread mirroring — 镜像分散分布在另外的机器上。这要求集群中的主机数量多于每台主机上的 segment实例数量。
在下面的架构中,每个节点有8个主实例和8个备实例,采用GROUP策略(默认),将8个备实例全部放在另一台服务器上,如果一台服务器down了,另一台服务器上的镜像就会启动工作,这样的话该节点就相当于16个数据库实例在运行。相比其他运行8个数据库实例的节点,性能下降一半。
如果是采用SPREAD策略,则3个备实例是分别处于其他3台服务器上,这种策略相对于GROUP策略的性能下降没有那么严重,但是发生主备同时down的几率更大,更容易导致数据库不可用(只要有1对主备实例都down了就会不可用),相对没有那么稳定,这就需要根据实例需要选择某一个策略。
下面的图显示了一个在四台主机上带有八个主segment的group mirroring配置。
GROUP策略:
除非同一个segment实例的主segment和镜像都出现故障,最多可以有一半的主机故障并且集群将继续运行,只要 资源(CPU、内存和IO)足以满足需求。
任何主机故障将会让性能退化一半以上,因为具有镜像的主机将承担两倍的活动主segment。如果用户的资源利用 通常会超过50%,用户将不得不调整其负载,直至故障主机被恢复或者替换。如果用户通常的资源利用低于50%,则 集群会继续以退化的性能水平运行,直至故障被修复。
SPREAD策略:
通过spread mirroring,每台主机的主要Segment的镜像被散布在若干台主机上,涉及到的主机数量与每台主机上 segment数量相同。在集群初始化时设置spread mirroring很容易,但是要求集群中的主机数至少为每台主机上的 segment数加一。
下面的图展示了一个在四台主机上有三个主segment的集群的spread mirroring配置。
扩展使用spread mirroring的集群要求更多的规划并且可能会花费更多时间。用户必须要么增加一组数量等于每台 主机上主segment数加一的主机,要么在group mirroring配置中增加两个节点并且在扩展完成后移动镜像来重建 spread mirror配置。
对于单主机故障,spread mirroring的性能影响最小,因为每台主机的镜像都散布在多台主机上。负载的增加是 1/Nth,其中N是每台主机上主segment的数量。如果两台以上主机同时故障,spread mirroring 是最有可能遭受到灾难性故障的配置方案。
Group 策略
Group 策略将镜像节点和其对应的主节点放在同一个故障组(failure group)内。一个故障组通常对应于同一台物理主机或者相同的机架。使用这种策略的主要目的是为了数据的本地化和优化 I/O 性能。
1 2 3 4 5 6 | 优点: 可以减少跨网络传输的延迟,因为主节点和镜像节点位于同一个故障组。 提高 I/O 性能,因为数据不需要通过网络来传输。 缺点: 如果同一个故障组内的所有节点同时发生故障(例如整台服务器宕机),可能会导致数据不可用,因为主节点和镜像节点都位于同一个故障组。 |
Spread 策略
Spread 策略将镜像节点分布到不同的故障组中。故障组可以是不同的物理主机、机架,甚至是数据中心。这种策略的主要目的是提高数据的容错性和可用性。
1 2 3 4 5 6 7 | 优点: 提供更高的容错能力,因为即使一个故障组内的所有节点都发生故障,数据仍然可以在其他故障组中的镜像节点上得到保护。 更适合分布在多个物理位置的集群,因为它提供了跨机架或跨数据中心的冗余。 缺点: 可能会增加网络延迟,因为镜像节点和主节点之间需要通过网络进行数据同步。 在高负载或大规模集群中,跨机架或跨数据中心的同步可能会增加网络带宽的消耗。 |
现有集群查询镜像模式
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | [gpadmin@mdw ~]$ gpstate -m | grep Type 20240828:16:24:04:1775852 gpstate:mdw:gpadmin-[INFO]:--Type = Group [gpadmin@mdw ~]$ [gpadmin@mdw ~]$ gpstate -m 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:-Starting gpstate with args: -m 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.25.3 build commit:367edc6b4dfd909fe38fc288ade9e294d74e3f9a Open Source' 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.25.3 build commit:367edc6b4dfd909fe38fc288ade9e294d74e3f9a Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Oct 4 2023 23:27:58' 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:-Obtaining Segment details from master... 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:-------------------------------------------------------------- 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:--Current GPDB mirror list and status 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:--Type = Group 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:-------------------------------------------------------------- 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- Mirror Datadir Port Status Data Status 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw2 /opt/greenplum/data/mirror/gpseg0 7000 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw2 /opt/greenplum/data/mirror/gpseg1 7001 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw2 /opt/greenplum/data/mirror/gpseg2 7002 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw2 /opt/greenplum/data/mirror/gpseg3 7003 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw3 /opt/greenplum/data/mirror/gpseg4 7000 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw3 /opt/greenplum/data/mirror/gpseg5 7001 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw3 /opt/greenplum/data/mirror/gpseg6 7002 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw3 /opt/greenplum/data/mirror/gpseg7 7003 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw4 /opt/greenplum/data/mirror/gpseg8 7000 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw4 /opt/greenplum/data/mirror/gpseg9 7001 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw4 /opt/greenplum/data/mirror/gpseg10 7002 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw4 /opt/greenplum/data/mirror/gpseg11 7003 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw1 /opt/greenplum/data/mirror/gpseg12 7000 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw1 /opt/greenplum/data/mirror/gpseg13 7001 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw1 /opt/greenplum/data/mirror/gpseg14 7002 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:- sdw1 /opt/greenplum/data/mirror/gpseg15 7003 Passive Synchronized 20231017:13:53:00:013224 gpstate:mdw:gpadmin-[INFO]:-------------------------------------------------------------- [gpadmin@mdw ~]$ [gpadmin@mdw ~]$ gpstate -c 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:-Starting gpstate with args: -c 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.25.3 build commit:367edc6b4dfd909fe38fc288ade9e294d74e3f9a Open Source' 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.25.3 build commit:367edc6b4dfd909fe38fc288ade9e294d74e3f9a Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Oct 4 2023 23:27:58' 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:-Obtaining Segment details from master... 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:-------------------------------------------------------------- 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:--Current GPDB mirror list and status 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:--Type = Group 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:-------------------------------------------------------------- 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Status Data State Primary Datadir Port Mirror Datadir Port 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw1 /opt/greenplum/data/primary/gpseg0 6000 sdw2 /opt/greenplum/data/mirror/gpseg0 7000 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw1 /opt/greenplum/data/primary/gpseg1 6001 sdw2 /opt/greenplum/data/mirror/gpseg1 7001 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw1 /opt/greenplum/data/primary/gpseg2 6002 sdw2 /opt/greenplum/data/mirror/gpseg2 7002 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw1 /opt/greenplum/data/primary/gpseg3 6003 sdw2 /opt/greenplum/data/mirror/gpseg3 7003 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw2 /opt/greenplum/data/primary/gpseg4 6000 sdw3 /opt/greenplum/data/mirror/gpseg4 7000 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw2 /opt/greenplum/data/primary/gpseg5 6001 sdw3 /opt/greenplum/data/mirror/gpseg5 7001 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw2 /opt/greenplum/data/primary/gpseg6 6002 sdw3 /opt/greenplum/data/mirror/gpseg6 7002 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw2 /opt/greenplum/data/primary/gpseg7 6003 sdw3 /opt/greenplum/data/mirror/gpseg7 7003 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw3 /opt/greenplum/data/primary/gpseg8 6000 sdw4 /opt/greenplum/data/mirror/gpseg8 7000 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw3 /opt/greenplum/data/primary/gpseg9 6001 sdw4 /opt/greenplum/data/mirror/gpseg9 7001 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw3 /opt/greenplum/data/primary/gpseg10 6002 sdw4 /opt/greenplum/data/mirror/gpseg10 7002 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw3 /opt/greenplum/data/primary/gpseg11 6003 sdw4 /opt/greenplum/data/mirror/gpseg11 7003 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw4 /opt/greenplum/data/primary/gpseg12 6000 sdw1 /opt/greenplum/data/mirror/gpseg12 7000 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw4 /opt/greenplum/data/primary/gpseg13 6001 sdw1 /opt/greenplum/data/mirror/gpseg13 7001 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw4 /opt/greenplum/data/primary/gpseg14 6002 sdw1 /opt/greenplum/data/mirror/gpseg14 7002 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized sdw4 /opt/greenplum/data/primary/gpseg15 6003 sdw1 /opt/greenplum/data/mirror/gpseg15 7003 20231017:14:00:06:013843 gpstate:mdw:gpadmin-[INFO]:-------------------------------------------------------------- [gpadmin@mdw ~]$ |
Block mirroring
Block mirroring是一种自定义镜像配置,它把集群中的主机划分成相等尺寸的块并且在块内的主机上均匀 地分布镜像。如果一个主segment故障,其在同一个块的另一主机上的镜像会变成活动的主segment。如果一台segment 主机故障,在该块中其他每一台主机上的镜像segment都会变成活动segment。
对于block mirroring,节点被划分成块,例如具有四台或者八台主机的块,而每台主机上segment的镜像被放置在 块中的其他主机上。根据块中主机的数量以及每台主机上主segment的数量,每台主机会为其他每一台主机的segment 维护超过一个镜像。