linux之磁盘阵列实战
一、磁盘阵列概述
磁盘阵列（Redundant Arrays of Independent Disks，RAID），有“独立磁盘构成的具有冗余能力的阵列”之意。是防止硬盘突然损坏导致数据丢失的一种冗余备份机制。
磁盘阵列有三种形式：外接式磁盘阵列柜、内接式磁盘阵列卡，利用软件来仿真。
二、常见磁盘阵列RAID类型：
1、RAID 0
RAID 0(也称条带模式)是组建磁盘阵列中最简单的一种形式，只需要2块以上的硬盘即可，成本低，可以提高整个磁盘的性能和吞吐量。RAID 0没有提供冗余或错误修复能力，但实现成本是最低的。如下图，如数据量为10G，第一块硬盘存储5G，第二块硬盘存储剩下的5G。当其中一块硬盘损坏时，数据就损坏了。生产环境不建议使用。
2、RAID 1
RAID 1(也称磁盘镜像)，是把一个磁盘的数据镜像到另一个磁盘上，也就是说数据在写入一块磁盘的同时，会在另一块闲置的磁盘上生成镜像文件。磁盘使用率为50%，其中一块磁盘损坏，不会影响数据，如下图。以四块80GB容量的硬盘来讲，可利用的磁盘空间仅为160GB。RAID 1多用在保存关键性的重要数据的场合。
3、RAID 5
RAID 5是分布式奇偶校验的独立磁盘结构。如下图，它的奇偶校验码存在于所有磁盘上，其中的p0代表第0带区的奇偶校验值，至少需要三块硬盘，第一块磁盘和第二块磁盘做一个异或运算得到一个校验位，每两块磁盘做一次异或运算，这样在一块磁盘损坏的时候，根据其余两块磁盘就可以推算出损坏的部分，从而保证了数据的完整性。磁盘使用率为（n-1）/ n ,工作中使用最多的就是raid 5的模式。
4、RAID 10
Raid 10是一个Raid 0与Raid1的组合体，它是利用奇偶校验实现条带集镜像，所以它继承了Raid0的快速和Raid1的安全，最少需要4块硬盘。如下图
三、各RAID类型实战
注：通过以上类型描述，以下均采用软RAID实现。创建软RAID命令为mdadm
mdadm命令常用参数如下：
-C或--creat：建立一个新阵列
-A：激活磁盘阵列
-D或--detail：打印阵列设备的详细内容
-s或--scan：扫描配置文件或/proc/mdstat得到阵列缺失信息
-f：将设备状态定为故障
-a或--add：添加设备到阵列
-v或--verbose：显示详细信息
-r：移除设备
-l或--level=：设定磁盘阵列的级别
-n或--raid-devices=：设定阵列成员(分区/磁盘)的数量
-x或--space-devices=：设定阵列中备用磁盘数量
-c或--chunk=：设定阵列的块chunk大小，单位为KB。chunk(块)，RAID的存储数据时每个数据段大小为4KB、64KB
-G或--grow：改变阵型大小或形态
1、RAID 0
示例要求：创建RAID 0；格式化并且永久挂载到指定目录。
示例环境：准备2块磁盘，/dev/sdb /dev/sdc，每一个硬盘上分出一个5G的分区，在/mnt目录下创建RAID0目录(mkdir -pv /mnt/RAID0,供挂载使用)
1)查看当前硬盘信息
[root@hc03 ~]# fdisk -lDisk /dev/sda: 42.9 GB, 42949672960 bytes255 heads, 63 sectors/track, 5221 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x000ecb54 Device Boot Start End Blocks Id System/dev/sda1 * 1 103 819200 83 LinuxPartition 1 does not end on cylinder boundary./dev/sda2 103 3927 30720000 8e Linux LVMDisk /dev/sdb: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sdc: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000
2)为sdb硬盘分区,创建新主分区sdb1
[root@hc03 ~]# fdisk /dev/sdbDevice contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel with disk identifier 0x19ab7785.Changes will remain in memory only, until you decide to write them.After that, of course, the previous content won't be recoverable.Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').
3)输入n回车新建分区，接着再输入p回车新建主分区
Command (m for help): nCommand action e extended p primary partition (1-4)p
4)此处要求选择分区号在1-4间，输入1回车
Partition number (1-4): 1First cylinder (1-1305, default 1): Using default value 1
5)设置分区分区大小为5G，输入+5G后回车
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): +5G
6)输入p查看分区是否成功，输入p回车，如下图：显示分区成功
Command (m for help): pDisk /dev/sdb: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x19ab7785 Device Boot Start End Blocks Id System/dev/sdb1 1 654 5253223+ 83 Linux
7)最后输入w回车，进行保存退出
Command (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.
8)按上述方法，为硬盘sdc创建5G大小sdc1分区
Command (m for help): pDisk /dev/sdc: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x356d6b05 Device Boot Start End Blocks Id System/dev/sdc1 1 654 5253223+ 83 Linux
9)使用mdadm命令把sdb/sdc硬盘分别sdb1、sdc1分区创建为RAID 0
[root@hc03 ~]# mdadm -C -v /dev/md0 -l 0 -n 2 /dev/sdb1 /dev/sdc1mdadm: chunk size defaults to 512Kmdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md0 started. 注释：-C创建新磁盘阵列，-l 0 为RAID类型为0，2为硬盘个数。
10)查看阵列信息，命令mdadm -Ds，以及创建raid 0后/dev/md0硬盘分区大小信息
[root@hc03 ~]# mdadm -DsARRAY /dev/md0 metadata=1.2 name=hc03:0 UUID=b524bc38:194dae34:41d4adec:24b3a69e[root@hc03 ~]# fdisk -l /dev/md0Disk /dev/md0: 10.8 GB, 10750001152 bytes2 heads, 4 sectors/track, 2624512 cylindersUnits = cylinders of 8 * 512 = 4096 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 524288 bytes / 1048576 bytesDisk identifier: 0x00000000
11)为/dev/md0新生成硬盘分区(与上述sdb或sdc分区相同)
[root@hc03 ~]# fdisk /dev/md0Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel with disk identifier 0xe58709bd.Changes will remain in memory only, until you decide to write them.After that, of course, the previous content won't be recoverable.Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-2624512, default 257): Using default value 257Last cylinder, +cylinders or +size{K,M,G} (257-2624512, default 2624512): Using default value 2624512Command (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.
12)查看创建后的分区信息
[root@hc03 ~]# fdisk -l /dev/md*last_lba(): I don't know how to handle files with mode 40755Disk /dev/md0: 10.8 GB, 10750001152 bytes2 heads, 4 sectors/track, 2624512 cylindersUnits = cylinders of 8 * 512 = 4096 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 524288 bytes / 1048576 bytesDisk identifier: 0xe58709bd Device Boot Start End Blocks Id System/dev/md0p1 257 2624512 10497024 83 LinuxDisk /dev/md0p1: 10.7 GB, 10748952576 bytes2 heads, 4 sectors/track, 2624256 cylindersUnits = cylinders of 8 * 512 = 4096 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 524288 bytes / 1048576 bytesDisk identifier: 0x00000000
13)格式化/dev/md0p1
[root@hc03 ~]# mkfs.ext4 /dev/md0p1mke2fs 1.41.12 (17-May-2010)Filesystem label= #文件系统标签OS type: Linux #操作系统Block size=4096 (log=2) #块大小Fragment size=4096 (log=2)Stride=128 blocks, Stripe width=256 blocks657072 inodes, 2624256 blocks131212 blocks (5.00%) reserved for the super userFirst data block=0Maximum filesystem blocks=268854886481 block groups32768 blocks per group, 32768 fragments per group8112 inodes per groupSuperblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632Writing inode tables: done #写入inode表 Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 29 mounts or180 days, whichever comes first. Use tune2fs -c or -i to override.
14)设置开机启动自动挂载/dev/md0p1至/dev/RAID0目录
[root@hc03 ~]# mount /dev/md0p1 /mnt/RAID0[root@hc03 ~]# mount -l/dev/mapper/my03-LogVol01 on / type ext4 (rw)proc on /proc type proc (rw)sysfs on /sys type sysfs (rw)devpts on /dev/pts type devpts (rw,gid=5,mode=620)tmpfs on /dev/shm type tmpfs (rw)/dev/sda1 on /boot type ext4 (rw)/dev/mapper/my03-LogVol02 on /usr type ext4 (rw)none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)/dev/md0p1 on /mnt/RAID0 type ext4 (rw) #挂载成功[root@hc03 ~]# cat /etc/fstab #加入开机启动自动挂载## /etc/fstab# Created by anaconda on Sat Oct 31 03:45:15 2015## Accessible filesystems, by reference, are maintained under '/dev/disk'# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info#/dev/mapper/my03-LogVol01 / ext4 defaults 1 1UUID=c06d0732-24b8-4dfa-baba-f5ed54bea494 /boot ext4 defaults 1 2/dev/mapper/my03-LogVol02 /usr ext4 defaults 1 2/dev/mapper/my03-LogVol00 swap swap defaults 0 0tmpfs /dev/shm tmpfs defaults 0 0devpts /dev/pts devpts gid=5,mode=620 0 0sysfs /sys sysfs defaults 0 0proc /proc proc defaults 0 0/dev/md0p1 /mnt/RAID0 ext4 defaults 0 0
2、RAID 1
示例要求：创建RAID 1；开机自动挂载至/mnt/RAID1并准备一个热备盘，模拟RAID 1中一个硬盘故障，并移除阵列。
示例环境：准备3块磁盘，/dev/sdb、/dev/sdc、/dev/sdd，每一个硬盘上分出一个2G的分区,分区以此/dev/sdb2、/dev/sdc2、/dev/sdd1
1)查看当前硬盘信息
[root@hc03 ~]# fdisk -lDisk /dev/sda: 42.9 GB, 42949672960 bytes255 heads, 63 sectors/track, 5221 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x000ecb54 Device Boot Start End Blocks Id System/dev/sda1 * 1 103 819200 83 LinuxPartition 1 does not end on cylinder boundary./dev/sda2 103 3927 30720000 8e Linux LVMDisk /dev/sdb: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x19ab7785 Device Boot Start End Blocks Id System/dev/sdb1 1 654 5253223+ 83 LinuxDisk /dev/sdc: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x356d6b05 Device Boot Start End Blocks Id System/dev/sdc1 1 654 5253223+ 83 LinuxDisk /dev/sdd: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000
2)为sdb硬盘分区,创建新主分区sdb2（创建方法同上）
[root@hc03 ~]# fdisk /dev/sdbWARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 2First cylinder (655-1305, default 655): Using default value 655Last cylinder, +cylinders or +size{K,M,G} (655-1305, default 1305): +2GCommand (m for help): P^HDisk /dev/sdb: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x19ab7785 Device Boot Start End Blocks Id System/dev/sdb1 1 654 5253223+ 83 Linux/dev/sdb2 655 916 2104515 83 LinuxCommand (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.WARNING: Re-reading the partition table failed with error 16: Device or resource busy.The kernel still uses the old table. The new table will be used atthe next reboot or after you run partprobe(8) or kpartx(8)Syncing disks.
注：建议重启，因为sdb1被作为RAID0并且挂载
3)按上述方法，为硬盘sdc创建2G大小sdc2分区，硬盘sdd创建2G大小sdd1分区
[root@hc03 ~]# fdisk -l /dev/sdcDisk /dev/sdc: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x356d6b05 Device Boot Start End Blocks Id System/dev/sdc1 1 654 5253223+ 83 Linux/dev/sdc2 655 916 2104515 83 Linux[root@hc03 ~]# fdisk /dev/sdd #创建sdd1分区Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel with disk identifier 0x96e37a1b.Changes will remain in memory only, until you decide to write them.After that, of course, the previous content won't be recoverable.Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-1305, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): +2GCommand (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.[root@hc03 ~]# fdisk -l /dev/sddDisk /dev/sdd: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x96e37a1b Device Boot Start End Blocks Id System/dev/sdd1 1 262 2104483+ 83 Linux
4)使用mdadm命令把sdb/sdc硬盘分别sdb2、sdc2分区创建为RAID 2,sdd1为热备盘
[root@hc03 ~]# mdadm -C -v /dev/md1 -l 1 -n 2 -x 1 /dev/sdb2 /dev/sdc2 /dev/sdd1mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90mdadm: size set to 2102400K Continue creating array? Continue creating array? (y/n) ymdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md1 started.注释：-C创建新磁盘阵列，-l 1 为RAID类型为1，2为硬盘个数，-x 1 为热备盘
5)查看阵列信息，命令mdadm -Ds，以及创建raid 0后/dev/md0硬盘分区大小信息
[root@hc03 ~]# mdadm -DsARRAY /dev/md/hc03:0 metadata=1.2 name=hc03:0 UUID=b524bc38:194dae34:41d4adec:24b3a69eARRAY /dev/md1 metadata=1.2 spares=1 name=hc03:1 UUID=ac6a9ef0:abadb228:f9c02876:1fad13c3
6)为/dev/md1新生成硬盘分区(与上述sdb或sdc分区相同)
[root@hc03 ~]# fdisk /dev/md0[root@hc03 ~]# fdisk /dev/md1Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel with disk identifier 0x34b7f63d.Changes will remain in memory only, until you decide to write them.After that, of course, the previous content won't be recoverable.Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-525600, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-525600, default 525600): Using default value 525600Command (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.
7)查看创建后的分区信息
[root@hc03 ~]# fdisk -l /dev/md1*Disk /dev/md1: 2152 MB, 2152857600 bytes2 heads, 4 sectors/track, 525600 cylindersUnits = cylinders of 8 * 512 = 4096 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x34b7f63d Device Boot Start End Blocks Id System/dev/md1p1 1 525600 2102398 83 LinuxDisk /dev/md127: 10.8 GB, 10750001152 bytes2 heads, 4 sectors/track, 2624512 cylindersUnits = cylinders of 8 * 512 = 4096 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 524288 bytes / 1048576 bytesDisk identifier: 0xe58709bd Device Boot Start End Blocks Id System/dev/md127p1 257 2624512 10497024 83 LinuxDisk /dev/md127p1: 10.7 GB, 10748952576 bytes2 heads, 4 sectors/track, 2624256 cylindersUnits = cylinders of 8 * 512 = 4096 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 524288 bytes / 1048576 bytesDisk identifier: 0x00000000Disk /dev/md1p1: 2152 MB, 2152855552 bytes2 heads, 4 sectors/track, 525599 cylindersUnits = cylinders of 8 * 512 = 4096 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000
8)格式化/dev/md1p1
[root@hc03 ~]# mkfs.ext4 /dev/md1p1mke2fs 1.41.12 (17-May-2010)Filesystem label=OS type: LinuxBlock size=4096 (log=2)Fragment size=4096 (log=2)Stride=0 blocks, Stripe width=0 blocks131648 inodes, 525599 blocks26279 blocks (5.00%) reserved for the super userFirst data block=0Maximum filesystem blocks=54106521617 block groups32768 blocks per group, 32768 fragments per group7744 inodes per groupSuperblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912Writing inode tables: done Creating journal (16384 blocks): doneWriting superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 24 mounts or180 days, whichever comes first. Use tune2fs -c or -i to override.
9)设置开机启动自动挂载/dev/md1p1至/mnt/RAID1目录
[root@hc03 ~]# mkdir -pv /mnt/RAID1mkdir: created directory `/mnt/RAID1'[root@hc03 ~]# mount /dev/md1p1 /mnt/RAID1[root@hc03 ~]# mount -l/dev/mapper/my03-LogVol01 on / type ext4 (rw)proc on /proc type proc (rw)sysfs on /sys type sysfs (rw)devpts on /dev/pts type devpts (rw,gid=5,mode=620)tmpfs on /dev/shm type tmpfs (rw)/dev/sda1 on /boot type ext4 (rw)/dev/mapper/my03-LogVol02 on /usr type ext4 (rw)none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)/dev/md1p1 on /mnt/RAID1 type ext4 (rw) #挂载成功[root@hc03 ~]# cat /etc/fstab ## /etc/fstab# Created by anaconda on Sat Oct 31 03:45:15 2015## Accessible filesystems, by reference, are maintained under '/dev/disk'# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info#/dev/mapper/my03-LogVol01 / ext4 defaults 1 1UUID=c06d0732-24b8-4dfa-baba-f5ed54bea494 /boot ext4 defaults 1 2/dev/mapper/my03-LogVol02 /usr ext4 defaults 1 2/dev/mapper/my03-LogVol00 swap swap defaults 0 0tmpfs /dev/shm tmpfs defaults 0 0devpts /dev/pts devpts gid=5,mode=620 0 0sysfs /sys sysfs defaults 0 0proc /proc proc defaults 0 0/dev/md0p1 /mnt/RAID0 ext4 defaults 0 0/dev/md1p1 /mnt/RAID1 ext4 defaults 0 0
10)使用/proc/mdstat查看阵列信息
[root@hc03 ~]# cat /proc/mdstatPersonalities : [raid0] [raid1] md1 : active raid1 sdd1[2](S) sdc2[1] sdb2[0] 2102400 blocks super 1.2 [2/2] [UU] md127 : active raid0 sdb1[0] sdc1[1] 10498048 blocks super 1.2 512k chunks unused devices:注释：#sdd1[2](s):其中s表示备用盘#[2/2]表示raid 1中的两块磁盘都是正常的#active表示此阵列正常读写
11)生成mdadm.conf配置文件
[root@hc03 ~]# mdadm -Ds > /etc/mdadm.conf[root@hc03 ~]# cat /etc/mdadm.conf ARRAY /dev/md/hc03:0 metadata=1.2 name=hc03:0 UUID=b524bc38:194dae34:41d4adec:24b3a69eARRAY /dev/md1 metadata=1.2 spares=1 name=hc03:1 UUID=ac6a9ef0:abadb228:f9c02876:1fad13c3注释：#spares=1 存在一个热备盘
12)模拟/dev/sdc2为故障盘，重启一个终端动态刷新/proc/mdstat配置信息，命令：watch -n 1 cat /proc/mdstat
[root@hc03 ~]# mdadm -f /dev/md1 /dev/sdc2 #设定sdc2为故障mdadm: set /dev/sdc2 faulty in /dev/md1[root@hc03 ~]# cat /proc/mdstat Personalities : [raid0] [raid1] md1 : active raid1 sdd1[2] sdc2[1](F) sdb2[0] 2102400 blocks super 1.2 [2/2] [UU] md127 : active raid0 sdb1[0] sdc1[1] 10498048 blocks super 1.2 512k chunks unused devices:注释：sdc2[1](F) 为故障sdd1[2]并且已经接替故障盘
13)移除故障盘sdc2，并且更新/etc/mdadm.conf配置文件
[root@hc03 ~]# mdadm -r /dev/md1 /dev/sdc2mdadm: hot removed /dev/sdc2 from /dev/md1[root@hc03 ~]# cat /proc/mdstat Personalities : [raid0] [raid1] md1 : active raid1 sdd1[2] sdb2[0] 2102400 blocks super 1.2 [2/2] [UU] md127 : active raid0 sdb1[0] sdc1[1] 10498048 blocks super 1.2 512k chunks unused devices:[root@hc03 ~]# mdadm -Ds > /etc/mdadm.conf
3、RAID 5
示例要求：创建RAID 5；开机自动挂载至/mnt/RAID5并准备一个热备盘，模拟RAID 5中一个硬盘故障，并移除阵列，同时再扩展硬盘。
示例环境：准备5块磁盘，由于资源比较充裕，直接添加新的盘/dev/sdc、/dev/sdd、/dev/sde、/dev/sdf、/dev/sdg，每一个硬盘上分出一个2G的分区,分区以此/dev/sdc1、/dev/sdd1、/dev/sde1、/dev/sdf1、/dev/sdg1
1)创建以上新分区方法，请参照上述相关操作完成，这里不再阐述！
Device Boot Start End Blocks Id System/dev/sdc1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System/dev/sdd1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System/dev/sde1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System/dev/sdf1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System/dev/sdg1 1 262 2104483+ 83 Linux
2)使用mdadm命令把sdc/sdd/sde/sdf硬盘分别sdc1/sdd1/sde1分区创建为RAID 5,sdf1为热备盘
[root@lvs ~]# mdadm -C -v /dev/md5 -l 5 -n 3 -x 1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1mdadm: layout defaults to left-symmetricmdadm: layout defaults to left-symmetricmdadm: chunk size defaults to 512Kmdadm: size set to 2102272Kmdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md5 started.
注释：-C创建新磁盘阵列，-l 5 为RAID类型为5，3为硬盘个数，-x 1 为热备盘
3)查看阵列信息，命令mdadm -Ds，以及创建raid 0后/dev/md0硬盘分区大小信息，并生成配置文件/etc/mdadm.conf
[root@lvs ~]# mdadm -Ds > /etc/mdadm.confARRAY /dev/md5 metadata=1.2 spares=1 name=lvs:5 UUID=8a62a5ad:9b1fd7ae:09b5f7e2:c0756e62
4)使用/proc/mdstat查看阵列信息
[root@lvs ~]# cat /proc/mdstatPersonalities : [raid6] [raid5] [raid4] md5 : active raid5 sde1[4] sdf1[3](S) sdd1[1] sdc1[0] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]注释：#sdf1[3](s):其中s表示备用盘#[3/3]表示raid 1中的两块磁盘都是正常的#active表示此阵列正常读写
5)模拟/dev/sde1为故障盘，重启一个终端动态刷新/proc/mdstat配置信息，命令：watch -n 1 cat /proc/mdstat
[root@hc03 ~]# mdadm -f /dev/md5 /dev/sde1 #设定sde1为故障[root@lvs ~]# mdadm -f /dev/md5 /dev/sde1mdadm: set /dev/sde1 faulty in /dev/md5[root@lvs ~]# cat /proc/mdstatPersonalities : [raid6] [raid5] [raid4] md5 : active raid5 sde1[4](F) sdf1[3] sdd1[1] sdc1[0] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices:注释：sde1[4](F) 为故障sdf1[3]并且已经接替故障盘 6)移除故障盘sde1,更新/etc/mdadm.conf配置文件
[root@lvs ~]# mdadm -r /dev/md5 /dev/sde1mdadm: hot removed /dev/sde1 from /dev/md5[root@lvs ~]# cat /proc/mdstatPersonalities : [raid6] [raid5] [raid4] md5 : active raid5 sdf1[3] sdd1[1] sdc1[0] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices:[root@lvs ~]# mdadm -Ds > /etc/mdadm.conf
7)md5磁盘分区格式化，并开机自动挂载至/mnt/RAID5目录
[root@lvs ~]# mkdir -pv /mnt/RAID5 #创建/mnt/RAID5目录mkdir: 已创建目录 "/mnt/RAID5"[root@lvs ~]# fdisk /dev/md5 #磁盘分区Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel with disk identifier 0xc24c42ef.Changes will remain in memory only, until you decide to write them.After that, of course, the previous content won't be recoverable.Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-1051136, default 257): Using default value 257Last cylinder, +cylinders or +size{K,M,G} (257-1051136, default 1051136): Using default value 1051136Command (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.[root@lvs ~]# mkfs.ext4 /dev/md5p1 #格式化分区mke2fs 1.41.12 (17-May-2010)文件系统标签=操作系统:Linux块大小=4096 (log=2)分块大小=4096 (log=2)Stride=128 blocks, Stripe width=256 blocks262944 inodes, 1050880 blocks52544 blocks (5.00%) reserved for the super user第一个数据块=0Maximum filesystem blocks=107793612833 block groups32768 blocks per group, 32768 fragments per group7968 inodes per groupSuperblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736正在写入inode表: 完成 Creating journal (32768 blocks): 完成Writing superblocks and filesystem accounting information: 完成This filesystem will be automatically checked every 39 mounts or180 days, whichever comes first. Use tune2fs -c or -i to override.[root@lvs ~]# mount /dev/md5p1 /mnt/RAID5 #挂载[root@lvs ~]# mount -l #查看挂载/dev/mapper/vg_lvs-lv_root on / type ext4 (rw)proc on /proc type proc (rw)sysfs on /sys type sysfs (rw)devpts on /dev/pts type devpts (rw,gid=5,mode=620)tmpfs on /dev/shm type tmpfs (rw)/dev/sda1 on /boot type ext4 (rw)none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)/dev/md5p1 on /mnt/RAID5 type ext4 (rw) #正常挂载[root@lvs ~]# cat /etc/fstab #添加至开机启动挂载## /etc/fstab# Created by anaconda on Mon Nov 2 18:20:45 2015## Accessible filesystems, by reference, are maintained under '/dev/disk'# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info#/dev/mapper/vg_lvs-lv_root / ext4 defaults 1 1UUID=146bb43b-343e-4f5a-90c6-4056c695bfa1 /boot ext4 defaults 1 2/dev/mapper/vg_lvs-lv_swap swap swap defaults 0 0tmpfs /dev/shm tmpfs defaults 0 0devpts /dev/pts devpts gid=5,mode=620 0 0sysfs /sys sysfs defaults 0 0proc /proc proc defaults 0 0/dev/md5p1 /mnt/RAID5 ext4 defaults 0 0
8)操作停止和激活阵列，使用mdadm -S 或 mdadm -As(注意：挂载中的阵列不允许停止，需要卸载挂载)
如：停止上面阵列RAID5
[root@lvs ~]# mdadm -S /dev/md5mdadm: Cannot get exclusive access to /dev/md5:Perhaps a running process, mounted filesystem or active volume group?[root@lvs ~]# umount /dev/md5p1 /mnt/RAID5umount: /mnt/RAID5: not mounted[root@lvs ~]# mdadm -S /dev/md5mdadm: stopped /dev/md5注：mdadm -Ss 为停止所有阵列
激活停止的RAID5，需要重新挂载
激活raid 5阵列：
[root@lvs ~]# mdadm -As mdadm: /dev/md/5 has been started with 3 drives.mdadm: Found some drive for an array that is already active: /dev/md/5mdadm: giving up.[root@lvs ~]# mount /dev/md/5p1 /mnt/RAID5 #重新挂载
9)上面模拟一个硬盘故障，如何向阵列中添加一个磁盘，这里以另外一个磁盘/dev/sdg1为例(注意：挂载中的队列不允许，添加磁盘)
[root@lvs ~]# umount /dev/md/5p1 /mnt/RAID5 #卸载挂载[root@lvs ~]# mdadm -a /dev/md5 /dev/sdg1mdadm: added /dev/sdg1[root@lvs ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sdg1[4](S) sdc1[0] sdf1[3] sdd1[1] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices:注：sdg1[4](S) #说明sdg1已经是备用盘了[root@lvs ~]# mdadm -G /dev/md5 -n 4 #添加至md5[root@lvs ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sdg1[4] sdc1[0] sdf1[3] sdd1[1] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU] [=======>.............] reshape = 36.0% (757580/2102272) finish=0.2min speed=108225K/sec unused devices: 注：active raid5 sdg1[4] sdc1[0] sdf1[3] sdd1[1] #当前活动的已经有4个盘了[root@lvs ~]# mdadm -Ds > /etc/mdadm.conf #刷新配置[root@lvs ~]# mdadm -Ds ARRAY /dev/md/5 metadata=1.2 name=lvs:5 UUID=8a62a5ad:9b1fd7ae:09b5f7e2:c0756e62
4、RAID 10
示例要求：创建RAID 10；开机自动挂载至/mnt/RAID10。
示例环境：准备4块磁盘，直接添加新的盘/dev/sdb、/dev/sdc、/dev/sdd、/dev/sde，每一个硬盘上分出一个5G的分区,分区以此/dev/sdb1、/dev/sdc1、/dev/sdd1、/dev/sde1。
1)创建以上新分区方法，请参照上述相关操作完成，这里不再阐述！
Device Boot Start End Blocks Id System/dev/sdb1 1 652 5237158+ 83 Linux Device Boot Start End Blocks Id System/dev/sdc1 1 652 5237158+ 83 Linux Device Boot Start End Blocks Id System/dev/sdd1 1 652 5237158+ 83 Linux Device Boot Start End Blocks Id System/dev/sde1 1 652 5237158+ 83 Linux
2)使用mdadm命令把sdb/sdc和/sdd/sde硬盘分别sdb1/sdc1和/sdd1/sde1分区分别两个创建为RAID 1
[root@node1 ~]# mdadm -C -v /dev/md00 -l 1 -n 2 /dev/sdb1 /dev/sdc1mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90mdadm: size set to 5233024KContinue creating array? Continue creating array? (y/n) ymdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md00 started.[root@node1 ~]# mdadm -C -v /dev/md01 -l 1 -n 2 /dev/sdd1 /dev/sde1mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90mdadm: size set to 5233024KContinue creating array? Continue creating array? (y/n) ymdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md01 started.注释：-C创建新磁盘阵列，-l 1 为RAID类型为1，2为硬盘个数
3)把上面创建新/dev/md00和/dev/md01创建为raid 0
[root@node1 ~]# mdadm -C -v /dev/md10 -l 0 -n 2 /dev/md0 /dev/md1mdadm: chunk size defaults to 512Kmdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md10 started.
注意：上面创建用新的磁盘/dev/md00、/dev/md01用做raid 1 之后，名称变为/dev/md0和/dev/md1(可用fdisk -l 查看)
4)查看阵列信息，命令mdadm -Ds或/proc/mdstat，并生成配置文件/etc/mdadm.conf
[root@node1 ~]# cat /proc/mdstat Personalities : [raid1] [raid0] md10 : active raid0 md1[1] md0[0] 10457088 blocks super 1.2 512k chunks md1 : active raid1 sde1[1] sdd1[0] 5233024 blocks super 1.2 [2/2] [UU] md0 : active raid1 sdc1[1] sdb1[0] 5233024 blocks super 1.2 [2/2] [UU] unused devices:注释：#sdc1[1] sdb1[0]和sde1[1] sdd1[0]:为RAID 1#md1[1] 和 md0[0]:为RAID 0[root@lvs ~]# mdadm -Ds > /etc/mdadm.confARRAY /dev/md0 metadata=1.2 name=node1:00 UUID=dc9713a3:487f0925:0b8dcd0f:d13aec68ARRAY /dev/md1 metadata=1.2 name=node1:01 UUID=edce41b0:6bb6b1cc:de7233fb:41cc27fbARRAY /dev/md10 metadata=1.2 name=node1:10 UUID=58b21c66:8a8d9b01:c3900b13:6d2d5292
5)md10创建后进行磁盘分区格式化，并开机自动挂载至目录(这里不再演示，请自行参照上述操作)
5、如何删除RAID 阵列，这里上面的raid 10(md10)为例子
1)取消挂载 umount /dev/md10p1(如/dev/md10有挂载)
2)停止raid 10设备
[root@node1 ~]# mdadm -S /dev/md10mdadm: stopped /dev/md10
3)删除/etc/mdadm.conf文件
[root@node1 ~]# rm -rf /etc/mdadm.conf
4)清除磁盘/dev/md0和/dev/md1的raid标识
[root@node1 ~]# mdadm --misc --zero-superblock /dev/md0[root@node1 ~]# mdadm --misc --zero-superblock /dev/md1注释：--misc #修改MD相关设备--zero-superblock #擦除设备中MD的超级块
[root@node1 ~]# mdadm -Ds #查看ARRAY /dev/md0 metadata=1.2 name=node1:00 UUID=dc9713a3:487f0925:0b8dcd0f:d13aec68ARRAY /dev/md1 metadata=1.2 name=node1:01 UUID=edce41b0:6bb6b1cc:de7233fb:41cc27fb[root@node1 ~]# cat /proc/mdstat #查看当前阵列Personalities : [raid1] [raid0] md1 : active raid1 sde1[1] sdd1[0] 5233024 blocks super 1.2 [2/2] [UU] md0 : active raid1 sdc1[1] sdb1[0] 5233024 blocks super 1.2 [2/2] [UU] unused devices: