Bash Script for Zero-Downtime Linux OS Mirroring: Converting a Single-Disk Server to Live High-Availability RAID 1

By | September 23, 2026
0 Comment

When you first commission a home server or NAS, setting up the operating system on a single, fast SSD is quick and straightforward. But as your services grow—hosting Docker containers, databases, storage shares, and automation routines—that single OS drive becomes a dangerous single point of failure (SPOF).

If that primary drive suffers a controller failure or NAND degradation, your server goes dark until you find replacement hardware, reinstall Linux, re-configure network interfaces, restore users, and rebuild boot configs.

In this guide, we walk through the architecture, real-world edge cases, and a fully automated, battle-tested script to convert a live, single-disk Linux (Fedora / RHEL / CentOS / Debian) system into a mirrored RAID 1 array online with zero downtime and zero data loss.

🏗 High-Availability Architecture Overview

To achieve true hardware redundancy, every layer of the storage and boot hierarchy must be mirrored across both physical drives:

Key Architectural Decisions:

  1. /boot on RAID 1 (/dev/md10) with metadata=1.0:
    Standard mdadm versions (metadata 1.2) place the superblock at the start of the partition, which legacy BIOS and older bootloaders cannot read directly without specialized modules. Using --metadata=1.0 writes the RAID superblock at the very end of the partition, leaving the raw ext4 filesystem structure at sector 0 so standard BIOS/GRUB loaders can read the kernel and initramfs natively.
  2. LVM on RAID 1 (/dev/md11) with metadata=1.2:
    The entire LVM Volume Group (root, home, swap) runs on top of a single mirrored mdadm block device.
  3. Dual-Disk GRUB Installation:
    GRUB is written to the Master Boot Record / boot sectors of both physical disks. If Drive A is unplugged or fails mechanically, the BIOS simply falls back to Drive B and boots into the mirror without manual intervention.

⚠️ Real-World Gotchas & Lessons Learned

During the migration, three critical engineering hurdles were solved:

1. The pvmove Metadata Deficit Trap

When you create an mdadm array on a partition that matches your source drive’s size, mdadm consumes ~128 MB (32 LVM extents) for its internal superblock and bitmap logs.
When attempting to live-migrate the volume group via pvmove, LVM will reject the operation with:

Insufficient free space: 56977 extents needed, but only 56945 available
Unable to allocate mirror extents for vg/pvmove0.
Failed to convert pvmove LV to mirrored.

The Solution:
Instead of resizing complex ext4 filesystems like / or /home, we safely trim the raw swap logical volume by 1 GB:

  1. Disable active swap (swapoff).
  2. Wipe the swap filesystem signature (wipefs -af) so LVM’s automated fsadm check doesn’t block the resize.
  3. Shrink the LV (lvresize -L 6G).
  4. Recreate and re-enable swap (mkswap & swapon).

This frees up 256+ extents in milliseconds with zero risk to production data, giving pvmove plenty of headroom to complete the live mirror migration.

2. Device Busy Locks on Incomplete Initializations

If a script run is interrupted, the kernel or udev may hold degraded RAID descriptors open. A reliable automation script must actively stop lingering arrays (mdadm --stop), settle the udev event queue (udevadm settle), and wipe leftover partition and filesystem headers before partitioning.

3. Persistent Bootloader & Dracut Modules

To ensure subsequent kernel updates (e.g. dnf upgrade kernel) generate bootable images containing the required RAID drivers, you must declare add_dracutmodules+=" mdraid lvm " in /etc/dracut.conf.d/raid.conf and update /etc/mdadm.conf.

🛠 Complete Automated Conversion Script

Here is the sanitized, production-ready script generated with agy (Gemini CLI). It can be run on any live Linux server running LVM over MBR/BIOS.

#!/bin/bash
# ==============================================================================
# Script: setup_live_os_raid1.sh
# Description: Converts a single-disk Linux OS installation to a live, mirrored
#              High-Availability RAID 1 array (LVM + /boot) with dual-MBR GRUB.
# Requirements: root privileges, mdadm, dracut, grub2, lvm2, rsync, parted.
# ==============================================================================
set -euo pipefail

LOGFILE="/var/log/setup_os_raid1_$(date +%Y%m%d_%H%M%S).log"
exec > >(tee -a "$LOGFILE") 2>&1

echo "=========================================================="
echo "    Linux High-Availability OS RAID 1 Mirroring Setup     "
echo "=========================================================="
echo "Timestamp: $(date)"
echo "Log file:  $LOGFILE"

if [ "$EUID" -ne 0 ]; then
  echo "[-] ERROR: This script must be run as root: sudo bash $0"
  exit 1
fi

# ------------------------------------------------------------------------------
# 1. Drive Identification & Target Safety Verification
# ------------------------------------------------------------------------------
DISK_SRC="/dev/sda"
DISK_DST="/dev/sdb"

# (Optional) Hardcode device serial numbers for maximum safety in production
SRC_SERIAL=$(lsblk -d -no SERIAL "$DISK_SRC" 2>/dev/null || true)
DST_SERIAL=$(lsblk -d -no SERIAL "$DISK_DST" 2>/dev/null || true)

echo "[1/10] Verifying target drives..."
echo "  - Source Disk (Active OS): $DISK_SRC (Serial: ${SRC_SERIAL:-N/A})"
echo "  - Target Disk (New Mirror): $DISK_DST (Serial: ${DST_SERIAL:-N/A})"

# ------------------------------------------------------------------------------
# 2. Safety Backups
# ------------------------------------------------------------------------------
echo "[2/10] Creating safety backup of /boot and critical configs..."
BACKUP_DIR="/root/os_pre_raid_backup_$(date +%Y%m%d_%H%M%S)"
mkdir -p "$BACKUP_DIR"
cp -a /etc/fstab /etc/default/grub "$BACKUP_DIR/" 2>/dev/null || true
cp -a /etc/mdadm.conf "$BACKUP_DIR/" 2>/dev/null || true
rsync -aHAX /boot/ "$BACKUP_DIR/boot_backup/"
echo "[+] Backup saved to $BACKUP_DIR"

# Clean any lingering RAID descriptors
echo "[*] Cleaning existing RAID descriptors..."
pvmove --abort 2>/dev/null || true
vgreduce fedora00 /dev/md11 2>/dev/null || true
pvremove -ff -y /dev/md11 2>/dev/null || true
mdadm --stop /dev/md10 2>/dev/null || true
mdadm --stop /dev/md11 2>/dev/null || true
udevadm settle

# ------------------------------------------------------------------------------
# 3. Swap Resize (Prevent pvmove metadata extent shortfall)
# ------------------------------------------------------------------------------
echo "[3/10] Resizing swap LV to 6GB (freeing extents for mdadm metadata)..."
swapoff /dev/mapper/fedora00-swap 2>/dev/null || true
wipefs -af /dev/mapper/fedora00-swap 2>/dev/null || true
lvresize -y -L 6G /dev/mapper/fedora00-swap
mkswap /dev/mapper/fedora00-swap
swapon /dev/mapper/fedora00-swap
echo "[+] Swap resized successfully!"

# ------------------------------------------------------------------------------
# 4. Wipe & Partition Target Drive (/dev/sdb)
# ------------------------------------------------------------------------------
echo "[4/10] Wiping $DISK_DST and cloning partition geometry..."
for p in $(lsblk -lno NAME "$DISK_DST"); do
    umount -f "/dev/$p" 2>/dev/null || true
done
wipefs -af "${DISK_DST}1" 2>/dev/null || true
wipefs -af "${DISK_DST}2" 2>/dev/null || true
wipefs -af "$DISK_DST"
dd if=/dev/zero of="$DISK_DST" bs=1M count=20 conv=fsync status=none

# Create matching MBR partition table with Linux RAID flags (0xfd)
parted -s "$DISK_DST" mklabel msdos
parted -s -a optimal "$DISK_DST" mkpart primary 2048s 2099199s
parted -s -a optimal "$DISK_DST" mkpart primary 2099200s 468861951s
parted -s "$DISK_DST" set 1 boot on
parted -s "$DISK_DST" set 1 raid on
parted -s "$DISK_DST" set 2 raid on
partprobe "$DISK_DST"
udevadm settle

# ------------------------------------------------------------------------------
# 5. Create Degraded RAID 1 Arrays on Target Disk
# ------------------------------------------------------------------------------
echo "[5/10] Creating degraded RAID1 arrays on $DISK_DST..."
mdadm --zero-superblock --force "${DISK_DST}1" 2>/dev/null || true
mdadm --zero-superblock --force "${DISK_DST}2" 2>/dev/null || true

# /dev/md10 for /boot (metadata 1.0 puts superblock at partition end for BIOS/GRUB)
mdadm --create /dev/md10 --level=1 --raid-devices=2 missing "${DISK_DST}1" --metadata=1.0 --force
# /dev/md11 for LVM OS PV
mdadm --create /dev/md11 --level=1 --raid-devices=2 missing "${DISK_DST}2" --metadata=1.2 --force
udevadm settle

# ------------------------------------------------------------------------------
# 6. Format and Synchronize /boot to /dev/md10
# ------------------------------------------------------------------------------
echo "[6/10] Formatting and synchronizing /boot to /dev/md10..."
mkfs.ext4 -F -L boot_raid /dev/md10
mkdir -p /mnt/new_boot
mount /dev/md10 /mnt/new_boot
rsync -aHAX --delete /boot/ /mnt/new_boot/
umount /mnt/new_boot
rmdir /mnt/new_boot

# ------------------------------------------------------------------------------
# 7. Live LVM Migration (pvmove from sda2 to md11)
# ------------------------------------------------------------------------------
echo "[7/10] Live migrating LVM volume group fedora00 from ${DISK_SRC}2 to /dev/md11..."
pvcreate -ff -y /dev/md11
vgextend fedora00 /dev/md11
echo "[+] Starting live pvmove (this migrates all root, home, and active data online)..."
pvmove "${DISK_SRC}2" /dev/md11
vgreduce fedora00 "${DISK_SRC}2"
pvremove -ff -y "${DISK_SRC}2"
echo "[+] LVM live migration completed successfully!"

# ------------------------------------------------------------------------------
# 8. Remount /boot to /dev/md10 in /etc/fstab
# ------------------------------------------------------------------------------
echo "[8/10] Updating /boot mount in /etc/fstab to use RAID1 UUID..."
NEW_BOOT_UUID=$(blkid -s UUID -o value /dev/md10)
echo "[+] New /boot RAID UUID: $NEW_BOOT_UUID"
sed -i -E "s|^UUID=[^[:space:]]+[[:space:]]+/boot|UUID=$NEW_BOOT_UUID /boot|" /etc/fstab
umount /boot
mount /boot
echo "[+] /boot successfully remounted from /dev/md10!"

# ------------------------------------------------------------------------------
# 9. Configure Dracut, mdadm.conf & Regenerate Initramfs
# ------------------------------------------------------------------------------
echo "[9/10] Configuring dracut and regenerating initramfs..."
mkdir -p /etc/dracut.conf.d
echo 'add_dracutmodules+=" mdraid lvm "' > /etc/dracut.conf.d/raid.conf

mkdir -p /etc/mdadm
echo "DEVICE /dev/sd[a-z][0-9]" > /etc/mdadm.conf
mdadm --detail --scan >> /etc/mdadm.conf

CURRENT_KVER=$(uname -r)
echo "[+] Rebuilding initramfs for running kernel ($CURRENT_KVER)..."
dracut -f --add "mdraid lvm" "/boot/initramfs-${CURRENT_KVER}.img" "$CURRENT_KVER"

for vmlinuz in /boot/vmlinuz-*.x86_64; do
    KVER=$(basename "$vmlinuz" | sed "s/vmlinuz-//")
    if [ "$KVER" != "$CURRENT_KVER" ] && [ -d "/lib/modules/$KVER" ]; then
        echo "[+] Rebuilding initramfs for installed kernel ($KVER)..."
        dracut -f --add "mdraid lvm" "/boot/initramfs-${KVER}.img" "$KVER" 2>/dev/null || true
    fi
done

echo "[+] Updating GRUB configuration..."
grub2-mkconfig -o /boot/grub2/grub.cfg

# ------------------------------------------------------------------------------
# 10. Repartition Source Disk & Join to RAID 1 Arrays
# ------------------------------------------------------------------------------
echo "[10/10] Partitioning $DISK_SRC and joining into RAID 1 arrays..."
wipefs -af "$DISK_SRC"
dd if=/dev/zero of="$DISK_SRC" bs=1M count=20 conv=fsync status=none
parted -s "$DISK_SRC" mklabel msdos
parted -s -a optimal "$DISK_SRC" mkpart primary 2048s 2099199s
parted -s -a optimal "$DISK_SRC" mkpart primary 2099200s 468861951s
parted -s "$DISK_SRC" set 1 boot on
parted -s "$DISK_SRC" set 1 raid on
parted -s "$DISK_SRC" set 2 raid on
partprobe "$DISK_SRC"
udevadm settle

echo "[+] Adding ${DISK_SRC}1 to /dev/md10 (/boot)..."
mdadm --add /dev/md10 "${DISK_SRC}1"

echo "[+] Adding ${DISK_SRC}2 to /dev/md11 (LVM OS pool)..."
mdadm --add /dev/md11 "${DISK_SRC}2"

# Install GRUB Bootloader on BOTH Physical Disks
echo "[+] Installing GRUB bootloader to both drives ($DISK_SRC and $DISK_DST)..."
grub2-install "$DISK_SRC"
grub2-install "$DISK_DST"

echo "DEVICE /dev/sd[a-z][0-9]" > /etc/mdadm.conf
mdadm --detail --scan >> /etc/mdadm.conf

echo "=========================================================="
echo "    HIGH AVAILABILITY RAID 1 OS SETUP COMPLETE!           "
echo "=========================================================="
echo ""
echo "Current RAID Status (/proc/mdstat):"
cat /proc/mdstat
echo ""
echo "Active Mounts:"
df -hT / /boot /home
echo ""
echo "Both SSDs ($DISK_SRC and $DISK_DST) are bootable with GRUB."
echo "Synchronization will complete in the background."
echo "=========================================================="

🔍 Verification & Failover Readiness

Once the script completes, Linux synchronizes the arrays in the kernel background without consuming unnecessary CPU cycles.

1. Check Mirror Health

cat /proc/mdstat

When fully synchronized, both arrays will display [UU]:

Personalities : [raid1] 
md11 : active raid1 sda2[2] sdb2[1]
      233249280 blocks super 1.2 [2/2] [UU]

md10 : active raid1 sda1[2] sdb1[1]
      1048512 blocks super 1.0 [2/2] [UU]

2. Verify Detailed Array Health

sudo mdadm --detail /dev/md10
sudo mdadm --detail /dev/md11

Look for:

  • State : clean
  • Active Devices : 2
  • Working Devices : 2
  • Failed Devices : 0

3. Optional: Speeding up Initial RAID Sync

If your SSDs support higher sustained writes, you can temporarily increase the Linux kernel rebuild limits:

sudo sysctl -w dev.raid.speed_limit_max=500000
sudo sysctl -w dev.raid.speed_limit_min=50000

🏁 Summary

With this setup:

  • Storage High Availability: Your root filesystem (/), home directories (/home), and /boot continue operating seamlessly if either SSD fails.
  • Boot Redundancy: Both drives have independent Master Boot Records with GRUB installed.
  • Seamless Maintenance: Replacing a degraded SSD in the future requires only partitioning the replacement drive and running mdadm --add.
  • Zero Downtime: The entire conversion was performed live while web servers, databases, and Docker containers remained online.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.