Mirror pve-root Using LVM RAID1

What this means:

You configure a mirror (RAID1) so that any write to pve-root is also written to sda1. If your NVMe dies, you can still boot from sda1.

🧱 Requirements:

  • sda1 must be equal to or larger than pve-root (96 GB in your case)
  • You must convert pve-root into a RAID1 logical volume (LVM mirror)
  • Some downtime or maintenance mode required

🧰 How-To (Overview Only):

  • Backup First! (Always)
  • Check current setup:
    • lvdisplay pve/root
  • Wipe and prep sda1:
    • pvcreate /dev/sda
    • vgextend pve /dev/sda
  • Convert pve-root to RAID1:
    • lvconvert --type mirror -m1 --mirrorlog core pve/root /dev/sda

This mirrors pve/root from your NVMe disk onto sda.

OptionMeaning
--type mirrorConvert the LV to a mirror (RAID1)
-m1Use 1 mirror copy = total of 2 devices
--mirrorlog coreStore mirror log in RAM
pve/rootThe logical volume to convert (your root)
/dev/sdaThe new disk to mirror onto
  • Confirm with:
    • lvs -a -o +devices
root@pve:~# lvs -a -o +devices
  LV              VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
  data            pve twi-aotz-- 794.30g             0.01   0.24                             data_tdata(0)
  [data_tdata]    pve Twi-ao---- 794.30g                                                     /dev/nvme0n1p3(26624)
  [data_tmeta]    pve ewi-ao----   8.10g                                                     /dev/nvme0n1p3(229966)
  [lvol0_pmspare] pve ewi-------   8.10g                                                     /dev/nvme0n1p3(232040)
  root            pve mwi-aom---  96.00g                                    100.00           root_mimage_0(0),root_mimage_1(0)
  [root_mimage_0] pve iwi-aom---  96.00g                                                     /dev/nvme0n1p3(2048)
  [root_mimage_1] pve iwi-aom---  96.00g                                                     /dev/sda(0)
  swap            pve -wi-ao----   8.00g                                                     /dev/nvme0n1p3(0)

  • Optional but smart: Update your bootloader (grub) to know how to boot from either disk:
    • update-initramfs -u
    • update-grub
    • grub-install /dev/sda

✅ Pros:

  • Real-time mirroring (RAID1)
  • Transparent failover if one device fails (bootable if configured)

⚠️ Cons:

  • Adds complexity
  • If misconfigured, can break boot
  • Doesn’t protect against file deletion or config mistakes (RAID is not a backup)

how to check the mirror status, detect failures, and know when to do maintenance:

Use lvdisplay for more detail:

lvdisplay /dev/pve/root

Look for:

  • Mirror status: OK (or similar)
  • If a device has failed, you’ll see something like “failed” or “inconsistent”

Suggested Disk Layout Strategy

Use CaseCurrent DiskSuggested Role
Proxmox Root FSnvme0n1✅ Keep for now (fast, wear low)
VM/LXC storagesdc1 (SSD)✅ Good, isolate high I/O loads
Backup / ISOssdb1 (HDD)✅ Archive/slow storage
Spare/Buffersda1 (SSD)⚠️ Could mirror root or use as L2ARC/ZIL (if ZFS)

2. Watch NVMe Write Wear Over Time

Your NVMe shows:

yamlCopyEditPercentage Used: 0%

That’s excellent — you’re still early in the wear cycle. But with Proxmox, check every few months using:

bashCopyEditsmartctl -a /dev/nvme0n1 | grep -i percentage

3. Add Log Management

To reduce wear:

  • Use tmpfs for /var/log (if RAM allows)
  • Limit journald persistence:
# /etc/systemd/journald.conf
Storage=volatile
SystemMaxUse=200M

4. Consider Backup OS Snapshots or Mirroring Root

Use sda1 to:

  • Mirror pve-root using LVM RAID1
  • Or just use it as a backup location via rsync or lvm-snapshots

Switching GPU Binding (Live) Toggle GPU Driver Script (vfio-pci ↔ nvidia)

A single NVIDIA GPU cannot:

  • Be passed through to a VM (via vfio-pci)
  • And be used on the host or in LXC at the same time

Why?

Because when you bind the GPU to vfio-pci on boot, it’s invisible to the host and cannot be used by NVIDIA’s kernel driver (nvidia.ko).

Switch Between VM and LXC Use (Rebind on demand)

If you don’t need both at the same time, you can manually switch the GPU between:

  1. Passthrough to VM (bind to vfio-pci)
  2. Use on host / LXC (bind to nvidia)

This lets you:

Then later give it back to the VM

Use the GPU for nvidia-smi or CUDA in an LXC container

here’s a single script that checks which driver is currently bound to your GPU, and automatically toggles between:

  • vfio-pci (for passthrough to VM)
  • nvidia (for use on host or LXC)
#!/bin/bash

# === CONFIGURATION ===
GPU="0000:0a:00.0"
AUDIO="0000:0a:00.1"
VMID=131         # Your Windows VM ID
LXCID=115        # Your LXC container ID using the GPU

# === FUNCTIONS ===

get_driver() {
    basename "$(readlink /sys/bus/pci/devices/$1/driver 2>/dev/null)"
}

unbind_driver() {
    echo "$1" > "/sys/bus/pci/devices/$1/driver/unbind"
}

bind_driver() {
    echo "$1" > "/sys/bus/pci/drivers/$2/bind"
}

switch_to_nvidia() {
    echo "→ Switching to NVIDIA driver (LXC use)..."

    echo "Stopping VM $VMID..."
    qm stop $VMID
    sleep 3

    echo "Unbinding GPU from current driver..."
    unbind_driver "$GPU"
    unbind_driver "$AUDIO"

    echo "Loading NVIDIA modules..."
    modprobe nvidia nvidia_uvm nvidia_drm nvidia_modeset

    echo "Binding GPU to nvidia..."
    bind_driver "$GPU" nvidia
    bind_driver "$AUDIO" snd_hda_intel

    echo "Starting LXC container $LXCID..."
    pct start $LXCID

    echo "✔ Switched to NVIDIA mode."
}

switch_to_vfio() {
    echo "→ Switching to VFIO (VM passthrough)..."

    echo "Stopping LXC container $LXCID..."
    pct stop $LXCID
    sleep 3

    echo "Unbinding GPU from current driver..."
    unbind_driver "$GPU"
    unbind_driver "$AUDIO"

    echo "Loading VFIO modules..."
    modprobe vfio-pci

    echo "Binding GPU to vfio-pci..."
    bind_driver "$GPU" vfio-pci
    bind_driver "$AUDIO" vfio-pci

    echo "Starting VM $VMID..."
    qm start $VMID

    echo "✔ Switched to VFIO mode."
}

# === MAIN ===

MODE="$1"
CURRENT_DRIVER=$(get_driver "$GPU")
echo "Detected GPU driver: ${CURRENT_DRIVER:-none}"

case "$MODE" in
    --to-nvidia)
        switch_to_nvidia
        ;;
    --to-vfio)
        switch_to_vfio
        ;;
    "")
        if [ "$CURRENT_DRIVER" == "vfio-pci" ]; then
            switch_to_nvidia
        elif [ "$CURRENT_DRIVER" == "nvidia" ]; then
            switch_to_vfio
        elif [ -z "$CURRENT_DRIVER" ]; then
            echo "⚠️ No driver bound. Defaulting to NVIDIA..."
            switch_to_nvidia
        else
            echo "❌ Unknown driver bound: $CURRENT_DRIVER"
            exit 1
        fi
        ;;
    *)
        echo "Usage: $0 [--to-nvidia | --to-vfio]"
        exit 1
        ;;
esac

# === FINAL STATUS DISPLAY ===
echo
echo "🔍 Final GPU driver status:"
SHORT_GPU=$(echo "$GPU" | cut -d':' -f2-)
lspci -k | grep "$SHORT_GPU" -A 3

Auto-toggle based on current driver

./toggle-gpu.sh

Force switch to NVIDIA for LXC

./toggle-gpu.sh --to-nvidia 

Force switch to VFIO for VM passthrough

./toggle-gpu.sh --to-vfio 

Bind proxmox Inteface to a specifi IP

To achieve this setup — where Proxmox’s web interface on port 8006 is only accessible via one specific NIC and not the other — you need to bind the Proxmox web GUI to a specific IP address.

Here’s how you can do that:


🔧 Steps to Bind Port 8006 and 3128(spice) to a Specific NIC/IP

  1. Identify the NIC/IPs: Run: bashCopyEditip a Let’s assume:
    • NIC1 (management): 192.212.5.245 — this should allow port 8006
    • NIC2 (isolated): 10.10.10.10 — this should block port 8006
  2. Edit Proxmox Web GUI service config: Open this file: bashCopyEditnano /etc/default/pveproxy
  3. Bind it to a specific IP (management interface): Find or add the line: LISTEN_IP="192.212.5.245"
  4. Restart the pveproxy service: systemctl restart pveproxy and systemctl restart spiceproxy This change will make the Proxmox GUI listen only on 192.212.5.245, and not on all interfaces.

✅ Optional: Confirm It’s Working

You can test by running:

ss -tuln | grep 8006

You should see:

LISTEN  0  50  192.212.5.245:8006  ...

And not 0.0.0.0:8006 or 10.10.10.10:8006.