# HowTo setup a 3-node Proxmox VE 7.1 cluster and configure it for High Availability on a single machine - Part 5 - for learning & testing purposes only In our past guides ([Part 1](https://blog.habitats.tech/howto-setup-a-3-node-proxmox-ve-71-cluster-and-configure-it-for-high-availability-on-a-single-machine-part-1-for-learning-and-testing-purposes-only), [Part 2](https://blog.habitats.tech/howto-setup-a-3-node-proxmox-ve-71-cluster-and-configure-it-for-high-availability-on-a-single-machine-part-2-for-learning-and-testing-purposes-only), [Part 3](https://blog.habitats.tech/howto-setup-a-3-node-proxmox-ve-71-cluster-and-configure-it-for-high-availability-on-a-single-machine-part-3-for-learning-and-testing-purposes-only), [Part 4](https://blog.habitats.tech/howto-setup-a-3-node-proxmox-ve-71-cluster-and-configure-it-for-high-availability-on-a-single-machine-part-4-for-learning-and-testing-purposes-only)) we covered a few concepts regarding Cluster, Node and Guest setup. In this guide we will summarise and consolidate what we discussed so far, prior to taking a dive into HA. Visit the [Corosync](https://corosync.github.io/corosync/) website to dive deeper into the technology PVE uses for Clustering & HA. > **Guest** = a CT or VM allocated virtualised hardware resources to run > **GID** = Guest ID (CTID or VMID) - a unique number > 100 across a cluster > **CT** = Container (LXC) > **VM** = Virtual Machine (KVM) > **FS** = FileSystem > **HA** = High Availability > **PVE** = Proxmox VE > **WAC** = Web Admin Console (the web browser GUI to administer PVE nodes & clusters) ### Nodes On a physical x86 PC (suggested 8 core / 16 thread, 32GB RAM, 1TB SSD) bare-metal install PVE 7.1 (FS =** ext4** the default) and name it: **\> pve-253 <** ---> IPv4: 192.168.0.253 | WAC: https://192.168.0.253:8006 <--- > note: the X86 PC **must** support virtualisation [PVE Admin Guide - System Requirements](https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_system_requirements). Read this guide: [**How to install PVE**](https://blog.habitats.tech/howto-configure-proxmox-ve-71-steps-to-success-update-final) On node pve-253 we create three VMs (suggested 4 Core, 8GB RAM, 256GB SSD) to host PVE 7.1 (FS = **zfs (RAID 0)**). The three PVE nodes are named: **\> pve-0 <** ---> IPv4: 192.168.0.10 | WAC: https://192.168.0.10:8006 <--- **\> pve-1 <** ---> IPv4: 192.168.0.11 | WAC: https://192.168.0.11:8006 <--- **\> pve-2 <** ---> IPv4: 192.168.0.12 | WAC: https://192.168.0.12:8006 <--- Read this guide: [How to create a 3-node PVE cluster](https://blog.habitats.tech/howto-setup-a-3-node-proxmox-ve-71-cluster-and-configure-it-for-high-availability-on-a-single-machine-part-1-for-learning-and-testing-purposes-only) On each Node, update the node repositories and save a Debian 11 ISO file/image and a Debian 11 CT Template. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644489779547/afSvsHlJ7.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644487783405/CSb-eLfyq.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644487949147/m-UXQSyY0.png) ### Cluster We then create the Cluster and name it: **# pvec-0 #** ---> IPv4: 192.168.0.10 | WAC: https://192.168.0.10:8006 <--- **# pvec-0 #** ---> IPv4: 192.168.0.11 | WAC: https://192.168.0.11:8006 <--- **# pvec-0 #** ---> IPv4: 192.168.0.12 | WAC: https://192.168.0.12:8006 <--- ### Guests On each of the three Clustered Nodes, let's create a CT and a VM based on Debian 11 to test a number of things. To keep a consistent environment we name Guests as follows. \# pve-0 # **> guest-ct-0 <** ---> GID: 100 | IPv4: 192.168.0.13 <--- \# pve-0 # **> guest-vm-0 <** ---> GID: 101 | IPv4: DHCP <--- \# pve-1 # **> guest-ct-1 <** ---> GID: 102 | IPv4: 192.168.0.14 <--- \# pve-1 # **> guest-vm-1 <** ---> GID: 103 | IPv4: DHCP <--- \# pve-2 # **> guest-ct-2 <** ---> GID: 104 | IPv4: 192.168.0.15 <--- \# pve-2 # **> guest-vm-2 <** ---> GID: 105 | IPv4: DHCP <--- ### Guests - CT ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491441067/YBdwHl7BF.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491465327/qAjmzJe4j.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491515323/XoOfWolJ0.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491532686/oFzaMwrpF.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491548859/bn5bvV2k-.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491603031/zNYvwX6sa.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491621126/8tbpMFu22.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491647122/4J45YwQMy.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491680765/s2Nqujqx5.png) Start the CT and there is nothing to install, Debian 11 server already running. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644493183700/h0vrxd-rC.png) ### Guests - VM ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491832772/d2EoioCdh.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491758485/VFNSWb3La.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644487152352/D5TbQZDP0.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644487190834/uAM-Q4U6G.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644487227958/Si8v7uYTv-.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644487267690/ZcQgxmorH.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644487302447/jIz0ouhEj_.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644491964438/GJfR4iCmq.png) Start the VM to install Debian 11. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644492085545/VDQNsJQxz.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644492271018/1qazPLqG4.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644492350286/caKlY-JM6.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644492412934/iNKf9KjSD.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644492747377/EH_MrfQYn.png) All done! Debian 11 server installed on VM. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644492868250/xTz2WrReZb.png) Repeat the above CT & VM installation for the other two Nodes (pve-1 and pve-2). ### Snapshots Let's create a snapshot for each of the Guests on Node pve-0. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644496704848/F2BoENJu9.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644497643365/COuAfXAGh.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644496784901/B4u56lxjl.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644496829304/ddo9eXgE-.png) We will now update the Debian 11 repositories for both the CT and the VM and then upgrade Debian. ```apt update``` ```apt upgrade``` ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644497168555/Ih7Bi2bRT.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644497302863/Y21NWtMPi.png) Now let's take another snapshot of the updated Guests (Debian 11 servers). > Note: the snapshot name **must** be a unique name. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644497602765/HhrGE5voi.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644497643365/COuAfXAGh.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644497836629/5XbPUAioX.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644497880697/IdfMpBTS6.png) Can you use the CT snapshot to jump back in time? Unfortunately NOT. You can only go back to the last saved snapshot. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644498071274/REYBvcw7s.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644498096301/0TNVXfpUv.png) Let's try rollback on the VM. Same here, cannot rollback at any point in time, only at the last snapshot. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644498146963/VxOv59VGM.png) Try to install anything on either the CT or VM with apt install (e.g. ```apt install php```). Once installed try ```php -v```. Now our VM running instance has been updated with PHP, while our last snapshot had no PHP installed. Let's see if we can rollback the running instance to our last snapshot. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644498548128/nON3t0LRO.png) Hooray!!! Yes we can rollback. Try ```php -v``` on the running instance again. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644498651322/ZSEPWYpy7.png) Snapshots are especially useful, when we are testing or developing something (an OS, an application, an appliance, a piece of software, etc). A snapshot allows us to go back to the last known good running instance. ### Clones When developing, the most useful tool is cloning, which allows us to experiment with versioning. A cloned Guest can be spun up/down at will. A clone creates a new Guest instance with all dependencies and the tech stack exactly the same as the ancestor. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644499205108/1v59I_xZC.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644499322323/SgVIzJq0c.png) Clones and snapshots are very useful in development environments. > Note: There is a limitation when cloning a VM based on a snapshot, if the FS = zfs. Read this [thread](https://forum.proxmox.com/threads/full-clone-feature-is-not-supported-for-drive-efidisk0-500.71511/) for more background information. **CTs do not have this limitation and can be cloned from any snapshot**. Snapshots in effect freeze the state of a Guest at a particular time, which we can then use with cloning to go back in time to that specific snapshot; it is like creating a milestone, which we can use to travel back in time. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644501098133/h2iulLR-u.png) Cloning and Snapshoting creates volumes in the background, like in our example. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644501247980/UDfnYxWdj.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1644501325105/NQ-FsezfT.png) At this point we have covered most of the topics surrounding clustering. Our next article will be purely focused on automated failover, now that we have the necessary basic skills. For those not afraid to dive deeper into things, here is one tip: Guest configuration information is saved in text configuration files. These files are in the following locations: > /etc/pve/nodes (the root for all Node and Guest configs) > /etc/pve/nodes/pve-0 (node SSL certs) > /etc/pve/nodes/pve-0/lxc (.conf files for CTs) > /etc/pve/nodes/pve-0/qemu-server (.conf files for VMs) Moving a Guest from node to node consists of actually moving the Guest * volume, and * config file --- Please consider subscribing to my blog, as you will only ever get quality content; no time wasting, advertising, spamming or other unproductive activities/practices. Please also consider visiting and subscribing to our YouTube channel; we have recently started posting videos. We are committed to improving and enhancing over time. If there is something you would like us to cover in a future topic/guide please let us know.