example.com in Proxmox VE

Running the example.com domain in Proxmox VE

Let’s create a fully blown example.com domain on Proxmox including Certificates and e-Mail in less than 20 minutes! You can use this domain to test drive software or as a staging environment.

The blog article w/r to the certificates is here

The deployment scripts are on my github

Watch the video on YouTube

Click to view the entire transcript

(the use case) The example.com domain is a special domain that is owned by the Internet Assigned Numbers Authority, the IANA. It has specifically been designed for documentation purposes. What does that mean? Whenever you look at software for self-hosting, then you will find that the config files of that software usually are using the example.com domain. This is great. Everyone knows that they have to replace that with their own domain name. However – if you just want to test drive a Software and see if you like it or not, then you will have to go through a more or less painful process of using nano or vi and adapt the config files before you can do that. If the software needs certificates or uses e-mail, then you will need to handle that as well before you can even start it up. So I thought – why not just build a real example.com domain? Isolated – with its own network – here on my Proxmox server. This way I would not have to change config files or the like if I just wanted to test drive. Now – creating a domain as such is no rocket science. You just need an isolated network and a DNS Server. We’ve done that before on this channel. But there are three things that would be real game changers. First – have the whole setup in a ready-to use, repeatable or “canned” environment. Click and Go. Second – readily deployed certificates. And - last but not least – a fake e-mail server intercepting any mail and delivering it to a mailbox that can be used with your favorite e-mail client. As an entry point into that platform. I’ll use a container and install a graphical Linux environment that I can access with RDP. I will show later how to use a MACOS or a Windows Machine for that. You can use Linux but you don’t have to. DNS can be done by a small router VM that will also act as a Gateway to my real LAN and from there to the internet. For maximum Flexibility I’ll also add a Docker host inside the environment. The e-mail server is a Docker Container running on the Docker host. I’ll also put Portainer here and of course I’ll have certificates everywhere. I will create and distribute them at install time. Now all I need on the client is a Browser and a Mail client. I’ll use Firefox and Thunderbird for that. To make editing of config files easier, I’ll also add filezilla and vs code to the client. (Installation Instructions) So – here is how to build this. There is very few manual interaction required. You can find the required software and instruction on my Github Server. Log into your Proxmox GUI, go to the Network node and create a Linux Bridge. This will be our virtual network. Give it a name – for example vmbr111. Tick the boxes “Autostart” and “VLAN aware” – that’s it. Leave all others blank. Click on Create and then on “Apply Configuration” up here. Next – open a shell on the Proxmox PVE server either from the GUI or over ssh and copy paste the commands from the README on my Github Server. If you have git installed, then you can clone the repo, if not – download it as ZIP and unzip it. Do a cd into the subfolder where you have put the software and quickly review or edit the config file. Little Tip here – you may of course run nano or vi inside the shell. Or – you connect to your Proxmox using WinSCP or FileZilla and edit the file there with your favorite editor. I like to use Visual Studio Code for this. The config file contains very few settings. Basically the names of the network adapters that will act as WAN and LAN inside our environment, the name of the domain – for instance example.com, but you could use any other domain if you wanted. Next – the storage where the containers and VM will be put on your Proxmox, the name of the templates to use and a list of available Container or VM IDs. Last but not least a download URL where we can pull an OpenWrt image from. Cool – once you are happy with everything, just type ./deploy-sandbox.sh and the script should run. It will first give you a short summary and ask you to press Enter. You may stop it at any time by typing CTRL-C. Next, it’s asking for a password for the root user in the test box. Please do not use any characters here that might interfere with the script such as Dollar, Exclamation Mark, Quotes or the like – I do not do any filtering here. The password will not be shown as you type. Next, it asks for a non-root user – thats the name of the user that we will use to log in to the platform later. And the last question is the password for that user. Now you will see a lot of text fly by. The first thing that the script installs is the perimeter router. Once it has done that it will actually stop so that you can check if your router has internet access. We will need internet access for the next steps actually. So go to the Proxmox GUI again and open a shell on the newly created Router. Just type enter to activate the shell and see if you can ping www.google.com. That’s it. Go back to the script and press Enter. Next – the script will set up the containers. After it has done that, it will stop again so that you can double check if the containers have the right settings. Make changes as needed. The Client container should have two network interfaces. eth0 in your real LAN and eth1 in the virtual LAN. As I am using VLANs at home I can just add the VLAN tag. But that’s related to my specific config here at home. Back to the script, press Enter. Now it will run through unattended until the end and install all required components for you. At the end it prints out the IP address of the Client container. Just to be safe, restart the router and the two containers to make sure they’ve picked up everything. (How to use the platform) You can now connect to the client container by using let’s say Remmina from Linux or Remote Desktop Connection from Windows with the Non root user and password we specified. This is what you should see. A MATE desktop running on Debian. If you don’t like MATE and rather want xfce4 or cinnamon or the like, then just replace mate-desktop-environment with the one you want in the client init script. If you don’t like Linux at all, then just use any other VM and hook it up to the virtual bridge that you created in the beginning. You can then use it as a client. For now, let’s use this client. You may want to browse to the portainer interface of the docker host first. That’s https://exc-docker.example.com:9443 – You should be prompted by Portainer to create a new admin account. If this timed out for security reasons, then just either restart the docker host from the Proxmox GUI or log into it and type “docker restart portainer”. Specify a name and a password and then connect to the local environment. That’s it. We’re in. If you click on “Containers” then you should see Portainer, the Portainer agent and the imap server up and running. Please note that little padlock here – all TLS. If we examine the Certificate then you can see that this is actually a Wildcard certificate and a self-signed CA which the script has generated and also imported into the Firefox Browser so that you don’t get any nasty warnings here. I’ve made the certificates valid for ten years. Please only use them in this test environment. A copy of all the keys and certificates can be found in the /etc/certificates subfolder of the client and of the docker host. Now let’s check the e-Mail integration. I have installed Thunderbird as a mail client here. When you start it for the first time, then it will ask you for a Name, a Mail address and a password here. You are free to chose the name. For the mail address, type in bob@example.com. The password is literally just “password” – all minor letters. If you click on “continue”, then Thunderbird should actually find all the server names. Still – we need to click on “Configure manually” because the user name is not just bob but rather bob@example.com. Apply that by clicking on “Done” and it should go through. The IMAP and SMTP Servers do have TLS certificates from the same CA like Firefox has. The script has also deployed those certificates to the Thunderbird client. You can now send mail to any example.com mail address. Like for example alice@example.com. The imap server has a catch all mailbox. Let’s open that one as a second profile. Click on the top right menu, then “Account Settings”. Here under “Account Actions” click on “Add mail account”. Do the same steps like before. Just this time the mailbox and account are debug@example.com and the password is “debug” – all in minors. Whatever address you now send mail to, it will end up in this catch all mailbox here. I found this a great way to have e-mail without needing to go through the internet. The container that I am using by the way is a fork of Antonio Espinosas project on Github. He did all the heavy lifting. All I had to do was adapt the container to debian 11 and add some bells and whistles really. Antonio – many thanks for the great project. (Additional use cases) Awesome – that’s more or less it – please let me know in the comments if this is useful for you. Also – if you could think of any useful extensions, maybe a code server on the docker host or the like. Do leave me a comment on YouTube please. Thank you very much. (How to use Windows client) If you wanted to use a Windows or MAC client, then this is how you could do it. Let’s say I want to use this Windows 11 machine here. All I have to do is to go to the hardware node, select the Network Device, and change it to the virtual example.com bridge. When I now boot the VM, it will be inside the example.com Test box. If I now use let’s say Microsoft Edge to browse to the Portainer then of course I’ll get the certificate warning here, because I don’t have the CA installed. But no problem – we can easily download it from the web page itself. If you click on the “Not secure” icon here, then on the red warning, you see this little certificate icon up here? Click on it. That shows the details of the self signed certificate. Click on “Details”. Here we select the AAA_TestCA entry and click on Export. Now close that window and click on the three dots in the upper right corner. Then settings. Search for “cert” – now click on the “Manage certificates” link here. That will bring you to the Certificates applet. On Windows, the certificates are managed by the OS, not by the browser. Select the “Trusted Root CA” tab, then click on “Import”. Browse to the file that you have just downloaded and click through with Next-Next. You’ll get a security warning – confirm with yes – Now we can close everything. You’ll have to close and reopen Edge so that the changes are taken into account. If we browse to the site again – no more warning. Beautiful. Now for email. Add a new account, select the advanced setup. Then Internet eMail. Enter the information as shown on the screen, untick the SSL boxes – we are using STARTTLS. Click on OK. That should all go through. If you want to add the Catch All Mailbox then do the same thing again with the debug account. (MacOS) With a MacOS VM – same game – hook it up to the virtual network interface and boot. Once I had logged into MACOS, I used FileZilla to download a copy of the certificates from the Docker Host. The install script has placed a copy in /etc/certificates/example.com on the server. I then launched the Keychain App and imported the rootCA, the IMAP Full chain and the Wildcard full chain into the system node. I then did a double click on each one of them and set the Trust level to “Always Trust”. Now I could just add the email Account as before and also browse to the Portainer interface with no issues. (The network layout) Just a quick remark with regards to the IP address range of the test environment. By default, OpenWrt uses the 192.168.1.1 address. If you are using the same address range in your real LAN, then you would have to move the example.com environment to another subnet. In order to do this, you need to change the ROUTERIP setting in the config file and also run this one-liner on the router once it has come up, when the install script halts to check if the router has internet access. (Summary) Awesome – we now have a working example.com domain with an isolated virtual network, and all bells and whistles that we need in order to test drive software without having to change too much in config files etc. Here are just some more ideas what you could do with it. Maybe you want to have two sites and test things like wireguard connections between the sites. In this case you could just spawn the example.com domain twice, with different container ID’s of course and have the routers talk to each other or port forward or the like. Another idea I had was to actually use a second environment with my own domain name as kind of a staging environment. That means, I could test in the example.com domain and then see what I need to change in order to run it in my own domain. The nice thing with Proxmox is that you can take a backup of a container or VM and then restore it with a different ID. I could therefore test everything let’s say with container ID 100 in the example.com domain, take a backup and restore it with let’s say container ID 200 and hook that one up to the production environment. Or I’d do it the other way round. If I want to test something on a production container, then I would just back it up and restore it in the staging environment and run my tests on the copy. Once I’m happy with everything, then I could do the changes on the production container. This way I would have minimum interruption in the real environment and achieve a good Family Acceptance Factor. That’s all I wanted to show you today. Thank you so much for watching. Please do not forget to leave a comment and a like. Stay safe, stay healthy, bye for now