- 1. Installation to USB<\/a>
- 1.1 Solution with USB<\/a>
- 1.1.1 Caveats with USB<\/a><\/li><\/ul><\/li><\/ul>
- 2. Installation to SSD<\/a>
- 2.1 Some observations <\/a>
- 2.1.1 Why do we even retry?<\/a><\/li><\/ul>
- 2.2 Solution with SSD<\/a>
- 2.2.1 How to run the script<\/a><\/li><\/ul><\/li><\/ul>
- 3. References<\/a><\/ul><\/div><\/div><\/div><\/div><\/div>\n\n\n
1. Installation to USB<\/h2>\n\n\n\n
It is no longer recommended to install the Nutanix AHV to a USB, while I could not find any official documentation of this, this is what Nutanix Employees are saying on Reddit and on their Discord (references will be linked at the bottom of the page).<\/p>\n\n\n\n
However, installing to USB also caused a timeout as it would appear to not be fast enough, regardless of USB 2.0 or 3.2. Maybe there is a driver “issue” that limits it to slow speeds regardless. Attempted Kingston USBs and SanDisk USBs, all timed out.<\/p>\n\n\n\n
![\"\"](\"https:\/\/binaryvikings.io\/wp-content\/uploads\/2024\/07\/image-1-1024x167.png\")
Nutanix Installation to USB timeout – current retry 0<\/figcaption><\/figure>\n\n\n\n A Reddit user told me to wait it out, as it’s not a “retry” that restarts the installation, it will indeed continue the installation. I left it running and at around 500\/2430 in retry 1, my installation was successful, on USB.<\/p>\n\n\n\n
1.1 Solution with USB<\/h3>\n\n\n\n
Do not terminate the installation after it has failed its retry, just let it run and eventually it will install.<\/p>\n\n\n\n
As an additional thing, if you can SSH to the machine or have possibility to access
\/tmp\/installer_vm.log<\/code><\/strong>, dotail -f \/tmp\/installer_vm.log<\/strong><\/code> and follow the actual installation progress.<\/p>\n\n\n\n1.1.1 Caveats with USB<\/h4>\n\n\n\n
Worth mentioning, again, Nutanix Employees have stated that they no longer recommend USB. <\/p>\n\n\n\n
If this is your only option, please make sure you understand that AHV updates (through LCM) is not supported (though can sometimes be done with some hackery).
And you will face USB storage driver issues, which you can read about here: Nutanix \u2013 Updating CE2 from LCM, server not coming back up<\/a><\/p>\n\n\n\n<\/p>\n\n\n\n
2. Installation to SSD<\/h2>\n\n\n\n
After having successfully installed on USB, I figured I wanted to retry installing AHV onto my SSD to avoid the caveats listed above.<\/p>\n\n\n\n
I started out having the installation run through the night and woke up to the screen below. “InstallerVM timeout occurred, current retry 4”.<\/p>\n\n\n\n
![\"\"](\"https:\/\/binaryvikings.io\/wp-content\/uploads\/2024\/07\/image-1024x656.png\")
Nutanix Installation to SSD timeout – current retry 4<\/figcaption><\/figure>\n\n\n\n This screenshot shows it reaching its final retry and still not finishing installation, the python “crash” is simply due to poor error handling, as there is no error messages if the installer reaches 4 retries without succeeding. I then thoroughly investigated the logs, to figure out what is going on. The log at \/tmp\/installer_vm.log<\/strong> clearly showed the installation being finished, having run post-installation scripts, however never showed me<\/strong> that it had finished and finally crashed on retries. <\/p>\n\n\n\n
Excerpt from the installer_vm.log <\/strong>file here:<\/p>\n\n\n\n
Installation\n\n 1) [x] Language settings 2) [x] Time settings\n (English (United States)) (UTC timezone)\n 3) [x] Installation source 4) [x] Software selection\n (Local media) (Custom software selected)\n 5) [x] Installation Destination 6) [x] Network configuration\n (Warning checking storage (Wired (enp0s2) connected)\n configuration)\n 7) [ ] User creation\n (No user will be created)\n\nProgress\nSetting up the installation environment\n.\nCreating disklabel on \/dev\/sda\n.\nCreating efi on \/dev\/sda1\n.\nCreating ext4 on \/dev\/sda2\n.\nRunning pre-installation scripts\n.\nStarting package installation process\nPreparing transaction from installation source\nInstalling libgcc (1\/517)\n... ((Commented out due to too big file))\nInstalling nutanix-ahv-release (515\/517)\nInstalling grub2-efi-x64 (516\/517)\nInstalling dosfstools (517\/517)\nPerforming post-installation setup tasks\nInstalling boot loader\n.\nPerforming post-installation setup tasks\n.\n\nConfiguring installed system\n.\nWriting network configuration\n.\nCreating users\n.\nConfiguring addons\n.\nGenerating initramfs\n.\nRunning post-installation scripts<\/code><\/pre><\/div>\n\n\n\nSo, the progress we are presented is not the actual<\/em><\/strong> installation progress.<\/p>\n\n\n\n
2.1 Some observations <\/strong><\/h3>\n\n\n\n
(As I’ve posted it on Reddit<\/a>). The installer starts an “Installer VM”.. So…<\/p>\n\n\n\n
- \n
- The installation takes place from the InstallerVM (from which we were reading logs doing tail -f \/tmp\/installer_vm.log<\/strong>)<\/li>\n\n\n\n
- The script runs a default 40 minutes, before retrying ((2400 seconds) + 30 seconds interval), [30\/2430], at a default of 4 retries.<\/li>\n\n\n\n
- To determine if the installation is successful, the loop checks if the InstallerVM is still on, if it’s on, it’s not done and runs another 30 seconds. If it’s done it says “Hypervisor installation is done, brother, we golden”.<\/li>\n<\/ol>\n\n\n\n
I noticed from my logs that my installation was indeed finished, all packages were transferred, but it never shut down the InstallerVM, hence never successful. But it appears<\/strong><\/em> some part of either the timeout or retry terminology in the script is making the InstallerVM “stall” or at least not shut down upon finishing. Not saying that’s definitely what happens, and it only failed this way when installing to SSD directly – but an observation nonetheless.<\/p>\n\n\n\n
2.1.1 Why do we even retry?<\/strong><\/h4>\n\n\n\n
According to comments in the installation script:<\/p>\n\n\n\n
# ESXi has a ~5% chance to stuck at writing to disk in QEMU, retries are helpful to recover.<\/pre>\n\n\n\n
Having changed too much of the script to dig deeper into what actually happens, my assumption is that some part of the “unstuck” fix for ESXi users are messing with us occasionally. My logs however reported nothing.<\/p>\n\n\n\n
I wonder though – How many Nutanix users are using ESXi? So in total of all Nutanix users, whether esx, hyperv, kvm or xen – how many % of all users combined is it really that had an issue that was solved with this retry?<\/p>\n\n\n\n
Additionally – The progress of 30 second interval increment up to a hardcoded of 2400 seconds, to then see a “failed” message after which it will continue and finish installation, masqueraded in a “retry” sequence seems like .. a… curious solution. Not hating, merely questioning – I would have loved to see the installer_vm.log output instead of the 2400 seconds.<\/p>\n\n\n\n
Oh well, maybe next release will measure [5\/290 miles per hour] — Obviously kidding.<\/p>\n\n\n\n
2.2 Solution with SSD<\/h3>\n\n\n\n
Welcome back from my observations\/rant.<\/p>\n\n\n\n
At this point I had removed some of the retry code of the installation script, while leaving as much as possible in the script to not hit other obstacles. (Why the script may also look a bit messy).<\/p>\n\n\n\n
I have chosen to share my installation file as I’m seeing so many poor, frustrated souls on Reddit and Discord, facing same issues as I did, hopefully this post will find its way onto Google. Unfortunately I’m seeing people who I know, now know, the solution, but misguiding these poor souls on Reddit and Discord.<\/p>\n\n\n\n
Link to GitHub script<\/strong>: https:\/\/github.com\/Oreax\/nutanixinstaller\/tree\/main<\/a><\/p>\n\n\n\n
2.2.1 How to run the script<\/h4>\n\n\n\n
The process is also written in the script itself, but for good measure I will post it here as well:<\/p>\n\n\n\n
- \n
- Once the installer GUI has popped up, press CTRL + C to exit the installer.<\/li>\n\n\n\n
- Type in the following command (optional – but easier for step 3):
service sshd start<\/code><\/strong><\/li>\n\n\n\n- Replace the following file with the contents of this file
\/phoenix\/imaging_helper\/installer_vm.py<\/strong><\/code><\/li>\n\n\n\n- Relaunch the installer (from server screen) in
\/root\/<\/code><\/strong> with:.\/ce_installer && screen -r<\/strong><\/code><\/li>\n\n\n\n- (Optional, but highly recommended) Follow the installer progress in SSH terminal with:
tail -f \/tmp\/installer_vm.log<\/strong><\/code><\/li>\n<\/ol>\n\n\n\n3. References<\/h2>\n\n\n\n
- \n
- Reddit (Nutanix): https:\/\/www.reddit.com\/r\/nutanix\/<\/a> <\/li>\n\n\n\n
- Discord (Nutanix): https:\/\/discordapp.com\/invite\/XkSDNWZ<\/a><\/li>\n\n\n\n
- Reddit post: https:\/\/www.reddit.com\/r\/nutanix\/comments\/1drxkr5\/setting_up_nutanix_ce2_failing_24022430\/<\/a><\/li>\n\n\n\n
- GitHub script: https:\/\/github.com\/Oreax\/nutanixinstaller\/tree\/main<\/a><\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
I hope this helps some of you out!<\/p>\n\n\n\n
Over’n’Out,<\/p>\n\n\n\n
me.<\/p>\n","protected":false},"excerpt":{"rendered":"
I had several incidents, attempting various installation media when installing Nutanix CE2. Both installing to USB and directly to SSD got timeouts and did not<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_eb_attr":"","footnotes":""},"categories":[12,34],"tags":[],"class_list":["post-452","post","type-post","status-publish","format-standard","hentry","category-guides","category-nutanix"],"_links":{"self":[{"href":"https:\/\/binaryvikings.io\/index.php?rest_route=\/wp\/v2\/posts\/452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/binaryvikings.io\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/binaryvikings.io\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/binaryvikings.io\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/binaryvikings.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=452"}],"version-history":[{"count":5,"href":"https:\/\/binaryvikings.io\/index.php?rest_route=\/wp\/v2\/posts\/452\/revisions"}],"predecessor-version":[{"id":472,"href":"https:\/\/binaryvikings.io\/index.php?rest_route=\/wp\/v2\/posts\/452\/revisions\/472"}],"wp:attachment":[{"href":"https:\/\/binaryvikings.io\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/binaryvikings.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/binaryvikings.io\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- Discord (Nutanix): https:\/\/discordapp.com\/invite\/XkSDNWZ<\/a><\/li>\n\n\n\n
- Replace the following file with the contents of this file
- 3. References<\/a><\/ul><\/div><\/div><\/div><\/div><\/div>\n\n\n
- 2.2 Solution with SSD<\/a>
- 2.1.1 Why do we even retry?<\/a><\/li><\/ul>
- 2. Installation to SSD<\/a>
- 1.1.1 Caveats with USB<\/a><\/li><\/ul><\/li><\/ul>
- 1.1 Solution with USB<\/a>