Emergency support request - all Cloudmin guest systems are gone

Submitted by yngens on Sun, 07/22/2018 - 07:49

Dear James,

We are experiencing probably the most serious trouble since we decided to switch to your software several years ago. One of our Cloudmin systems failed to come up after reboot and so more than a dozen guest systems running on it are inaccessible for already a week. Never before we failed our customers like this time and we need urgent assistance from you, guys, to get your own software running.

Please check your e-mail info@virtualmin.com and you will find detailed discussion on the issue between our team and our data-center staff-members, and also access information sent from info@drupion.com.

Please login to the system through IPKVM and get the system up. Note that possible source of the issue is that partition where KVM guest were saved was defined as /cloud in the very beginning, so it could be related to discussion on https://www.virtualmin.com/node/29243

Status: 
Closed (won't fix)

Comments

Submitted by yngens on Mon, 07/23/2018 - 14:01

It's real shame you, guys, could not respond to this sooner. James, please save our trust to Cloudmin or otherwise we will never ever come back to use any of your products. Lot's of our customers are awaiting for this issue to be resolved ASAP.

Submitted by JamieCameron on Mon, 07/23/2018 - 15:03

Can you post the login details here, or email them to me at jcameron@virtualmin.com ?

Also, if a physical system completely failed to start up, this is out of the scope of what Cloudmin can fix - but if it's the VMs that aren't starting, we can help debug that.

Submitted by yngens on Mon, 07/23/2018 - 18:17

Can you post the login details here, or email them to me at jcameron@virtualmin.com ?

What's wrong with info@virtualmin.com? Anyway, I've just forwarded conversation to jcameron@virtualmin.com.

Also, if a physical system completely failed to start up, this is out of the scope of what Cloudmin can fix - but if it's the VMs that aren't starting, we can help debug that.

Well, it's failed because network interfaces got mixed up after the kernel update. I am afraid Cloudmin is not capable to re-instate network interfaces if OS kernel gets updated.

Submitted by yngens on Wed, 07/25/2018 - 06:12

Hi Jamie,

We have to finally give something to our customers, so could you please update this one with any kind of information. Cloudmin is clearly failing in restoring network interfaces after the kernel update and eventually it should be fixed on Cloudmin's repository level, however in our case we need a temporarily solution as soon as possible. If you are not able to fix this, then please admit it as we can not wait forever and have to start looking for other venues.

Jamie, it's really high time for you to save our and your own reputation. We relied on your soft which is now terribly failing, so please find some time to sit on this. I do believe it can be fixed by establishing network for the main interfaces and then re-adding the bridge for the guest systems.

Submitted by yngens on Wed, 07/25/2018 - 17:25

Jamie, please give us something, anything.

Submitted by JamieCameron on Thu, 07/26/2018 - 00:27

Ok, I was able to login - sorry I didn't realize that you hadn't even gotten the machine onto the network and only had IPKVM access.

I was able to get the network online by running ifup enp5s0f0 and ifup br0 . Why this didn't happen at boot time I'm not sure , as normally all interfaces are brought up at boot.

You should now try starting up some VMs.

Submitted by yngens on Fri, 07/27/2018 - 12:39

Hi Jamie,

I was able to get the network online by running ifup enp5s0f0 and ifup br0 . Why this didn't happen at boot time I'm not sure , as normally all interfaces are brought up at boot.

I am afraid after the last OS update Cloudmin's network settings got broken. The system is down again after reboot. Could you please login through IPKVM again and make sure the network interfaces are stable and automatically come up after each reboot.

Please also consider to address this issue on Cloudmin's repository as we have already seocnd Cloudmin loosing all its network interfaces after the OS update.

Submitted by yngens on Sun, 07/29/2018 - 17:07

Status: Active ยป Closed (won't fix)

Making this private again and closing it as we have secured all the guest systems by migrating them away to another platform. However, please note this happened on two different Cloudmin systems, so I am absolutely sure Cloudmin can not handle properly OS upgrades, so you might want to make thorough tests on this issue.

Submitted by JamieCameron on Wed, 08/01/2018 - 20:57

Can you tell us more about this OS upgrade? What version of the kernel or distribution did you go from and to?

If an upgrade renamed network interfaces, this is kind of outside the scope of what Cloudmin can be expected to deal with on it's own (in my opinion).