Hello,
I recently bought and installed Cloudmin. Installation was a little bumpy on Ubuntu 10.04, but I managed to get around the bumps of the installation itself. I had 1 problem where it tried to install libmd5-perl, but that was a virtual package. Following a guide here http://www.kelvinwong.ca/2010/05/22/installing-webmin-on-ubuntu-server-1... I was able to get around the problem.
After that, I started doing replication. Couple issues there I reported in an issue ticket here: https://www.virtualmin.com/node/15074
Temporarily got around that problem by patching in the virtualmin_ui functons from vui-lib, which is mentioned in the issue ticket, but I'm down to one problem remaining for me to actually make use of it as-is for now.
I'm trying to do only partial backups since I'm using shared storage such as glusterfs or NFS, and the issue is:
Backing up virtual servers on source system .. .. backup failed : Starting backup.. A new format backup can only be done when the home directory is included Backup failed!
I've read in a previous message that using old format backup would resolve this, but there's no such option that I'm seeing for that in Cloudmin for Physical Servers version 4.6, and basically I followed the documentation as per https://www.virtualmin.com/documentation/cloudmin/vm/vsync for selecting specific advanced options in the backup.
I wanted to do Global backup, because I wanted the entire server to be replicated to another for load balancing.
Any input on this matter?
Eric
This is just the sort of thing the support ticket tracker is for. (You've paid for premium support, you're encouraged to use it when you need it.) ;-)
But, I'll give Jamie a head's up to take a look at this thread, as I'm not sure where to start on a solution. It's pretty deep Virtualmin code stuff, and we definitely want to make it easy to use shared/distributed storage, so this is very bug-like all around.
--
Check out the forum guidelines!
Yep.. I put in a support ticket and a bug ticket as well, to no avail, so I'm getting a little upset. Especially since the replication is exactly why I paid for Cloudmin, and is the one thing that is totally and completely broken.
I appreciate the response on this, at the least now.
The support ticket is at http://www.virtualmin.com/node/15091
We can continue this subject there if you wish.
Currently, Cloudmin always uses the "new-style" backup format for replication, which doesn't allow you to exclude the home directory, and thus causes the error that you saw. However, it would be possible to change this, or make the format option if needed.
In fact, you could manually fix this right now by editing /usr/share/webmin/server-manager/vsync-lib.pl and changing line 189 to :
my @args = ( [ "newformat" ],then running
/etc/webmin/restartLet us know if that works for you, as a work-around.
''
Erm... That's exactly what line 189 already looks like, exactly.
With this "new-style" format also as you describe, that makes this documentation http://www.virtualmin.com/documentation/cloudmin/vm/vsync inaccurate.
Sorry, I copied that wrong - the line should be :
my @args = ( [ "separate" ],I will make this flag fully automatic in the next release..
''
Still no good.
The output of the manual attempted replication process is as follows:
Starting replication from cweb1.hostdruids.com of virtual servers domain1 , domain2 ..
Replication failed - see the output above for the reason why.
The options specified for the backup are:
Virtualmin global settings to replicate: Module configuration, FTP directory restrictions, Server templates and plans, Custom fields, links and shells (everything else was off)
Virtualmin features to replicate: Administration user, BIND DNS Domain, Apache Website, SSL Website, Webmin login (all other features disabled)
That's with the change from newformat to seperate.
Oops, I forgot that you need to also change line 236 to :
my @rargs = grep { $\_->[0] ne "dest" && $\_->[0] ne "newformat" && $\_->[0] ne "separate" } @args;''
Getting closer, but not quite. I changed the line that is on 236 as suggested and the error output is:
Restoring backups on destination systems .. .. 0 restores succeeded, 1 failed
Failed to restore on cweb2.omitted : Checking for missing features .. .. all features in backup are supported Starting restore.. Extracting backup archive files .. .. done Backup does not contain virtual server information for missing domain domain2. Restore failed!
Note that cweb1 has domain1 and domain2, but cweb2 currently only has domain1, not domain2. As I understand, it's supposed to create it if it doesn't yet exist, so this is still a problem apparently.
What do you have selected in the "Virtualmin features to replicate" list?
''
The same ones I said earlier:
Virtualmin global settings to replicate: Module configuration, FTP directory restrictions, Server templates and plans, Custom fields, links and shells (everything else was off)
Virtualmin features to replicate: Administration user, BIND DNS Domain, Apache Website, SSL Website, Webmin login (all other features disabled)
Ok, I see why this happens now - the replication UI isn't allowing you to select the special "virtual server configuration" feature, which is needed to re-create missing domains.
I will attach a fix for this to your related bug report.
''
Excellent. That was what it was. Since that patch it is now fully replicating as expected.
So far in a 2-domain setup it's functional. A little slow while watching it manually run, which means after I get a few domains and sub-domains (virtual servers and sub-servers), going, the schedule may have to be adjusted, which is the only lasting concern of mine remaining.
For now, I'm at least satisfied though that it's working. What I'm curious about is what are some of the goals still for improving this? If I were, for example, running a hosting service, I'd want all domains on each virtualmin cluster to be replicated within short periods of time, say like 5-10 minutes, because in such an environment, 1 hour is near a lifetime.
Case example:
Running two Virtualmin servers replicating the master to 1 to 2 other hosts. Router system using LVS to Round-Robin load-balance to all three simaltaneously for all http and https traffic. Client makes a change on the Virtualmin on node1. Their changes/additions aren't going to show up on the remaining 2 nodes until the replication occurs.
Question 1: How will the client know the process has been done besides just hitting their site until it works every time?
Idea 1: Possibly keeping a status of each replication status to let them know the process is pending, processing, or up-to-date.
Question 2: What can be done to keep replication polling times down and everything up-to-date within the replication cluster?
Idea 2: Keep timestamp tags or something, so that the status of changes can be identified and only those that need it be replicated. This would allow a quick query to say, "hey, anyone got any new stuff compared to the rest of the cluster?"
Question 3: Will there be a way to setup essentially multi-master replication, so that any node can be modified and the change replicates based from the newest to the older from any point?
Idea 3: By using idea 2, you could also implement this as well.
Currently the replication feature is quite new, so I am always looking for new ideas for additional capabilities..
The initial design was more to automate backups, so that if a master server goes down you have a complete backup elsewhere that is relatively recent. It wasn't designed to support load balancing, as that becomes much more complex due to the need to deal with data consistency. Moving the home directory and DBs to shared storage gets you most of this, but as you've seen Cloudmin still has some issues to resolve before this works smoothly..
''
Oh yeah, and not mentioned in the previous reply.
How can I get it to just replicate every single server, instead of just selected ones?
I see this post is old now, but (on Virtualmin GPL 3.84) if I select to backup ONLY "Contents of server's MySQL databases" I still ge the error "A new format backup can only be done when the home directory is included"
workaround is to also enable "Server's home directory and web page". will this change in the future?
thanks!
The correct solution here is to select the "One file per server (old format)" backup format, which allows you to include MySQL databases without any other features.
''