migrating email users

As I understand virtualmin/webmin solution it is only wrapper for doing rutines such as adding user (email or ftp) and doing all repeptitive tasks and sets of tasks for you in one large procedure. If I have to migrate as many users I should setup that domain regulary (because you are talking only about emails if I understand it right) and then I'll try to play with postfix configuration for this single domain. And if you have allready setuped postfix+mysql+dovecot (or other IMAP server) I think that you will have no problem with this. I don't know how virtualmin handles users in MySQL databasem, but I am sure that here on virtualmin.com will be some documentation about this. And this sounds that is your biggest production site - I will care of this site little bit special myself and test, test, test everything well. And maybe try to setup testing server for migration, because there is a lot of emails going through probably (so your secondary MX server should catch all of those not passed to migrated server in case of DNS replication tooks a long time for someone).

Anyway good luck and plan this well. And test it well ;o)))

<div class='quote'>we have 50K users on an old email server using postfix and using mysql for its authentication. I would like to migrate it on our new mail server which has virtualmin/webmin so far we're ok with domains which has 10-100 users using the virtual file but I doubt it for a domain with 50K users.</div>

So, I usually admonish folks to avoid introducing unnecessary complexity. And they often reply, &quot;But I have a <i>lot</i> of mailboxes!!!&quot; And, I reply, &quot;How many?&quot; And they reply, &quot;Over 500!&quot; And I laugh and laugh, because I know that 500 passwd entries will fit comfortably into a few kilobytes of RAM, and will end up being permanently cached in RAM and will be vastly faster than those same 500 users in any database.

But, in your case, you actually have enough users to where a database makes excellent sense. I would have gone LDAP, but I reckon MySQL isn't the worst possible way to store users.

Virtualmin supports both LDAP and MySQL user storage...but I've never used MySQL, and I'm not even sure if it works with Postfix in Virtualmin. But, if you're starting fresh, you might want to move to LDAP. I dunno.

Give a try to getting it running in the way you're most comfortable with, and come back to us when you get confused. If MySQL is the way you're most comfortable with, try it and we'll try to help you make it work. If you're eager to try something new, LDAP is quite well-supported and there's quite a bit of documentation on the topic. Just be patient, take it one step at a time, and let us know when you get overwhelmed and we'll try to step in and provide some guidance (I'll probably mostly defer to Jamie on providing that guidance, as he knows more about the LDAP and MySQL support than I do).

<div class='quote'>Joe can you please advise us how many users we should store in /etc/passwd file ? I will soon have many users (I think about thousands) for emails and FTP accounts. So could it be over 10k ? Or over 50k ?</div>

10k is probably still fine. I managed a system several years ago with about four thousand users in /etc/passwd, and it didn't have any trouble at all. It only had 2GB of RAM, I think, and was an old dual core Intel Xeon at about 1GHz.

Are things getting slow? The places you'd notice it would be when adding, editing, or removing users.

10,000 users would lead to an /etc/passwd of about 500 kilobytes. When loaded into a Perl data structure, it'd be about two or three times that size, so a few MB in RAM. Still quite simple to manage--I've built Perl apps that deal with far larger data in RAM without beginning to worry about doing more complex things like memory mapping or storing it in custom data structures written in C. The C data structures that PAM uses (linked list, probably, but maybe a hash table of some sort) would be even smaller, most likely. Pretty much everything on the system is going to be dealing with PAM for user data interactions, so, as long as PAM is fast, it should work fine.

So, you tell us when it starts getting slow, or acting funny, and then we'll know. It may even be more than 50k on modern hardware. 2.5MB of raw data is not all that intimidating--even if it balloons up to five times that in data structures within the software, it's still only 15MB. We aint breaking the bank on RAM here, or on what can be loaded into RAM quickly (and what can easily be permanently cached by the kernel on a system with reasonable amounts of RAM). ;-)

But, my main argument whenever anyone wants to introduce a database into a hosting system is, &quot;Think about whether you <i>need</i> that extra complexity to achieve what you want to achieve.&quot; Usually, the answer is no. Performance is not a benefit of moving to a database for users--plain text passwd storage is faster in every case I've ever personally managed, and if you think about it, it'll be obvious why. But, there are some cases where you need other tools. If you have many machines and they all need to share users...obviously you need a directory that all machines authenticate to. But many users is rarely a good reason to introduce that complexity, because it's just going to be slower and harder to configure and manage.

I just imagine there <i>must</i> be some point at which the standard passwd tools begin to breakdown, and loading the whole list into RAM for editing becomes a problem. I've just never run into it in person.