I installed PHP 7.3 and now Apache won't start. It won't start with this error:
httpd: Syntax error on line 1295 of /etc/httpd/conf/httpd.conf: Expected but saw
I've started going through the file, but there are a couple of hundred virtual servers on this server. I've already found some issue with mismatching opening and closing tags, but this is going to take some time. It seems to me that Virtualmin has completely bollocksed (am I allowed to say that?) this file.
I'm opening this ticket to start getting help and guidance, but my searches have only found on instance at https://www.virtualmin.com/node/33943 , but a search inside this system didn't show any tickets opened as a result of that. I'm also not using Coudmin.
I just followed the instructions at https://www.virtualmin.com/documentation/web/multiplephp , and now I have this problem when trying to get Virtualmin to recognise the new installation.
Craig
Comments
Submitted by craigh on Fri, 06/18/2021 - 18:37 Pro Licensee Comment #1
Is there a way to automatically rebuild /etc/httpd/conf/httpd.conf?
Submitted by craigh on Fri, 06/18/2021 - 18:42 Pro Licensee Comment #2
I have at least 14 anomalies where VirtualHost contains are not closed. Just off the top of my head from memory (because it's been so long since I configured this by hand), I don't believe these can be nested.
Submitted by craigh on Fri, 06/18/2021 - 18:46 Pro Licensee Comment #3
Yeah, that last question ("Can Virtualmin automatically rebuild /etc/httpd/conf/httpd.conf?") is really the key here. Anyone have any idea?
Submitted by craigh on Fri, 06/18/2021 - 18:48 Pro Licensee Comment #4
Just to be clear, I did not manually edit this config file. Virtualmin seems to have broken it by trying to add PHP 7.3 to the VirtualHost configurations.
Submitted by craigh on Fri, 06/18/2021 - 18:58 Pro Licensee Comment #5
There is nothing at Webmin -> Servers -> Apache Webserver that would suggest that Virtualmin can automatically rebuild the borked file.
Submitted by craigh on Fri, 06/18/2021 - 19:06 Pro Licensee Comment #6
Would there be an automated back-up of this particular file somewhere? I may need to retrieve one from a deeper and more complicated location, but if I knew there was one created by Virtualmin somewhere, that would sure help.
Submitted by craigh on Fri, 06/18/2021 - 20:33 Pro Licensee Comment #7
Eric, Jamie,
Is there documentation on what lines Virtualmin adds (and possibly removes) to (and from) the httpd.conf file when trying to reconfigure it after adding PHP 7.3?
Craig
Submitted by craigh on Fri, 06/18/2021 - 21:11 Pro Licensee Comment #8
I restored an old copy of the file, and Apache is again running. I have not used Virtualmin to "Re-Check Configuration", as I need to analyse the differences between the working file and the corrupted one before I proceed.
However, answers to the above questions would be helpful in that regard. Thanks.
Craig
Submitted by JamieCameron on Sat, 06/19/2021 - 01:19 Comment #9
Would it be possible to attach the httpd.conf file after it was corrupted and after you fixed it to this bug report? I'd like to see the diff, and from that figure out what Virtualmin did to break it.
Submitted by craigh on Sat, 06/19/2021 - 14:07 Pro Licensee Comment #10
Jamie,
Yes, I suppose so. I did a diff to see the differences between the old and new files and, while I noticed one pattern, it was just too much for me to deal with at the time. I have yet to find time to get the newer version of PHP working on the server.
So are you suggesting that there is no way to have Virtualmin rebuild the httpd.conf file?
Craig
Submitted by craigh on Sat, 06/19/2021 - 14:18 Pro Licensee Comment #11
Oh, and in my panic yesterday I didn't notice that this system had stripped out the stuff between the angular brackets. This was the error:
httpd: Syntax error on line 1295 of /etc/httpd/conf/httpd.conf: Expected </VirtualHost> but saw </Directory>
But it's in the subject anyway.
Submitted by JamieCameron on Sat, 06/19/2021 - 15:02 Comment #12
Virtualmin doesn't have a way to rebuild the whole httpd.conf , as some settings for each virtual server are stored in the config file. We could offer a feature to re-generate a domain's virtualhost block, but that wouldn't work if the end of the block itself is corrupted as in your case.
Submitted by craigh on Sat, 06/19/2021 - 15:13 Pro Licensee Comment #13
I see. I don't know that "the end of the block itself is corrupted". It's not as if the end of the file is truncated in the middle of a directive or that it ends in a mass of control characters. It's hard to know what actually happened if, as I say, there seems to me 14 instances where VirtualHost directives are closed or opened where they shouldn't be, which is pretty much what the error message alludes to.
So are you say you "could offer a feature to re-generate a domain's virtualhost block" in a future version, or is there some way I can attempt it now?
Submitted by JamieCameron on Sun, 06/20/2021 - 11:24 Comment #14
You could disable the website for the domain, then enable it again. However, this will delete any redirects or per-directory PHP versions you have setup.
Submitted by craigh on Sun, 06/20/2021 - 11:55 Pro Licensee Comment #15
That's not going to work, as I'd have to do it manually a couple of hundred times and, as you say, I'd then have to go back and fix everything that was deleted.
What does Virtualmin do when I "Re-Check Configuration"? After I had installed PHP 7.3, I twice tried to do that; the page started to load, and then sat there for ages (several minutes anyway) with the Virtualmin progress bar going back and forth. Eventually that even stopped. Same happened the next time I tried. I then rebooted the server and that's when Apache wouldn't start.
Did the "Re-Check Configuration" do the damage?
Submitted by craigh on Sun, 06/20/2021 - 12:13 Pro Licensee Comment #16
I'm wondering if perhaps my interrupting the "Re-Check Configuration" twice caused the problem.
Submitted by JamieCameron on Sun, 06/20/2021 - 14:12 Comment #17
Not unless you ran it twice at the same time from different SSH sessions?
Submitted by craigh on Sun, 06/20/2021 - 15:56 Pro Licensee Comment #18
No, as I described above, I only ran it in the web-based control panel, but the output stopped there leading me to try again.
What is the command to do it via SSH, and will there be progress output so that I don't interrupt the process?
Submitted by craigh on Sun, 06/20/2021 - 17:09 Pro Licensee Comment #19
I found https://www.virtualmin.com/documentation/developer/cli/config-system (
virtualmin config-system
), but that doesn't seem to be it.Submitted by JamieCameron on Sun, 06/20/2021 - 20:30 Comment #20
Via SSH, you can run
virtualmin check-config
Submitted by craigh on Sun, 06/20/2021 - 21:27 Pro Licensee Comment #21
Oh, but according to https://www.virtualmin.com/documentation/developer/cli/check_config , "Unlike the Re-check Config page in the Virtualmin web UI, it will not perform any system changes triggered by configuration changes ...." So what would be the point of running this?
Submitted by JamieCameron on Tue, 06/22/2021 - 01:47 Comment #22
That doc is no longer accurate - the
check-config
command does update some configuration files if needed (ie. if Apache is configured to use a PHP version that's not installed).I have updated the docs.
Submitted by craigh on Sun, 06/27/2021 - 18:56 Pro Licensee Comment #24
I had to wait for the weekend to try and make progress on this, in case Apache went down again.
As suggested, I ran
virtualmin check-config
at the command line. It took about 20 minutes to complete (while it told me what it was doing), and httpd.conf grew by about 21 kB. Afterwards I ranapachectl configtest
and gotSyntax OK
. Inside Virtualmin (on the web, as opposed to on the command line) the newer version of PHP (7.3) was showing, as I believe it had previously. So I restarted Apache. Everything worked as expected.I would say that the lack of output from System Settings -> Re-Check Configuration the first time around -- described in message 15 -- led to me interrupting the loading of that page and trying to reload it a second time, and perhaps that's what caused httpd.conf to be borked. So I realise I can be a little impatient, but usually Virtualmin does a decent job of providing output to let me know that a process in underway. In this case it didn't, which led me to take actions which likely caused the problem.
Jamie, in message 9 you asked for copies of the corrupted httpd.conf file and what it looked like after I fixed it. I didn't fix it --
virtualmin check-config
did -- and a quick scan of the diff shows that these lines were added, for each virtual server:If you want the full files I can provide them privately if you wish.
So this issue has been solved. Now I'll carry on with what I started last week and install PHP 7.4 and probably 8.0.
Submitted by craigh on Sun, 06/27/2021 - 18:58 Pro Licensee Comment #25
I forgot to obfuscate a user name. Obviously the user names changed between virtual servers.
Submitted by craigh on Sun, 06/27/2021 - 19:01 Pro Licensee Comment #26
I should add that in the output of
virtualmin check-config
was the following:Should I be concerned about this?
Submitted by craigh on Sun, 06/27/2021 - 22:46 Pro Licensee Comment #27
Installing PHP 7.4 and 8.0 also went well, with using
virtualmin check-config
on the command line in between. I didn't try to use System Settings -> Re-Check Configuration in the web-based control panel again.Submitted by JamieCameron on Fri, 07/02/2021 - 20:28 Comment #28
No, that warning message about
$salt
is totally harmless.