Peenie Wallie

Web Server Crash of Feb 16, 2008

My web server crashed today in a big way. Jennifer and Allie were having a recording session with a USB microphone in my home office when the server freaked out. I'm not sure what happened exactly, but when they called me into the office, the monitor had diagonal lines on the screen. I shut it down and tried to reboot it but it never came back up.

The lights would come on., but it would never boot up. I tried accessing the website www.peeniewallie.com, but could not get a response. So, I knew I had a major problem. Cold booting it several times didn't solve the problem.

Running Apache on the Gateway P600

I had previously installed Apache on the Gateway PC, to serve as an emergency fallback when I had problems with the main server. So, I ran Apache from the console:

• Start - Programs - Apache HTTP Server 2.0.59 - Control Apache Server - Start Apache in Console.

This got Apache working. I found the index.html file it was serving at the following location:

• C:\Program Files\Apache Group\Apache2\htdocs\index.html

What I found odd was that the server had crashed 1 year ago to the day (Saturday Morning February 17th, 2007.)

I edited the index.html file first to say PeenieWallie was again experiencing technical difficulties.

Scavenging the Hard Drives

I had recently copied all of my Word documents over to the TeraStation (Z:\My Documents\_My Stories). It's sort of an odd directory to put them in, but that's where they are. So, I wasn't too concerned about losing my MS Word documents. But, I wanted to get the web site up and running in a big way.

So, I decided to see if the drives had crashed or not.

I opened up the Dell server My crashed Dell server had two hard drives in it:
• Western Digital WD1200JB - 75CRA0 Caviar Enhanced IDE hard drive(Manufactured in Sep 2002).This drive apparently came with the computer.
• Maxtor DiamondMax Plus 9 200GB 7200 RPM with an Ultra ATA/133 PCI Controller/adapter card(Purchased from CompUSA 01/05/2003).

Western Digital Drive Data

I took the c: drive (Western Digital) and hooked it up inside the only Gateway Pentium 600 running Windows 2000 because the cables looked like they would work. I would have stuck it in Jennifer's new Dell XP Pro PC, but her cables didn't look right. I think her drive is a SATA drive? Dunno for sure, but cables are totally different. Since it's an IDE drive, it fit into the Gateway no problem. I didn't change the little jumper on the drive or anything. I didn't do jack except hook the cables up and I have it hanging inside the PC right now.

I booted up the Gateway and saw the drive as an F: drive with all the data intact. So, at this point, I knew that my C: drive was OK, which meant that the server crash was probably not caused by a drive failure. If my C: drive was working, then even if my other drive crashed, I doubt that would cause the Dell PC server not to boot up. So, the good news is, it looks like I probably won't lose any data. The server crash may turn out to be a good thing if it causes me to do a better job of archiving the web site data and setting up a fall over option onto another PC.

Configuring Apache (Editing the Apache httpd.conf file)

First, I tried to point the DocumentRoot to the WD drive:

• Start - Programs - Apache HTTP Server 2.0.59 - Configure Apache Server - Edit the Apache httpd.conf Configuration File
• This edits the file: C:\Program Files\Apache Group\Apache2\conf\httpd.conf
• Changed the DocumentRoot from "C:/Program Files/Apache Group/Apache2/htdocs" to "F:/inetpub/wwwroot".

But this was pretty ugly. Eventually, what I decided to do was to just start copying the files over from the F: drive (my old C: drive from the crashed server) onto the Gateway C: drive. My plan was to make the Gateway look almost identical to the old crashed server. So, this meant that I would eventually copy over the following directories (and all subdirectories) onto the Gateway:

• C:\inetpub\wwwroot\*.* (web site html files, images, scripts, etc.)
• C:\Perl\*.* (Active Perl 5.8.6)
• C:\PHP512\*.* (PHP)
• C:\MySQL\*.* (MySQL)
• C:\usr\*.* (Active Perl 5.8.6)

Cannot copy file: Access is Denied. The source file may be in use.

When I was copying over these files, the copy kept failing when Windows 2000 would intermittently hit a file that it didn't think it had permission to copy (like Jennifer2.zip, for instance) and would balk with the following error message:

This is odd for a number of reasons.
1. It drops the file extension in the error message.
2. It only happens on certain files most files it copies over with no problem.

So, copying files off the drive is tedious, and I searched on this message many times. It has something to do with the Permissions as they were defined under XP, but now that Windows 2000 is trying to copy files off of an XP/NTFS drive.

The solution has something to do with changing the permissions of the file, but what I don't know about file permissions would fill volumes. To fix the permissions, there are a couple of ways that will work. I'm not knowledgeable enough to know what's the best way. But, basically, you have to fiddle with the file's security:

1. Right-click on the file.
2. Select Properties.
3. Select the Security tab.

Oddly enough, although it says it won't let you copy it (and it won't - trust me - I tried), it will let you take ownership of the file. Why? Who knows. This is roughly the equivalent of you going into a police station and trying to read a book there. "No. I'm sorry. You can't read this book." The policeman tells you. "…unless it's your book. Is it your book?" "Yes. It is." "Oh. OK. Then you can have it. Good day."

4. Click on OK button.
5. Click on the Advanced button.
6. Click on Owner tab.
7. Under the "Change Owner To" section, select your UserID.
8. Click Apply.
9. Click OK.

You may need to go back into the file's security properties more than once. It's confusing to me, I can assure you.

1. Right-click on the file.
2. Select Properties.
3. Select the Security tab.
4. Uncheck the box that says "Allow Inheritable Permissions from parent to propagate to this object."
5. Click Remove.
6. Click Add.
7. Select a User from the list ('Everyone' works).
8. Click Add (again).
9. Click OK.
10. Under Permissions, click "Allow Full Control".
11. Click Apply.
12. Click OK.

It's painful, but this does work. That's the good news. The bad news is that you'll have to do this every time you hit a file that causes a problem.

A faster way to do this might be if you go up a level or two and attack the permissions at the folder level. It is possible to take control of a folder higher up, go to the security, change the owner to be you, and select the little "Replace owner on subcontainers and objects". This option is only available if you're dealing with folder security, not the security of an individual file. It whirs and grinds for a while, and appears to fix the security of all the objects underneath the folder.

In any event, with much cursing, I finally got all files copied over from my old drive onto the Gateway.

• C:\inetpub\wwwroot\*.* (web site html files, images, scripts, etc.)
• C:\Perl\*.* (Active Perl 5.8.6)
• C:\PHP512\*.* (PHP)
• C:\MySQL\*.* (MySQL)
• C:\usr\*.* (Active Perl 5.8.6)
• http.conf

Maxtor Drive Data

I took the other (Maxtor) drive and controller and shoved it into Jennifer's new Dell XP Pro PC. The PCI card snapped into her motherboard no problem. I didn't change the little jumper on the drive or anything. I just hooked up the cables and I have it hanging inside the PC right now. The Maxtor didn't work right away, however. The PC balked at the new drive and wanted something more. So, I went down into the basement and found the box and grabbed the Maxtor CD with the drivers. XP was asking where the drivers were, so I just shoved the CD in and browsed around until I found them…they were labeled for XP, as I recall. That worked and then I could see all the files on the Maxtor drive using Jen's PC.

The first thing I wanted to copy off of the Maxtor drive was the data used by MySQL. I copied this over without much difficulty to the TeraStation.

PHP Problems

I had some problems with PHP. So, I downloaded and re-installed PHP5.1.2.

MySQL Problems

Went rounds and rounds with MYSQL. Finally figured out I had to do this:

C:\MySQL\bin>mysqld --install
Service successfully installed.

C:\MySQL\bin>net start mysql
The MySQL service is starting.
The MySQL service could not be started.

A system error has occurred.
System error 1067 has occurred.
The process terminated unexpectedly.

But this is worlds better than where I was.

I also figured out along the way that I probably need to edit the c:\mysql\my.ini file. For some reason, the MySQL data sort of half points to c:\MySQL and z:\MySQL. Not sure why this is, really.

http://forums.mysql.com/read.php?11,11388,16051#msg-16051

C:\MySQL\bin>mysqld --remove MySql
Service successfully removed.

C:\MySQL\bin>mysqld --install MySql --defaults-file=c:\mysql\my.ini
Service successfully installed.

C:\MySQL\bin>net start mysql
The MySql service is starting.
The MySql service was started successfully.

C:\MySQL\bin>mysql -u root mysql -p
Enter password: ********
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3 to server version: 4.1.10

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

If MySQL is not running, then when I try to go to http://www.peeniewallie.com/mt/mt.cgi with my browser, I'd get the following error message:

Got an error: Bad ObjectDriver config: Connection error: Can't connect to MySQL server on 'localhost' (10061)

All I have to do now to start the service is:

C:\>cd mysql
C:\MySQL>cd bin
C:\MySQL\bin>net start mysql
The MySql service is starting.
The MySql service was started successfully.

But, I probably need a cleaner way to do this.

Hosting Multiple Web Sites with Apache

I accidentally wiped out my most recent version of httpd.conf, and therefore lost the changes I had made to allow my PC to host multiple websites. So, www.killingstrangers.com was going to www.peeniewallie.com.

Took me a while to figure this out, but in the end, I made these changes to my httpd.conf file (again).

#NameVirtualHost *:80
########################################
# RAK - 2/16/08 - Uncommented the NameVirtualHost line.
########################################
NameVirtualHost *:80

# VirtualHost example:
# Almost any Apache directive may go into a VirtualHost container.
# The first VirtualHost section is used for requests without a
# knownserver name.
#
#
# ServerAdmin webmaster@dummy-host.example.com
# DocumentRoot /www/docs/dummy-host.example.com
# ServerName dummy-host.example.com
# ErrorLog logs/dummy-host.example.com-error_log
# CustomLog logs/dummy-host.example.com-access_log common
#

ServerName www.peeniewallie.com
DocumentRoot "C:/Inetpub/wwwroot"
ServerAdmin pw@wispertel.net

ServerName www.killingstrangers.com
DocumentRoot "C:/Inetpub/wwwrootks"
ServerAdmin pw@wispertel.net

I shut down and restarted Apache and now www.killingstrangers.com is working as well.

MS Outlook

You can't normally find your .pst files with a standard search because Microsoft sucks.
The default file location for *.pst files is:

C:\Documents and Settings\user_id\Local Settings\Application Data\Microsoft\Outlook

So, I copied all of my .pst files over to the NAT RAID as \Outlook\*.pst

Data That Needs to Be Backed Up to RAID 5 NAS TeraStation

• *.pst files
• files related to website(s) PeenieWallie and KillingStrangers
• MySQL Data files

Disaster Recovery Plan

Need to create and test a website backup/restore from one PC to another. Should be able to lose a PC or the TeraStation and switch over to 2nd computer without losing more than 1 day's data.

Nightly/Weekly Backup Process
Export database from MySQL.

I yanked out the hard drives and put them into the other computers and I was able to save all of the data for the Peenie Wallie and Killing Strangers websites.

It took me about 6 hours to get Peenie Wallie up and running on one of the other computers. Now, I have to talk to Dell and see what's wrong with my server.

Bad Video Card

The problem was that the video card had crashed. I called Dell and they helped me figure this out.

By taking out the video card, I was able to get the system booted up no problem. Odd, huh? So, in any event, I ordered me a new video card and it’s working like a champ now.

My old video card was an nVidia GeForce 4 64Meg DDR Ti4200 AGP video card with an Nvidia Geforce 4 Ti4200 128MB AGP video card. I think that what happened was the cooling fan seized up on my video card and it eventually overheated.