WTF!? I'm out of disk space already
Just as I was going to copy season 3 of Highlander to aenea, my 1TB NAS system, I ran out of free space on the /usr volume, to which the bulk of the storage had been allocated. Nearly 850GB, gone in a little over a year. Sure, I could clean house a bit and make a little room, but the sad fact is, I need more storage.
I haven’t made a build vs buy decision yet, but I will point out that there seems to be a blind spot in NAS offerings between 1TB and 3TB; plenty go up to 1TB, plenty pick up at around 3TB, but where’s the midrange?
Back From Rome; Lovely Tech Surprise
I returned from Rome a week ago last Friday. I’ve been busy with work non-stop since then, so the detailed debrief post will have to wait until later this weekend.
I shot 1000+ photos, plus whatever Rebecca shot, and I even geocoded them all with the sweet geocoding feature in Google Picasa. Why, then, haven’t I posted them? Read on…
I awoke Monday morning to the soothing tone of one of my UPSs on overload. Upon further investigation, the culprit was the APC BackUPS XP 900 that keeps aenea my 1TG SATA RAID NAS box going.
I powered down all the connected devices (a CRT, aenea, and boromir, the direct-to-disk backup of aenea), and figured I’d worry about it when I got home from work. When I got home (12 hours later) I tried to power on aenea, only to find that a few seconds after power on, her power would go dead, LEDs would go out, and fans would spin down, then a few seconds after that power would come back on, and of, and on, etc. It doesn’t take a genius to peg this as PSU failure.
aenea is in an Aspire X-Alien case, which came with its own Aspire-branded 430W PSU. It seemed ok, but then again, in over 10 years of hardware hacking this is the first time I’ve had a PSU fail, so I obviously wouldn’t know. I thought about whipping out my multimeter, measuring voltage on the rails, maybe power output in the few seconds when it will boot, but I didn’t see the point. So, I ordered a Thermaltake 500W PSU and waited.
Yesterday it arrived, and I dropped it in without incident. It has a 120mm fan, and nicer cables than the OEM PSU that came before it, plus its quieter. aenea suffered some file system damage as a result of the outage, but it was nothing a little fsck couldn’t cure.
Now, at last, after that yak-shaving expedition, I can actually get around to posting my photos, which are stored on aenea (yes, they’re backed up on boromir, but it’s the spirit of the thing). That, and watching my evaluation-purposes-only DVD rips from my friend the guy who uses BitTorrent…
I still haven't learned my lesson about power failure and software RAID
A while back I suffered a power failure in my townhouse that screwed up aenea’s software RAID volume. I swore I’d put her on an UPS so it wouldn’t happen again. And yet, last night, it did.
So, this morning (a Sunday morning during which I was looking forward to beating ejabberd into submission) I awoke to find my file store screwed to the wall. I frantically searched about for the grime-encrusted floppy where I put my custom-compiled HighPoint RocketRaid driver for FreeBSD 6.0-amd64. I finally found it and was about to boot it, only to run across my post from the last time it happened, which refreshed my memory on the coping mechanism and the glorious absence of using an external driver floppy therein.
Now the file system damage has been repaired (or at least isolated) and I’m running the full fsck -p on all the filesystems to be sure all is well. insha’allah there will be no serious damage.
This time, I really mean it. I’m putting aenea on an UPS. Really.
It turns out my memory wasn’t faulty; I did put aenea on an UPS. Trouble is, her huge power supply and array of power-hungry hard drives result in a power draw somewhat in excess of what my humble little UPS can dish out. So, yet again, I go to Best Buy, hat in hand, taking back all the shit I talk about big box retail tech stores, to get a new UPS with enough juice to power aenea. Oh well.
'Action Canceled' When Viewing Compiled Html Help (CHM) Files on Network Share
I have a large collection of technical e-books, about half of which are in Compiled HTML (CHM) format, typically opened by MS HTML Help. I keep them on aenea, my 1TB file server, for easy access anywhere on my network.
Recently I began having trouble opening the CHM files. When I would attempt to open any of the CHM files on the network share, HTML Help would load, but it would display the well-known Internet Explorer “you’re screwed, pal” message:
Action canceled
Internet Explorer was unable to link to the Web page you requested. The page might be temporarily unavailable.
I immediately suspected a recent IE security patch, though I had nothing concrete to go on. I messed around a bit with security settings, including adding aenea to the Trusted Sites list and ensuring that Trusted Sites has basically every imaginable permission. Still, no joy.
Finally, I’ve come across KB 896358, which pertains to the MS security patch MS05-026.
KB 896358 lists a few things that this patch breaks, and things you can try to fix them. The one that jumps out at me is:
Certain Web sites and HTML Help features may not work after you install security update 896358 or security update 890175
This particular problem is covered in detail in a separate KB, 892675.
From that article, here’s the cause:
This problem occurs because security update 896358 and 890175 prevent HTML content that is outside the Local Machine zone from creating an instance of the HTML Help ActiveX control (HHCTRL). This change was introduced to reduce security vulnerabilities in HTML Help.
Pure genius. Well, clearly, I trust the content on aenea to be safe, so how can I override this padded-cell protect-me-from-myself bullshit? Read on…
Warning The symptoms are an expected and intended effect of installing the security updates. This section provides examples for administrators who must re-enable the HTML Help ActiveX control for business-critical programs. The workarounds may make the computer more vulnerable to the threats the security updates address. The safest course is not to use the registry workarounds. If you must use workarounds, set the registry values to be as restrictive as possible.
Outstanding; the fix broke HTML Help on purpose, and the fix of the fix re-introduces a security vulnerability. Just the kind of trade-off to make me want to pay rapacious Microsoft licensing fees.
Warning Serious problems might occur if you modify the registry incorrectly by using Registry Editor or by using another method. These problems might require that you reinstall your operating system. Microsoft cannot guarantee that these problems can be solved. Modify the registry at your own risk.
Yeah, and? Caution: Suicide can be dangerous.
Anyway, the first attempt:
Example 1: Use the UrlAllowList entry to enable specific URLs
Warning Include only URLs for sites that you trust.
The .reg file in this example re-enables hosting of the HTML Help ActiveX control in the following remote content:
- Any .chm files that are in the \productmanuals\helpfiles folder
- A Web application that located at http://www.wingtiptoys.com/help.
Paste the following text in a text editor such as Notepad. Then, save the file by using the .reg file name extension.
REGEDIT4
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\HTMLHelp]
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\HTMLHelp\1.x\HHRestrictions]
"UrlAllowList"="\\productmanuals\helpfiles;http://www.wingtiptoys.com/help/"
You cannot use wildcard characters in the URL string of any site that is added to the UrlAllowList registry key. For example, you cannot use the following URL string:
"UrlAllowList"="http://*.wingtiptoys.com"
However, you can use the following URL string:
“UrlAllowList”=”http://help.wingtiptoys.com”
This string lets the following sites host the HTML Help ActiveX control:
- http://help.wingtiptoys.com/research
- http://help.wingtiptoys.com/sales
Results of Example 1
So I’ll allow aenea‘s UNC path. Awesome.
That didn’t help. I’ll try the even less secure option:
Example 2: Use the MaxAllowedZone entry to enable a security zone
Warning The MaxAllowedZone entry enables all sites in a particular zone. Using the UrlAllowList entry may be safer. If you must use the MaxAllowedZone entry, set the value no higher than is required. If you set the MaxAllowedZone value to 3 or higher, you expose systems to attack from the Internet.
Note By default, the value for the MaxAllowedZone entry is set to zero. The following table summarizes how different entries are interpreted by the value for the MaxAllowedZone entry.
(snipped)
Paste the following text in a text editor such as Notepad. Then, save the file by using the .reg file name extension. This .reg file lets all content in the Intranet zone host the HTML Help ActiveX control.
REGEDIT4
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\HTMLHelp]
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\HTMLHelp\1.x\HHRestrictions]
"MaxAllowedZone"=dword:00000001
Results of Example 2
Hmm, that didn’t help either. I wonder if this isn’t the problem that’s preventing me from opening HTML Help files…
That’s right, it’s not. The cause of the problem is described in KB 896054 – You cannot open remote content by using the InfoTech protocol after you install security update 896358, security update 840315, or Windows Server 2003 Service Pack 1.
The fix is the similar to the one I tried above, but the registry key is ItssRestrictions instead of HHRestrictions. Thus:
REGEDIT4
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\HTMLHelp]
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\HTMLHelp\1.x\ItssRestrictions]
"UrlAllowList"="\aenea\filestor\;file://\aenea\filestor"
Does the trick for me. Note that you must specify each UNC path using both the UNC and URI notation. Specifying just \aenea\filestor does not work.
This sucks pretty profoundly, but at least there’s a workaround.
Burning CDs in FreeBSD
Because HP DVD drives sucks, prospertine is temporarily without a CD/DVD burner, so I need to use the one in aenea.
I’d never burned a CD outside of Windows before, so I didn’t know where to begin. Turns out FreeBSD includes a tool, burncd, that interfaces with the burner itself. I just need to burn the Fedora Core 4 install CDs, so I already have the CD images, making it pretty straightforward.
There’s more info in the handbook, under Creating and Using Optical Media.
I’m trying this, straight from the handbook:
burncd -f /dev/acd0 data FC4-i386-disc1.iso fixate
burncd: open(/dev/acd0): Permission denied
Hmm, ok, so I need to run as root. That’s unfortunate:
aenea# burncd -f /dev/acd0 data FC4-i386-disc1.iso fixate
next writeable LBA 0
writing from file FC4-i386-disc1.iso size 649838 KB
written this track 649838 KB (100%) total 649838 KB
fixating CD, please wait..
Wow, can it really be that easy?
Um, no. I know the ISO images are correct, as I verified the SHA-1 hashes with the Fedora Core download page. And yet, when I verify the disc in the Fedora Core installer, discs 1 and 2 have failed, and I’m sure 3 and 4 will too. That said, they’re not totally bogus; I was able to boot disc 1 and get most of the way through the installer, but it destabilized and crashed during file system creation.
So, what’s wrong w/ burncd? Do I have to do something special to make it work? Maybe it’s working fine, and the FC media verifier doesn’t work right w/ 700MB CD-Rs. That seems unlikely.
I’ll try the install again…
UPDATE: It worked fine. FC4 reported each disk as failing verification, but it installed and is running fine. Oh well.
Big Samba Hassle
I’ve set up aenea with Samba, so that users on my network can access her software and music collections from their Windows PCs. I use user authentication, where Samba authenticates connections against the system users. This way, I can access my home directory on aenea, and write to the music and software directories.
What I wanted seemed simple enough: I want to connect as anelson and have access to my home dir and read/write access to the software and music collections, but everyone else on my network should be able to open \aenea\filestor in Explorer and automatically connect as guests, with read-only access to the software and music folders.
Samba has all that is needed to make this happen: I created a share with public = yes, and write list = anelson, which means that unauthenticated users can connect, but only authenticated user anelson can write. This even works using the Samba smbclient; I can connect w/o a password and get guest access, or with a password and get full access.
However, for some reason, I cannot get XP to connect to a share without sending a username; either the currently logged in user, or some other username I type. I want it to establish what is known as a NULL session, meaning an unauthenticated connection, but I cannot figure out how to make it do so. The KB is useless, and the samba list is more concerned with how to get Samba to connect to an XP share, which is the opposite of my problem.
In the end, I succumbed; I set map to guest = Bad User in the [global] section of /usr/local/etc/smb.conf. This causes Samba to automatically switch to guest whenever a client connects with a user it doesn’t recognize as a system user. So as long as the logged in client user isn’t anelson, it’ll seamlessly connect as guest. If it is anelson, then chances are that user is me, and I know my password. Of course, if I didn’t, I could type some non-existent user name like guest or bgates in the XP connect dialog, and it’d be the same thing.
Lame.
HINT: Set log level = 2 in smb.conf, and monitor the log.[machinename] files in /var/log/samba when debugging problems like this; very useful.
More trouble in paradise: software RAID controllers can suck
Yesterday while I was at work, there was a brief power fluctuation in my townhouse. Since I’m still setting up aenea, she isn’t yet in my server closet, or hooked up to an UPS. So, predictably, she lost power.
This is somewhat bad, since the Highpoint RocketRaid 2220 SATA RAID controller that powers her 1TB RAID 5 disk array does not deal at all well with unorderly shutdowns, since the RAID logic is implemented in a software driver, not hardware.
Predictably, I suffered some file system damage. I now can’t boot, because /var seems sufficiently damaged to cause a panic in some ffs_whatever module. Thankfully it was /var and not, say, /usr, but nonetheless it sucks badly.
I’ve booted the FixIt shell on the FreeBSD 6.0 install disc, and loaded the hptmv6.ko kernel module from a USB floppy, so now I’m hoping I can fsck the problem away from this shell.
First, I’m discovering that a standard fsck in the FixIt shell doesn’t recognize the /var filesystem. fsck_ufs does the trick, but when I run it with fsck_ufs /dev/da0s1d it just outputs the file system errors and calls it a day; it doesn’t fix them.
Hmm, fsck doesn’t work because it’s looking in /sbin and /usr/sbin for the fsck_* executables, but in the FixIt environment they’re in /mnt2/usr/sbin. The FixIt shell is just flaky; sometimes I’ll run a command (ls, fsck, mount, man; it doesn’t matter what) and it hangs. Over on VTTY 2 (Alt-F2) I see about 15 timeout errors from acd0 before the shell finally comes back, only to hang again on my next command.
Fortunately, I’ve read on the lists that the first thing to try when a file system is fucked is to boot in single user mode (option 4 on the boot menu iirc). That boots find and gets me to a shell prompt.
I run
fsck -p /dev/da0s1d
Where -p is preen mode, which from the man page I gather checks for minor inconsistencies, but won’t handle major problems. All the list posts I see use this first.
From this I get:
/dev/ds0s1d: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY
I gather that’s bad. I found a few things on the list:
First, this frightful message advocating I use vi on the directory to remove invalid file entries. Um, no.
Next, this USENIX paper on FreeBSD soft updates, which explains what they’re for (to allow fsck to run whilst the file system is mounted, for speedier recovery), and when it doesn’t work (when the soft update snapshot is inconsistent, eg on power failure or crash).
So, with little help from the ‘net, I went ahead with:
fsck /dev/da0s1d
And got the UNEXPECTED SOFT UPDATE INCONSISTENCY, this time with a prompt: REMOVE? [yn]. I’m going to go with ‘yes’ and hope for the best…another error, this one UNREF FILE. The prompt is RECONNECT? [yn]. I’ll go with ‘yes’ again. Another ‘yes’ to the NO lost+found DIRECTORY CREATE?
A ton more UNREF FILE msgs; ‘yes’ each time.
FREE BLK COUNT(S) WRONG IN SUPERBLK. SALVAGE? [yn] Most definitely.
SUMMARY INFORMATION BAD. SALVAGE? [yn] Sure, go ahead.
BLKS MISSING IN BIT MAPS. SALVAGE? [yn] Yeah, if you want…
And then, as if nothing had happened, FILE SYSTEM MARKED CLEAN. Yay.
Now I do:
fsck -p
Do do a preening check on all the file systems. A few minor errors on /dev/da0s1f and /dev/da0s1e, but nothing fsck couldn’t handle on its own. Took a long time to scan the huge ~900GB partition…
Done now. I’ll exit this shell and proceed with the boot process, hoping for the best.
Voila! Booted fine.
So the moral(s) of the story are:
- When using a software RAID driver, you mustn’t let the power go out
- When using a BSD UFS file system, you mustn’t let the power go out
- When using UNIX in general, you mustn’t let the power go out
It’s hard for me to get used to this, as the bulk of my computer hours have been spent on Windows, where I’ve forced shutdowns countless times, and never had any serious file system damage. Needless to say, aenea is going on an UPS right now.
UPDATE: aenea sucks so much power she overloads the UPS I have on prospertine. I’ll have to move her into the server closet early, just so she’ll have an available UPS.
Getting MySQL 5 and phpMyAdmin running on FreeBSD 6.0 amd64
I’m going to use MySQL 5 on aenea, so I thought this would be a good time to try out phpMyAdmin, to see what all the fuss is about.
First, I’ve installed and got working both Apache 2 and PHP 5, both via their respective ports.
I next installed MySQL 5 server and phpMyAdmin via their ports. I enabled MySQL by adding mysql_enable="YES" to /etc/rc.conf. I started it with /usr/local/etc/rc.d/mysql-server.sh start.
To get phpMyAdmin started, I added an alias to the /usr/local/www/phpMyAdmin folder created by the port, so requests for /phpmyadmin would resolve there. To do that was a trivial task for Alias in /usr/local/etc/apache2/httpd.conf:
# Make phpmyadmin available
Alias /phpmyadmin/ "/usr/local/www/phpMyAdmin/"
Based on the instructions in the phpMyAdmin Documentation.html file, I then edited /usr/local/www/phpMyAdmin/config.inc.php to set a few variables. I set PmaAbsoluteUri, blowfish_secret, socket, connect_type, user, and password. They are:
PmaAbsoluteUri: http://aenea/phpmyadmin/
blowfish_secret: Wouldn’t you like to know
socket: /tmp/mysql.sock (This is the UNIX socket that MySQL listens on by default
connect_type: socket
user: root (Needs to be superuser so it can do all the admin tasks I want, like create databases)
password: blank string (The root password defaults to an empty string)
After that, I navigated to http://aenea/phpmyadmin/, and got the phpMyAdmin main screen.
The first thing I did was click ‘Privileges’, then click the ‘Edit Privileges’ icon for root@localhost. In the resulting page, I changed the root password to something non-blank. After that I updated config.inc.php to reflect the new password. I simply drop root@aenea..., since I will only allow connections from localhost.
mod_php on Apache 2, FreeBSD 6.0 doesn't automatically process .php files
I just installed the apache2 and php5 ports on aenea, and found that accessing .php files via Apache returned the PHP source code, instead of running the PHP server-side.
I had to add the following entries into /usr/local/etc/apache2/httpd.conf in order to get mod_php to pick up the files:
#Register PHP mime types
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps
That worked, but index.php wasn’t run automatically if I navigate to a directory. For that I added index.php to the end of the DirectoryIndex directive:
DirectoryIndex index.html index.html.var index.php
One apachectl restart later, and all was well.
Updating ports tree under FreeBSD
Now that I have aenea running ok under FreeBSD 6.0, I want to update the ports collection. Under OpenBSD, this is just a matter of getting the latest ports tree using CVS or CVSup.
The FreeBSD handbook has a useful section on getting the ports tree, which describes using CVSup to get the very latest ports from CVS.
First I install CVSup, which is pretty straightforward. I already have the -Release ports tree, which contains CVSup in net/cvsup-without-gui.
Another bonus in FreeBSD is that a CSVup .sup file is provided in the install in /usr/share/examples/cvsup/ports-supfile, so doing this:
# cvsup -L 2 -h cvsup.FreeBSD.org /usr/share/examples/cvsup/ports-supfile
Will update the ports tree.
I’ll use a different CSVup mirror; specifically, cvsup3.FreeBSD.org. So the command is:
# cvsup -L 2 -h cvsup3.FreeBSD.org /usr/share/examples/cvsup/ports-supfile
To my surprise, there’re alot of diffs coming down, even though the 6.0 release was less than a month ago. I guess the ports collection really is huge, and very active.