More fallout from disk full problem on /var
A while back I had some problems with MySQL running out of space on /var. Today I found out my Gallery2 was down, responding to all requests with “An error has occurred while interacting with the database.”
In the config.php file in the gallery2 install folder, there’s a line that sets the debug mode. By default it’s:
$gallery->setDebug(false);
For debugging I changed it to:
$gallery->setDebug('buffered');
and reloaded the Gallery main page. That displays a shitload of debug messages at the end of the page. In my case, I saw the familiar MySQL message about a table being corrupted. This time, it was g2_CacheMap. I did a quick repair table g2_CacheMap in mysql, and voila!, Gallery is back up.
insha’allah, this is the last such problem.
Mysql Error After Disk Full Problem
Last weekend my problem with disk space exhaustion resurfaced, and seemed to upset MySQL, as I get the following warning when I log into Drupal:
user warning: Table ‘./drupal/watchdog’ is marked as crashed and should be repaired query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, ‘user’, ‘Session opened for root.’, 0, ”, ‘http://apocryph.org/?q=user/login’, ‘http://apocryph.org/?q=user/login’, ’65.89.8.127′, 1153755477) in /usr/local/www/drupal/includes/database.mysql.inc on line 120.
I’ve not seen this message before, and don’t know how to repair. Perhaps the MySQL docs…
I found a decent article on repairing MySQL tables; it’s pretty straightforward:
bonzo# mysql -u www -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 38 to server version: 5.0.9-beta
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql> use drupal;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> check table watchdog;
+-----------------+-------+----------+-----------------------------------------------------------------------+
| Table | Op | Msg_type | Msg_text |
+-----------------+-------+----------+-----------------------------------------------------------------------+
| drupal.watchdog | check | error | Table './drupal/watchdog' is marked as crashed and should be repaired |
+-----------------+-------+----------+-----------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> repair table watchdog;
+-----------------+--------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+-----------------+--------+----------+----------+
| drupal.watchdog | repair | status | OK |
+-----------------+--------+----------+----------+
1 row in set (1.09 sec)
mysql> \q
Bye
That was easy…
Running out of space on /var thanks to MySQL
This morning I noticed the /var filesystem was at 100% capacity on bonzo, leaving MySQL somewhat hosed. Upon poking around in /var (which, sadly, I only made about 300MB) I removed a CVSUp catalog for the FreeBSD src tree that was taking up alot of space, and with the time that bought, sought a permanent fix.
First, I see three big files in /var/db/mysql: ib_logfile0, ib_logfile1, and ibdata1. Upon a bit of googling, I found these were the tablespace and log files for the InnoDB storage engine in MySQL, somewhat akin to the data and log device files SQL Server uses.
The problem, though, is that none of my databases use InnoDB. The MyISAM storage engine is fine for my needs, and considerably less complicated. So, I poked around for the my.cnf config file that’s supposed to be in /var/db/mysql, but it’s not there, suggesting the InnoDB stuff was compiled in. At any rate, I moved the files elsewhere and made sure my databases were still working, which they are if you’re reading this.
Longer term, I need to grow the /var filesystem…
Maddening problem with clock slip in FreeBSD under VMWare
A few weeks ago my father pointed out that the date stamps on my blog posts were behind by a week. Upon investigation, I found that bonzo‘s clock was a week behind. I updated it and declared victory.
Then, he pointed it out again a few days ago. Sure enough, it had slipped by several days. When I logged into the VMWare Console to check for options to sync the clock or whatever, I noticed a repeated error from the FreeBSD kernel that I’ve been getting on bonzo forever and always ignored:
calcru: runtime went backwards from [some big number] usec to [another] usec for pid [pid]
I googled this message, and found a whole community of FreeBSD users suffering under slipping clocks when running FreeBSD under VMWare. There’s something on the freebsd-current list, and VMWare’s own support forums.
There are a few proposed fixes, most involving the kern.timecounter.hardware sysctl. I tried changing it from its default of APIC to TSC and i8245, but none worked.
I then ran across a post on the VMWare forums suggesting:
In FreeBSD:
'tools.timeSync = "true"' added to .vmx file sysctl -w kern.timecounter.hardware=i8254 kldload vmmemctl (from vmware-tools) and have vmware-guestd running add 'kern.hz="250"' to /boot/loader.confI don’t have APIC or ACPI disabled in my FreeBSD host either
Now, I don’t want to run the VMware tools just to keep the clock in sync, but I did put kern.hz="250" in /boot/loader.conf and kern.timecounter.hardware=i8254 in /etc/sysctl.conf, then rebooted.
It’s been several minutes now, and the clock seems to be holding. I’m afraid I don’t understand in detail why this helps, though a VMware knowledgebase article alludes to a problem of missed timer interrupts, with a fix being reducing the frequency of the timer interrupts requested by the OS. I think that’s kern.hz="250". The importance of switching the time counting method from APIC to i8254 is less clear, unless it’s just a more reliable source of ticks.
At any rate, this problem has caused me to notice that VMWare server is in RC-2. As it’s the free successor to GSX Server 3, I really need to upgrade. Perhaps over the coming long weekend…
Added a virtual host to Apache and installing WordPress on FreeBSD
Recently I had cause to set up a WordPress blog engine on a virtual host on bonzo. My experience follows:
First I need to set up DNS for the domain. The owner of the domain used the registrar’s control panel to set the authoritative nameservers to ns1.afraid.org through ns4.afraid.org, which are the nameservers provided by FreeDNS.
Next, I log into my FreeDNS account, add the new domain to my domains, and point the domain and www. to bonzo’s IP address. I don’t have mail setup yet, so I’ll ignore the MX record for now.
Now, querying the domain in a web browser should bring me to my site on bonzo…sure enough, it does.
The next step isto set up virtual hosting on bonzo, which I’ve not yet had a need to do. The Apache docs on virtual hosting refer to virtual hosting by host name ‘name-based virtual hosting’.
It’s pretty straightforward; you define what IP addresses and host names you want to associate with what server roots. There is one huge gotcha:
(From the name-based virtual hosting docs 
Now when a request arrives, the server will first check if it is using an IP address that matches the
NameVirtualHost. If it is, then it will look at each<VirtualHost>section with a matching IP address and try to find one where theServerNameorServerAliasmatches the requested hostname. If it finds one, then it uses the configuration for that server. If no matching virtual host is found, then the first listed virtual host that matches the IP address will be used.As a consequence, the first listed virtual host is the default virtual host. The
DocumentRootfrom the main server will never be used when an IP address matches theNameVirtualHostdirective. If you would like to have a special configuration for requests that do not match any particular virtual host, simply put that configuration in a<VirtualHost>container and list it first in the configuration file.
This is definitely not what I would expect. It means I have to make sure that the current server root is present as the first <VirtualHost> element, else my existing site will break.
So, first I’ll create the <VirtualHost> in /usr/local/etc/apache22/httpd.conf entry for my existing doc root and make sure that works:
# Apply virtual hosts to all requests
NameVirtualHost *:80
# The 'default' virtual host, used when the host doesn't match one of the others
<VirtualHost *:80>
DocumentRoot /usr/local/www/drupal
</VirtualHost>
Next, an apachectl graceful to restart with the new config, and all is well. Requests to apocryph.org and bonzo.celatrix.com are still handled by my Drupal install as before.
Now I can create a separate doc root for the new domain, and point queries to it there. I’ll use /usr/local/www/craeton.com/:
bonzo# mkdir /usr/local/www/craeton.com
bonzo# chown www /usr/local/www/craeton.com/
bonzo# chgrp www /usr/local/www/craeton.com/
And another <VirtualHost> in httpd.conf accordingly:
<VirtualHost *:80>
ServerName craeton.com
ServerAlias *.craeton.com
DocumentRoot /usr/local/www/craeton.com
</VirtualHost>
Then a <Directory> entry to define the behavior of the /usr/local/www/craeton.com directory when exposed by Apache:
# DocRoot for the craeton.com virtual host
<Directory "/usr/local/www/craeton.com">
Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
</Directory>
I basically just copied the settings from the <Directory> entry for my default doc root.
So now, requests to craeton.com should resolve to this new folder. I’ll drop a simple index.html file and see what happens…it works.
Next I’ll need to create a MySQL database, user, and password for WordPress to use. First, the database:
$ mysqladmin -u root create wp_craeton -p
Enter password:
$
Now the user and password:
mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 16893 to server version: 5.0.9-beta
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql> grant all privileges on wp_craeton.* to 'wp_craeton'@'localhost' identified by '[secret]';
Query OK, 0 rows affected (0.06 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.03 sec)
Too easy.
Now I’m ready to install WordPress into this folder. There’s a port in /usr/ports/www/wordpress, but PHP apps like WordPress are so easy to get running I’d rather grab the latest bits from wordpress.org so I have control over the install.
I’ll download the latest directly on bonzo via my trusty SSH session.
$ wget http://wordpress.org/latest.tar.gz
$ tar xzf latest.tar.gz
$ ls
index.html latest.tar.gz wordpress
So, it extracted into a ‘wordpress’ folder. I don’t want that; I want the WordPress stuff immediately under the creaton.com/ folder. Easy enough:
mv wordpress/* .
Looking at the readme.html included in the tarball, there is a ‘Famous Five-minute Install’:
- Unzip the package in an empty directory
- Open up
wp-config-sample.phpwith a text editor like WordPad or similar and fill in your database connection details- Save the file as
wp-config.php- Upload everything.
- Open
/wp-admin/install.phpin your browser. This should setup the tables needed for your blog. If there is an error, double check yourwp-config.phpfile, and try again. If it fails again, please go to the support forums with as much data as you can gather.- Note the password given to you.
- The install script should then send you to the login page. Sign in with the username admin and the password generated during the installation. You can then click on ‘Profile’ to change the password.
Ok, item 1 is done. I’ll copy wp-config-sample.php to wp-config.php and put the DB connection info in it.
It was pretty easy; I just edited the DB info as it said:
<?php
// ** MySQL settings ** //
define('DB_NAME', 'wp_craeton'); // The name of the database
define('DB_USER', 'wp_craeton'); // Your MySQL username
define('DB_PASSWORD', '[secret]'); // ...and password
define('DB_HOST', 'localhost'); // 99% chance you won't need to change this value
// You can have multiple installations in one database if you give each a unique prefix
$table_prefix = 'wp_'; // Only numbers, letters, and underscores please!
// Change this to localize WordPress. A corresponding MO file for the
// chosen language must be installed to wp-includes/languages.
// For example, install de.mo to wp-includes/languages and set WPLANG to 'de'
// to enable German language support.
define ('WPLANG', '');
/* That's all, stop editing! Happy blogging. */
define('ABSPATH', dirname(__FILE__).'/');
require_once(ABSPATH.'wp-settings.php');
?>
Step 3 is equivalent to the copy to wp-config.php and thus doesn’t apply.
Step 4 is not applicable either, since I’m doing all this directly on bonzo via SSH.
In step 5, I navigate to the craeton.com/wp-admin/install.php. I get a splash screen and a ‘First Step’ link, which I click.
The first step in this install process prompts for a blog title and an email address. I’ll provide it and move on.
In the next screen it pauses to create the database tables, then generates a temporary password for the admin user. I’m admonished to not forget it, and given a link to the login page. I’ll go there now.
Logging in w/ the admin account and random password, I get a clean, simple admin GUI. I’ll change the password to something memorable, and declare victory. Too easy!
Upgrading bonzo to Apache 2.2
Today I went to slightly change my Apache 2.0 config on bonzo, when was greeted with a nasty error when trying to restart the process:
bonzo# apachectl restart
/libexec/ld-elf.so.1: Shared object "libexpat.so.5" not found, required by "httpd"
I suspect this is due to some of the upgrading it with with the ports tree recently. Obviously something needed a new version of libexpat, which broke my older build of Apache. I put a band-aid on the problem with:
ln -s libexpat.so.6 libexpat.so.5
in /usr/local/lib, but I don’t like running older versions of software. I noticed that Apache 2.2 is now out and in the ports tree under www/apache22. The upgrade instructions seem fairly straightforward, so I’m going to go for it.
A quick cd into /usr/ports/www/apache22, and a make show-options yield:
bonzo# make show-options
Available knobs:
By default, modules are compiled as dynamically loadable (DSO) modules.
Modules knobs philosophy:
Modules are split in categories, "make show-categories" shows you
which modules they contain. You can enable/disable/customize a category:
- To enable a category: WITH_<CATEGORY>_MODULES=yes
[WITH_PROXY_MODULES=yes]
- To disable a category: WITHOUT_<CATEGORY>_MODULES=yes
[WITHOUT_DAV_MODULES=yes]
- To customize a category: WITH_CUSTOM_<CATEGORY>
[WITH_CUSTOM_PROXY="proxy proxy_http"]
Apache-related
WITH_MPM: prefork (default)
worker
perchild (deprecated)
threadpool (testing purpose only)
WITH_HTTP_PORT: default: 80
WITH_LDAP: Enable LDAP support (mod_auth_ldap) (implies WITH_LDAP_MODULES)
WITH_(MYSQL|PGSQL|SQLITE): Enable SQL backend *dbd
WITHOUT_V4MAPPED
WITH_IPV6_V6ONLY: Don't allow IPv6 sockets to handle IPv4
connections
WITHOUT_SSL: Disable SSL support
WITH_THREADS: Enable threads support !! USE IT WITH CARE !!
WITH_DBM: Choose your DBM: bdb (Berkeley DB), gdbm or
ndbm (default)
WITH_BERKELEYDB: Choose your BerkeleyDB version: db2, db3,
db4, db41, db42 or FreeBSD (1.85)(default)
WITH_STATIC_SUPPORT: Build statically linked support binaries
WITH_STATIC_APACHE: Build a static version of httpd (implies
WITH_STATIC_MODULES)
WITH_ALL_STATIC_MODULES: All modules will be statically linked.
WITH_STATIC_MODULES: List of modules to build modules statics
(usefull for slave ports)
(They must be already enabled (i.e.
WITH_MODULES or with default configuration
use 'make show-modules', to check if they are
enabled)
WITH_MODULES: List of modules you choose
WITHOUT_MODULES: Disable selected modules
WITH_SUEXEC: Enable suExec support
SUEXEC_DOCROOT: SuExec root directory
SUEXEC_USERDIR: User subdirectory (default public_html)
SUEXEC_SAFEPATH: Set the safepath
SUEXEC_LOGFILE: Set log file for suexec (default: /var/log/httpd-suexec.log)
SUEXEC_UIDMIN: Minimal allowed UID (default 1000)
SUEXEC_GIDMIN: Minimal allowed GID (default 1000)
SUEXEC_CALLER: User allowed to call SuExec (default
${WWWOWN} (www))
SUEXEC_UMASK: Defines umask for suexec'd process(default:
unset)
WITH_DEBUG: Build a debug versoin of apache (set CFLAGS
to "-O0 -g -ggdb3" or ${DEBUG_FLAGS} and
defines WITH_EXCEPTION_HOOK too)
WITH_EXCEPTION_HOOK: Enable fatal exception hook
Optionnal patches:
WITH_EXPERIMENTAL_PATCHES Add performance patches (generally backported
from apr/httpd CVS)
Available make targets:
show-options: prints this message
show-modules: prints list of available modules
show-categories: prints list of modules sorted by category
Examples:
make WITH_STATIC_MODULES="ssl rewrite include" WITH_EXPERIMENTAL_MODULES=yes \
WITH_CUSTOM_AUTH="auth auth_dbm"
make WITHOUT_MODULES="access speling status" WITH_PROXY_MODULES=yes
make WITH_MODULES="include rewrite auth"
Note: If you define your custom options in /etc/make.conf, don't forget
to do not use quotes.
From make show-categories I get:
bonzo# make show-categories
AUTH contains these modules:
auth_basic auth_digest
AUTHN contains these modules:
authn_file authn_dbd authn_dbm authn_anon authn_default
AUTHZ contains these modules:
authz_host authz_groupfile authz_user authz_dbm authz_owner authz_default
CACHE contains these modules:
cache disk_cache file_cache mem_cache
DAV contains these modules:
dav dav_fs
EXPERIMENTAL contains these modules:
bucketeer case_filter case_filter_in ext_filter charset_lite log_forensic optional_hook_export optional_hook_import optional_fn_import optional_fn_export
LDAP contains these modules:
ldap authnz_ldap
MISC contains these modules:
actions alias asis autoindex cern_meta cgi charset_lite dbd deflate dir env expires headers imagemap include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias filter version
PROXY contains these modules:
proxy proxy_connect proxy_ftp proxy_http proxy_ajp proxy_balancer
SSL contains these modules:
ssl
SUEXEC contains these modules:
suexec
THREADS contains these modules:
cgid
From make I see the defaults are:
Per default categories are:
AUTH AUTHN AUTHZ DAV CACHE MISC
Categories available:
AUTH AUTHN AUTHZ CACHE DAV EXPERIMENTAL LDAP MISC PROXY SSL SUEXEC THREADS
I also want SSL; the rest are fine. So, that’s make WITH_SSL_MODULES=yes
That seemed to work. Now the really risky move: make uninstall apache20, and make install apache22. If this goes badly, I’ll be left without a working web server.
It seemed to go okay; the make deinstall complained about a littany of stuff dependent upon apache2, and I had to change /etc/rc.conf to add apache22_enable="YES".
The configuration file httpd.conf is in /usr/local/etc/apache22, while my old was in /usr/local/etc/apache2. The Apache docs say the new config file is much simpler, so I’ll just migrate my old settings over.
I had to copy:
- the
DirectoryIndexentryindex.php - the
Aliasentries forgallery2,bytehoard, andmail - the
AddTypedirectives for PHP files LoadModuledirectives fordav_svn,authz_svn, andphp5
The first problem with these changes is:
httpd: Syntax error on line 103 of /usr/local/etc/apache22/httpd.conf: Cannot load /usr/local/libexec/apache2/mod_dav_svn.so into server: Shared object "libaprutil-0.so.9" not found, required by "libsvn_repos-1.so.0"
This is a pretty old version of the dav_svn module; I’ll disable it for now until I can build the updated version
Another problem:
httpd: Syntax error on line 105 of /usr/local/etc/apache22/httpd.conf: API module structure `php5_module' in file /usr/local/libexec/apache2/libphp5.so is garbled - perhaps this is not an Apache module DSO?
So, I’ll have to rebuild the PHP5 module as well. Awesome.
Argh, I keep running into shit along the dependency chain that I have to upgrade to build the php5-extensions. So far, pecl-magick and mysql50-client. Also have to (re)build www/mod_php5. Oddly, when I make search name=php5 I see the mod_php5 port listed as /usr/ports/www/mod_php5, but there’s no such folder in my ports tree. I’m cvsuping my ports tree just in case, but I’m not holding out hope.
Hmm, turns out the mod_php port has been removed and integrated into the PHP build process. Lame. I did a make config, checked the Apache box (and the multibyte box while I was at it), then did a make clean and a make reinstall. I’ll have to do the same thing on php5-extensions.
Ok, it looks good. Drupal is working, but Gallery is broken (requests for /gallery2 return Forbidden).
It seems the reason is that uninstalling Apache 2.0 removed the /usr/local/www/data symlink to /usr/local/www/data-dist. How lame. All my paths will change now.
Ok, all seems well. There’s only one little thing bugging me now:
bonzo# apachectl stop
bonzo# apachectl start
[Mon Jun 19 07:48:24 2006] [warn] (2)No such file or directory: Failed to enable the 'httpready' Accept Filter
I’ve done a bit of reading on the subject. Apparently, the accept filters are an optimization within Apache that takes advantage of an optimization in various OS kernels in which the accept() socket function doesn’t return until a particular filter condition is met; in the case of HTTP, that complete HTTP headers have been received from the client. I don’t really understand why this is a major performance enhancement, but I trust Apache and the FreeBSD kernel team to do the right thing.
At any rate, from this Google Groups conversation, it seems I can enable the relevant accept filters in the kernel by adding a apache22_http_accept_enable="YES" line in /etc/rc.conf. I’ll try that.
Yup, that takes care of it. Sweet.
Headless Azureus on FreeBSD 6.1
I want to run the popular Azureus BitTorrent client on bonzo, which has a ton of bandwidth to burn. Bonzo is a virtual machine on a rackmount server in a data center, so I need to run Azureus in headless mode, using its HTML UI to control it from afar.
I’m starting with a freshly upgraded FreeBSD 6.1-RELEASE box.
First, I’ll need to install Java 1.5. As it happens, Sun and the FreeBSD team have finally come up with an official Java release for FreeBSD. I downloaded the package for FreeBSD 6, and installed it with pkg_add.
Upon doing so, I got:
bonzo# pkg_add diablo-jdk-freebsd6-1.5.0.06.00.tbz
pkg_add: could not find package javavmwrapper-2.0_5 !
There’s a port called javavmwrapper in java/; I’m installing that.
After running pkg_add again, it worked with this warning:
pkg_add: warning: package 'diablo-jdk-1.5.0.06.00' requires 'javavmwrapper-2.0_5', but 'javavmwrapper-2.0_7' is installed
No big deal; this package was for FreeBSD 6.0, and I’m running 6.1; inshallah they’re compatible.
Sure enough:
bonzo# java -version
java version "1.5.0"
Java(TM) 2 Runtime Environment, Standard Edition (build diablo-1.5.0-b00)
Java HotSpot(TM) Client VM (build diablo-1.5.0_06-b00, mixed mode)
Now I’ll download Azureus. Since I’m not running Windows, I’ll want the jar download.
I also need a couple additional JARs in the classpath to make this work. After downloading them into the same directory as the Azureus jar file, I ran:
$ java -jar Azureus2.4.0.2.jar --ui=console
as per instructions. Sure enough, Azureus came right up. In fact, right off the bat it reported a few updates available:
Update available for 'Core Patch Checker', new version =
Update available for 'azupdater/Azureus Updater Support Patcher', new version = 1.8.3
Normally installed via auto-update.
This plugin contains support for restarting azureus when upgrading.
...
Update available for 'Platform-specific support', new version = 1.11
Normally installed via auto-update.
This plugin contains platform-specific support for Azureus. For windows this includes file association support and Azureus.exe.
To manually install this plugin, unzip the files into the Azureus program location, ensuring that the OS-specific path within the ZIP file is removed.
...
Unfortunately, neither the Console UI Help nor Google have any insight into how to auto update from the console UI, so I guess I’m screwed for the time being.
Moving on, I need to configure Azureus’ bandwidth limitations. Bonzo is plugged into a Fast Ethernet switch which ultimately links it to multiple redundant fiber backbones, however I’m limited to 5TB/month of transfer, and I don’t want to burn it all on Azureus. So, I’m thinking a 500kb/s up limit is reasonable.
set Core_iMaxUploadSpeed 500
I also want to override the file system locations where torrents and downloads go:
set General_sDefaultSave_Directory /home/anelson/downloads
set General_sDefaultTorrent_Directory /home/anelson/torrents
Next I’ll install the HTML UI plugin. Installing it was easy; I downloaded the JAR from the HTML UI page and copied it to the plugins directory under the directory where I extracted the Azureus JAR.
Configuration of the plugin becomes tricky, as it doesn’t have any visible options when I run set to list available parameters. In fact, I don’t think the plugin is being loaded; when I attempt to access http://localhost:6886/ that port isn’t open. I must be missing something required to get the plugin to actually load.
Doh! I made a n00b mistake. I put the jar into the plugins folder, but the correct way to install a plugin is to create a folder for the plugin within plugins, then put the JAR in that folder. I did that and the azhtmlwebui options are now listed when I do set.
The configuration of the HTML UI is somewhat non-obvious, so I used the Azureus install on hotsoup-p2p to look at the GUI options to translate them into console config settings. I want the HTML UI to be available only via localhost, so I can use an SSH tunnel to access it but otherwise it’s not exposed. Thus, the Plugin.azhtmlwebui.Access parameter is set to 127.0.0.1. I don’t need a username or password, and the other parameters are fine at their default values.
set Plugin.azhtmlwebui.Access 127.0.0.1
I set up a PuTTY shortcut to run this:
"C:\Program Files\PuTTY\putty.exe" -A anelson@69.13.38.69 -L 127.0.0.3:6886:localhost:6886
This will tunnel port 6886 on 127.0.0.3 to bonzo’s localhost port 6886. Then, I can access the admin interface with http://127.0.0.3:6886/ from the PuTTY client machine.
Voila! Works like a charm.
Now I’ll download a large, legal torrent to test out my transfer performance. I suspect some meddling by my bargain-basement ISP (CI Host), and would like to explore it with a legal download rather than stir up a ToS violation on my first day. DataGalaxy.net has a decent list of legal torrents; I chose the Fedora Core ‘stentz’ DVD, clocking in at over 2GB.
I got an error ‘uploading’ the torrent (it’s actually downloading, but the HTML web UI doesn’t distinguish between uploading a torrent file or specifying a URL). The error in the HTML UI interface was meaningless:
Error loading http://www.kanava.org/~bostonarch/btit/download.php?id=87ab90f08a3dacefe69b7e0de15d88e207efdcdc&f=stentz-dvd-i386.torrent
On the console it was more constructive:
org.gudy.azureus2.plugins.download.DownloadException: DownloadManager::addDownload: default data save directory must be configured
I knew that. The Console UI help list two parameters for that purpose:
set "Default save path" "/home/anelson/downloads" string
set "Use default data dir" true boolean
Splash one. Download worked now.
Well, sort of. Xfer rate is nearly nothing, and I suspect a firewall problem.
Sure enough. I ran nc from wintermute, which is on my home LAN, and got this:
$ nc -v -w 5 bonzo.celatrix.com 6881
nc: connect to bonzo.celatrix.com port 6881 (tcp) failed: Connection refused
So the bullshit CI Host firewall strikes again. Fortunately, I know of at least one TCP port (3389, Terminal Services) which their firewall passes. I’ll switch Azureus to using that port and try again.
set TCP.Listen.Port 3389
Ok, how about now:
nc -v -w 5 bonzo.celatrix.com 3389
Connection to bonzo.celatrix.com 3389 port [tcp/*] succeeded!
Ok, so it’s no longer a firewall issue. Maybe it’s the torrent itself. I’m trying the ‘bordeaux-dvd’ as well, which has several hundred seeders.
No, it’s still running at nearly 0 kb/s. Hmm.
I thought maybe it was an outgoing firewall rule preventing requests from bonzo to tcp/6881, but then I did this from bonzo:
nc -v -w 5 actinium.apocryph.org 6881
Connection to actinium.apocryph.org 6881 port [tcp/*] succeeded!
So that’s obviously not it. I’ll try enabling logging, to see if that helps:
set Logger_sDir_Directory /home/anelson/azureus/logs
set Logger.Enabled 1
set Logger_bEnable 1
Nothing really jumps out, but the log output a flood of messages, and the log filter parameters (like Logger_bLog0-0) are non-obvious.
I tried downloading another torrent which I’m currently fetching on hotsoup-p2p at an average of 70kB/s; on bonzo it’s not getting anything, despite multiple seeder connections. I’m beginning to suspect CI Host blockage, but I’m not giving up yet.
I’m installing BTQueue from the ports collection. BTQueue is (hopefully) a simpler client that’s more easily configured, which will enable me to compare performance meaningfully.
I couldn’t get BTQueue to work; it’s a strange tool with it’s own console and no obvious configuration files, and scarce docs. I’m trying rtorrent instead.
rtorrent is pretty easy to work with. I reconfigured it to use tcp/3389, and it suffers the same agonizing slow download problem exhibited by Azureus on bonzo. This leads me to conclude that CI Host is deliberately obstructing BitTorrent downloads, which would be egregious if true. I’ll email and ask.
After some time, rtorrent started to show decent transfer rates, at least as good as I get on my house DSL line. I know BitTorrent downloads take a while to get up to full speed, but Azureus normally gets cranking pretty quickly, and even after an hour on bonzo was still getting nowhere. For now I’ll continue to use rtorrent, though long term I’d like to get Azureus working.
The next step is to integrate the PeerGuardian P2P blocklist into either the P2P client (less desirable) or the BSD firewall (more desirable).
It looks like net/tableutil is a good candidate for this.
Unfortunately not. The PeerGuardian blocklist thing is all very obfuscated. Once I went through bullshit registration at BlueTack I was able to access their List Info Tool to get a link to their ‘level 1′ blocklist, which according to their blocklist FAQ is the list of anti-p2p addresses. But this list doesn’t seem to be available in the binary P2B format supported by tableutil; instead it’s in the text-only P2P format. I guess I’ll have to write a sed script to convert this into the lists of IP ranges which pf understands. Why is this such a PITA?
Upgrading bonzo from FreeBSD 5.4-RELEASE to FreeBSD 6.1-RELEASE
I’m taking advantage of my leave back in the US to do some much-needed maintenance on my stateside computer systems. One such maintenance task is upgrading bonzo from FreeBSD 5.4-RELEASE to FreeBSD 6.1-RELEASE.
I’m following the upgrade procedures in the installation notes. First, I’m making a note of the device names and mount points:
bonzo# mount
/dev/ad0s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad0s1g on /home (ufs, local, soft-updates)
/dev/ad0s1e on /tmp (ufs, local, soft-updates)
/dev/ad0s1f on /usr (ufs, local, soft-updates)
/dev/ad0s1d on /var (ufs, local, soft-updates)
/dev/ad0s1h on /usr/local/www/data-dist (ufs, local, soft-updates)
Next I booted the bootonly ISO image (I’m too lazy to download two full-sized ISOs), and selected Upgrade. I chose the ‘Developer’ install set, and dutifully entered the mountpoints above into the disk label editor.
After mounting the filesystems, the upgrade utility warned that /usr/src exists and sources were included in the list of upgrade packages, but the upgrade utility doesn’t handle source updates, and to use CVSup or CTM. /usr/src won’t be affected. Fine. I’ll CVSup to get to 6.1-RELEASE anyway.
I accepted the default /var/tmp/etc to save my current /etc
After that, the installer ran for a while, and finally presented me with a ‘Upgrade completed!’ message. All my old /etc files were restored, and the new /etc files were place in /etc/upgrade “in case you wish to upgrade these files by hand (though that should not be strictly necessary)”.
I rebooted, and everything came back up fine. Now I will fetch /usr/src to 6.1-RELEASE, and update the ports tree.
Trying to build the latest sysutils/portupgrade port I ran into a downstream problem:
An older version of databases/ruby-bdb is already installed (ruby18-bdb4-0.5.3)
A quick make deinstall; make reinstall should clear it up.
Installing SquirrelMail 1.4.5 in FreeBSD 5.4-Release
I’m in Baghdad now, where my only link to the ‘net is a high-latency satellite. Mozilla Thunderbird can’t hack the delays this introduces, and thus times out when attempting to fetch my IMAP mailbox on ender. Thus, rather than use one of the other shitty IMAP clients, I’ve decided to pull the trigger on SquirrelMail.
I have previously attempted to install RoundCube, but found it a bit too immature at this point, so I’ve decided to go with something tried and true: SquirrelMail.
Fortunately, the FreeBSD ports collection includes SM, and a pre-built package based on 1.4.5 is available. Unfortunately, the pre-built package uses Apache 1.3, while I run Apache 2 on bonzo. This leads to all sorts of fun:
bonzo# pkg_add ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/squirrelmail-1.4.5_2.tbz
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/squirrelmail-1.4.5_2.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/apache-1.3.34_2.tbz... Done.
pkg_add: package 'apache-1.3.34_2' conflicts with apache-2.0.54_2
pkg_add: please use pkg_delete first to remove conflicting package(s) or -f to force installation
pkg_add: pkg_add of dependency 'apache-1.3.34_2' failed!
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/php4-4.4.1_3.tbz... Done.
pkg_add: package 'php4-4.4.1_3' conflicts with php5-5.0.4_2
pkg_add: please use pkg_delete first to remove conflicting package(s) or -f to force installation
pkg_add: pkg_add of dependency 'php4-4.4.1_3' failed!
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/mhash-0.9.2.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/php4-xml-4.4.1_3.tbz... Done.
pkg_add: could not find package apache-1.3.34_2 !
pkg_add: could not find package php4-4.4.1_3 !
pkg_add: pkg_add of dependency 'php4-xml-4.4.1_3' failed!
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/php4-session-4.4.1_3.tbz... Done.
pkg_add: could not find package apache-1.3.34_2 !
pkg_add: could not find package php4-4.4.1_3 !
pkg_add: pkg_add of dependency 'php4-session-4.4.1_3' failed!
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/php4-pcre-4.4.1_3.tbz... Done.
pkg_add: could not find package apache-1.3.34_2 !
pkg_add: could not find package php4-4.4.1_3 !
pkg_add: pkg_add of dependency 'php4-pcre-4.4.1_3' failed!
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/php4-openssl-4.4.1_3.tbz... Done.
pkg_add: could not find package apache-1.3.34_2 !
pkg_add: could not find package php4-4.4.1_3 !
pkg_add: pkg_add of dependency 'php4-openssl-4.4.1_3' failed!
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/php4-mhash-4.4.1_3.tbz... Done.
pkg_add: could not find package apache-1.3.34_2 !
pkg_add: could not find package php4-4.4.1_3 !
pkg_add: pkg_add of dependency 'php4-mhash-4.4.1_3' failed!
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/php4-mbstring-4.4.1_3.tbz... Done.
pkg_add: could not find package apache-1.3.34_2 !
pkg_add: could not find package php4-4.4.1_3 !
pkg_add: pkg_add of dependency 'php4-mbstring-4.4.1_3' failed!
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/All/php4-gettext-4.4.1_3.tbz... Done.
pkg_add: could not find package apache-1.3.34_2 !
pkg_add: could not find package php4-4.4.1_3 !
pkg_add: pkg_add of dependency 'php4-gettext-4.4.1_3' failed!
Outstanding. Well, I’m not going back to apache 1.3, and I don’t feel like building from the port, since bonzo‘s ports collection is outdated. However, I really don’t want to download the sources from SM and build on FBSD. So, I guess I have no choice but to update the ports tree:
cvsup -L 2 /root/ports-supfile
Took a while; updated.
Now doing:
make WITH_DATABASE=1
The WITH_DATABASE enables PEAR support to ensure SM can use MySQL.
That was uneventful. Next up,
make WITH_DATABASE=1 install
During the build PEAR was installed also, which yielded this:
To use PEAR you have to add the correct include path into
your ${LOCALBASE}/etc/php.ini configuration file, like:
include_path = ".:/usr/local/share/pear"
I’ll have to remember to do that…
Ugh, it’s installing the new PHP 5.1.2. This’ll take a while…
Actually wasn’t bad at all. Final output:
You now need to add an alias to apache's httpd.conf pointing to
/usr/local/www/squirrelmail in order to access SquirrelMail from
your web browser, or create a VirtualHost with DocumentRoot set
to that directory.
For SquirrelMail to work properly you will need to make sure the
following option is set in your php.ini file:
file_uploads = On
If you have problems with SquirrelMail saying "you must login" after
you just have, the following php.ini option may help:
session.auto_start = 1
In order to do your administrative configuration you need to
cd /usr/local/www/squirrelmail && ./configure
SquirrelMail will not work until this has been done.
So, first I’ll add an alias to /usr/local/etc/apache2/httpd.conf
This seemed to do the trick:
Alias /mail/ "/usr/local/www/squirrelmail/"
Ok, that ‘worked’, inasmuch as I got this response in my browser:
ERROR: Config file "config/config.php" not found. You need to configure SquirrelMail before you can use it.
And configure it I shall. As per the instructions above, a quick cd /usr/local/www/squirrelmail && ./configure should be just the thing.
The configuration utility has a numbered menu system circa 1980. I’ll just wander through the menus, specifying values for stuff as they make sense to me.
That was easy. I like the Blue Grey theme, with the Verdana 08 custom style sheet. Nice.
WTF Are 'Port Huron Labs'?
Today I happened to be tailing my httpd-access.log on bonzo, and noticed several requests from 85.64.146.188 for various top-level pages on apocryph.org, with a user agent value of Port Huron Labs. A bit of googling leads me to believe that Port Huron Labs is an email harvester.
The IP address is registered to an ISP in the Netherlands, according to ARIN. That ISP’s internal WHOIS server reports the IP is assigned to an Israeli company. I suspect they are innocent in all this; they have an entire Class B subnet, so I suspect one of their machines has been compromised and used to crawl the web for email addresses.
At any rate, there are no email addresses on my site, so harvesters will be sorely disappointed.