Fed up with meddling scolds trying to control what they don't understand
Don Reisinger at CNET posted an angry blog entry yesterday titled “A tech lover’s call to arms”. It’s the usual hand-wringing over various and sundry attempts to regulate/control/limit technology, ranging from the RIAA’s and MPAA’s feckless crusade against the scourge of piracy to the seemingly endless procession of politicians banning violent video games “for the children”. Apart from being more alarmist and less..well..helpful than I think it should, it does resonate with the gripes I and I think my fellow geeks feel towards various encroachments upon our technological liberty.
It’s not clear what Don would have his readers do, save for the obvious:
For what it’s worth, I call on all journalists, readers and companies to forego their apathy and do what they can to stand together and fight the ridiculous notion that technology should be throttled back for fear of its inability to adapt to the expectations of the Old Guard.
What that means, exactly, I don’t know. I’m pretty sure the fix isn’t regulatory, though. The same government that can ban “excessive” broadband prices and force telecomms into network neutrality can just as easily go the other way. If the last 250 years have taught us anything, it’s that placing one’s fate in the hands of government is always and everywhere foolish.
I would like to see the tech community focus more energy on hardening the Internet and content distribution technologies against government and corporate munging. The early days of the Internet were preoccupied with the notion that cyberspace would be a new frontier free of the constraints and yokes of meatspace, and it was upon that Utopian vision that the modern Internet was built. As it turned out (not surprisingly, in retrospect), the Internet does not “[interpret] censorship as damage” and “[route] around it” as John Gilmore famously said back in ’93. At the time, though, it seemed government and corporate meddling in the free flow of information was finally over, thanks to a the holy trinity of freely available encryption, anonymity, and global interconnectivity.
Though the Internet’s architecture makes it easier to censor and regulate than I first thought, the advantage still lies with us. If the Pirate Bay can stand up to the worst the MPAA can throw at it, then defiance of the DMCA and government and corporate attempts to monitor/regulate/censor content is also possible. If rich content could be hosted and distributed anonymously, how can content bans be effectively enforced? Where do you send the DMCA takedown notice? To whom do you serve baseless search warrants and frivolous law suits?
There’s nothing wrong with agitating for less technological forms of change, but giving governments more regulatory power over the Internet should not be undertaken lightly. What happens if the RIAA buys the chairman of the Network Neutrality subcommittee? What if some fundamentalist Muslim or Christian religious figure gets control of the Internet regulatory agency? All those powers that seemed great in the hands of a Network Neutrality warrior would suddenly be turned against us. It’s possible somewhere in human history a government has been given awesome power and subsequently not abused it, but nothing springs to mind just now.
Much like an armed citizenry is (supposed to be) the last bulwark against government tyranny, a decentralized, anonymous, covert darknet tunneled through the Internet should be established as a last bastion of freedom of thought and expression if the world’s governments’ contempt for virtually limitless freedom of conscience reaches its logical conclusion.
Reading this back to myself, it sounds like an angry cryptoanarchist rant. I’m not some sort of fringe libertarian, I swear! I just don’t like to be told what to do by clueless bureaucrats and pandering politicians, and if the time comes I want to be sure I have the means to disobey those who presume to control me.
WPA Cracking with rt73, aircrack-ng, and Ubuntu Gutsy
There’s been a dearth of tech posts around here lately, but that’s about to change.
I’ve previously mentioned the Alfa AWUS036S USB WLAN adapter that I got a while back for OpenBSD wardriving. It’s been reliable for that purpose, but lately I’ve been interested in using it with aircrack-ng to crack WPA pre-shared keys.
Normally this isn’t the kind of thing I’m into, but then I learned about the Church of Wifi’s WPA Rainbow Tables, which consist of 40GB of compressed pre-computed WPA keys and their hashed derivations for the 1000 most common SSIDs and a selection of common passphrases. I downloaded the torrent (tip: the CoW link is bad; the torrent is here, and the Church of Wifi cracking tool CoWPAtty which reads the rainbow tables and uses them to crack WPA.
I ran them against a capture I’d taken using OpenBSD and kismet, only to find that I had not captured any of the WPA handshakes required to crack the WPA PSK. This led me down the rathole of actively forcing the WPA handshake by knocking a client off of an AP and monitoring the client’s attempt to reconnect, using the aircrack-ng suite of tools.
Rather than fight with getting aircrack-ng to inject packets with my USB wlan device on OpenBSD, I bit the bullet and went with Ubuntu Gutsy 7.10 instead.
Running Gutsy i386 on a VMWare Workstation 6.0 virtual machine, I plugged in the Alfa USB adapter, enabled it in VMWare with the VMWare UI, and did an ifconfig to confirm my adapter was detected. It showed up as wlan0 and that was that. Or so I thought.
Once I downloaded and built the latest 1.0 beta of aircrack-ng (an uneventful proces; just make sure the libssl-dev and libpcap0.8-dev packages are installed). Then I ran airmon-ng start wlan0 11, which unexpectedly failed. I could not get the wlan0 interface into monitor mode; attempts kept failing with an error about the mode not being supported. I then booted into a Backtrack 2 Live CD and ran airmon-ng there, but despite forcing a WPA handshake (first with aireplay-ng and later by manually disconnecting and reconnecting a client to the AP I was cracking) I wasn’t getting any WPA handshakes in any of my captures.
Now is a good time to digress for a bit about the Linux WLAN driver situation. If you’re going to use a wireless adapter on Linux, you have to wonder, what wireless chipset does the adapter use, are there drivers available, are they reliable? If you intend to use the adapter for wardriving, WEP cracking, WPA cracking, etc, then you also have to figure out if the tool you want to use supports the chipset, if the drivers you’re using support what you’re doing with the tool, and whether or not the drivers are so unstable as to kernel panic your machine at random.
In my case, I’m using a USB wireless adapter so I can use it from a virtual machine, which makes the situation even worse as USB wireless support is hit or miss at best, especially if you’re capturing all traffic on a channel which requires monitor mode.
As wireless USB devices under Linux go, there aren’t many options. The Alfa AWUS036S, based on the Ralink RT73 chipset, is the best option I’ve found. I’ve also read about the Alfa AWUS036H, based on the Realtek RTL8187. The aircrack-ng and Kismet sites both list a few other USB adapters, none of which I’ve read particularly good things about.
The RT73 driver situation is pretty complicated. Gutsy 7.10 ships with RT73 support, though I don’t know where the drivers came from. A while back I tried to get the serialmonkey drivers working with my shitty EDIMAX USB adapter and had nothing but problems under Linux; I’ve not tried the new RT73 stuff from serialmonkey but I assume it’s unstable.
Finally, there’s p_larbig’s stuff, which seems based on the serialmonkey code but is patched for stability, packet injection, and compatibility with aircrack-ng. He has drivers for rt73 and rt2570-based hardware, though I’ve not worked at all with the rt2570 drivers.
I finally had luck with the latest p_larbig rt73 driver, 2.0.1. It was the usual download tarball, make, make install, which installs the rt73 kernel module. However, that’s not enough.
I also had to disable the modules Ubuntu shipped with, which otherwise insist on loading instead. I’ve read elsewhere about a guy suggesting you actually delete the built-in modules from /lib/modules, but that seems a bit heavy-handed. Instead, I added the following lines to /etc/modprobe.d/blacklist:
# Disable the USB ralink drivers other than the custom build one
# AJN 11-2007
blacklist rt2500usb
blacklist rt73usb
blacklist rt2x00usb
I also added alias ra0 rt73 to /etc/modprobe.d/aliases.
Then ran sudo update-modules. After that I plugged in my Alfa and it came up at rausb0.
Using beta1 of aircrack-ng 1.0 I ran a few simple tests.
First, airmon-ng stop rausb0 and airmon-ng start rausb0 11, both of which worked this time.
Next I ran aireplay --test rausb0, which reported successful injection.
After that I used these instructions to force and capture a WPA handshake. I would note that it’s important to verify you can capture these handshake packets before you declare victory. I thought I had it made when I booted the Backtrack 2 live CD, which has built-in support for rt73 chips, only to find that I could capture everything but the WPA handshakes. Be sure that’s working if you intend to crack WPA PSKs.
Once I got a WPA handshake, I ran it against cowpatty with the WPA-PSK rainbow tables, and tried one million keys in about 20 seconds, roughly 50k keys/second. If you’ve every tried brute-forcing a WPA PSK with just aircrack-ng and no rainbow tables, you’re probably more used to 10-50 keys/second; that’s right, it’s an improvement of three orders of magnitude. Insane.
The final moral of the story is: change your access point’s default SSID, and choose a secure WPA passphrase. Failure to do either will result in vigorous pwnage.
Automatic Podcast Transcript Generator
I just read a TechCrunch post on companies that should exist but dont, which reminded me of an idea I had years ago during the dawn of the internet age, to use Internet technologies to farm out translation and transcription work to low-wage areas, not just India and China but the rural United States, etc.
One of the companies mentioned in the post is a podcast transcription service. It occurs to me that the new Amazon web service, Amazon Mechanical Turk, could be the key to implementing just such a service.
The service has some shittly limitations that I hope will go away:
- The requestor fixes the price for the service. An obvious alternative pricing mechanism would be a reverse auction wherein the requestor specifies a maximum price (equivalent to the reserve price in a standard eBay auction), and workers bid on the work
- There is no sandbox, so playing with the service is somewhat complicated
- The requestor funds an account with the amount required to pay for the request, along with commission, before the request can be submitted. Upon receiving the response, the requestor can accept or reject the work product. So, if an intermediary service were acting as a broker (as a podcast transcription service would), the broker takes on some risk by fronting for the service, or the broker must require the client to similarly prepay.
- The qualification system, whereby requestors can test the qualifications of prospective workers, is very limited. For transcription purposes, I’d want to attach an MP3 file of a conversation to be transcribed, maybe a link to some instructions on how the transcription should be formatted, and a free text box where the transcription should go. Of these, only the free text box is supported now.
- There is no support for competition among workers. You can allow a task to be performed by multiple workers, but the expectation is that anyone who follows the instructions will receive payment. So, telling three people to transcribe a conversation means that all three are supposed to be paid, even if one is clearly better than the other two. This needs to change if AMT is to be at all viable.
- Apart from the qualification system, there’s very little ability to restrict tasks to specific workers. In the case of transcription, you’d want the requestor to have some subjective feedback measure, and you’d want that to help determine if the worker will get work from you again. Or, perhaps you would want to allow the requestors to decide if they want to pay for a highly regarded translator, or only a mediocre one.
AMT intentionally hides the ‘man behind the curtain’, operating instead as a black box into which work orders go, and from which completed work emerges. This is an unfortunate decision; individual accountability, and the ability of an individual to excel above others, is very important in a competitive service economy.
- Even the qualification system is somewhat limited, in that workers must request a qualification before it can be granted. So, you can’t, say, create a
TopTenPercentWorkersqual and grant it programmatically to the best 10% of your workers; it must be requested and adjudicated on a per-worker basis. - Ideally, the requestor (or more accurately, software operating on behalf of the requestor) could review work at submission time, as a second-stage validation step, and provide feedback which the worker could incorporate into the work product. For example, when a worker inputs a transcription, it could check for adherence to transcription format conventions, case, spelling, etc.
AIM Bots: To my surprise, they seem to work
When AOL first added the ‘AIM bots’ to my account, I received the intro IMs via Gaim and dismissed them as another purile attempt by AOL to monetize their IM service.
However, in the interest of keeping an open mind, I played briefly with the MovieFone bot, which as the name implies is primarily concerned with providing movie showtimes. I sent it an IM with the name of a movie and my town, not sure what to expect. This woke it up, and it sent back instructions, with a few options, including sending a title/actor/whatever or typing ‘menu’ for a..erm..menu. I fired off a movie title, and immediately got back a message prompting for a zip code; following that was a crudely rendered table of theaters and showtimes, with the option to type the number next to the theater for all showtimes at that theater.
It actually reminded me of interacting with an IVR system on the phone, except way faster. I was rather shocked to go from nothing to local showtimes in a few seconds; that’s about how long it takes to load the ads on movietickets.com or even moviefone.
So, I wonder what other compelling services could be composed using the chatbot interface idiom….
Still Don't Get Google Base
I commented recently about how I don’t get Google Base. I read a post by Russel Beattie wherein he describes his epiphany about how this will all work.
He quotes Bill Burnham:
One need look no further then the detailed XML Schema and extensive RSS 2.0 specification to realize that Google intends to build the world’s largest RSS “reader” which in turn will become the world’s largest XML database.
No, I’m still not feeling it. The most significant thing that I can imagine is the online content world converging on some generally accepted schema elements modeled after Google Base’s most popular attributes. Things like price, description, etc.
But, let’s say you can feed Google Base with an RSS feed. What does that change? Why would you put content in Google Base in the first place? Who derives value from it? Are more structured searches based on somewhat-typed attributes really so worthwhile as to be fawning over them?
I’m sure there’s a brilliant insight somewhere in Google Base, that I’m just not imaginative enough to see. But, I don’t think Russel Beattie or Bill Burnham have seen it yet either.
Playing with Google Analytics in Drupal
Today I learned of a new Google endeavor, Google Analytics. It’s basically traffic analysis software, mostly targeted to advertisers (in fact, it’s free to AdWords customers), but also useful to bloggers and other amateur content authors. Since I’m not an AdWords customer, I’m limited to 5M page views (per month, presumably); that leaves me with a margin of barely 4.999 million.
At any rate, it requires you to include some Javascript on your site, not surprisingly.
I’m putting in it page.tpl.php, which is the template that governs page display. Google says it goes in head, after meta. As you wish.
Right now the Google Analytics site is painfully slow; it took a few minutes just to open the initial registration screen. Since it’s typically spartan Google fare, I assume this indicates back-end load. This is consistent with other Google betas; Google Reader was similarly unusable in the first days.
Ok, I clicked ‘Check Status’ to force the system to re-check for the javascript, and it’s now in ‘Waiting for Data’ status, which indicates:
Analytics has been successfully installed and data is being gathered now. Your first reports will be ready within twelve hours.
Wow; I can’t wait that long. Oh well. I’ll be back in 12 hrs…
A developer skill continuum courtesy Bill
My co-worker and boss, Bill, posted a concise, insightful article on the skill continuum of IT developers. I worked on the same project with him for a couple years, and feel like we’ve both explored pretty much the entire range of the spectrum.
A further wrinkle in the IT development experience (as in all multi-human activities) is along an orthogonal axis in humanspace: personality. Sometimes the ‘detrimenal’ and ‘useless’ devs are sociable, pleasant, likable as people, perhaps even ingratiating. Similarly, the ‘good’ and ‘heoric’ devs are occassionally insufferable assholes, or more likely emotionally stunted borderline-autistic introverts (like, uh, me). I personally have fired a borderline plugger/good developer due to a toxic personality, and I’ve knowm a couple pluggers who really needed firing, but were such decent people that the economically obvious termination decision was not so clearcut.
For those on the top of the pile, working with the mainstream masses can be quite trying at times, as Bill relates. I suspect this is a problem at even the most elite of organizations; even with an elite population, some are more elite than others. The answer is clear, and I’ve known it all along: humans are not worth the hassle.
64-bit 1TB File Server (Or, 'How to dispose of $2k')
For roughly a year now, I’ve been tweaking the specs on a 1TB file server I keep meaning to build. When I started the project, I was confined to artemis’ meager ~30GB RAID storage. I added a 120GB RAID 1 volume running off a Highpoint RocketRaid PATA RAID controller connected to aragorn, which helped a little. Unfortunately, aragorn ate shit and died, and one of the 120GB drives in the RAID array seems damaged, as I can’t duplicate the primary drive onto the secondary now that I’ve moved the controller and drives over to boromir.
When I upgraded to prospertine, with 3×75GB Western Digital 10kRPM SATA drives (two volumes; 1 RAID0 volume @150GB, and one POD volume @75GB), the situation improved, but I’ve already maxed prospertine out, primarily with my VMWare virtual machine collection and files I pulled of the ailing 120GB array on boromir.
With my digital photography habbit getting worse, my compulsion to create VMs growing more pronounced, and my insatiable appetite for ISOs of pre-release MS software and new games, my disk space situation is dire. It may be time to make the leap into terabyte land.
The latest config, as of right now, is a Prescott 531 processor (chosen for a combination of EM64T 64-bit extensions, hyperthreading, and price), an Intel server mobo based on the E7221 chipset, a Highpoint RocketRaid 2220 SATA-II RAID controller, and five WD Caviar 250GB 7200RPM SATA II disks. I’m also throwing in some GigE gear; prospertine has on-board GigE, as does the mobo included in this config, and I’ll want that speed if I’m to offload a bunch of data onto a network share.
The plan is to create a single RAID 5 volume, which lacks the blistering performance of RAID 0, but by distributing parity information across the volume, provides the redudancy I require. Under RAID 5, a volume with n same-sized disks with m bytes per disk has a usable capacity of *m bytes; in other words, you lose one disk’s worth of storage to the parity blocks.
Given this math, my five 250GB drives will provide an even terabyte of usable storage. The other Seagate Barracuda you see in the wishlist is to replace the failed drive from aragorn’s/boromir’s array: I’ll be using a RAID 1 volume composed of the two 120GB PATA drives as the system drive for the new file server, so I need them working.
I’ll run FreeBSD, either 5.4-RELEASE or 6.0-RELEASE, depending upon what’s available. FreeBSD supports the EM64T 64-bit extensions, so I’ll be able to run in 64-bit mode. FreeBSD is also supported by Highpoint, which provides both binary and source versions of its drivers.
I could run Slackware Linux instead, but as Linux has emerged as the bohemian rebel OS of choice, I’ve found myself drawn to the niche BSD OSes instead.
I considered a Pentium D (dual-core), which has alot of appeal to me, but the increased cost and power requirements didn’t seem worth it. I also considered AMD’s 64-bit line, but I’m less familiar w/ the ancillary components (mobo etc) so went w/ what I know.
I found a nice X-Alien case, with a huge 450W power supply. I used a power supply calculator to estimate my power needs, since I was concerned a total of 7 SATA drives would overload a normal PSU, but I was surprised to find an estimate of slightly over 300W. I guess those modders w/ 500W PSUs are running 200W video cards? Perhaps SLI configs really suck down the juice.
New VMWare Player!
I just got an email from VMWare announcing the new VMWare Player. As the name implies, it allows anyone who downloads the free player to run virtual machines created w/ VMWare Workstation.
The download site seems to be pretty heavily loaded, so I’ve not had a chance to review its limitations, but if they’re reasonable, this could be huge. Perhaps some enterprising group will create a stock Windows XP image, activate all the DRM players on it, and share it the world over for communal music/video access.
Already, Oracle, IBM, BEA, Red Hat, et al are offering prefab VMs for evaluation purposes, which has significant appeal for both the vendor and the client. Clients don’t get stuck w/ reams of resource-hungry eval-ware bogging down their primary machines, and vendors don’t have to support the half-upgraded Windows 98 bastard box which inevitably someone will run an eval on.
Now, if only they were smaller, and not monolithic; in other words, I’d like to be able to download pre-installed components which could just be folded into an existing VM, but I’m not sure how that would work anyway.
Update:
It’s even better than I thought:
On Windows hosts, VMware Player also opens and runs Microsoft® Virtual PC
and Virtual Server virtual machines and Symantec® LiveState Recovery system images.
Kickass!
Some limitations:
- VMware Player does not support Virtual SMP. You cannot use VMware Player to power on a virtual machine that has more than one virtual processor assigned.
- VMware Player can run only one virtual machine at a time. You must close the virtual machine currently running in VMware Player before you can open another virtual machine.
And, obviously, you can’t create new virtual machines.
Still, I’m amazed VMWare is giving away so much virtualization technology. They’ve really thrown down the gauntlet to MS with its too-little-too-late Virtual PC offering.