In my quest to wrap Wireshark to dissect packet captures into something Ruby can handle, I’ve eliminated the PDML export option, and am now trying to write a Ruby C extension to wrap the Wireshark libraries.
As a Windows user of open source software, I’m used to being a second class citizen, worth little more than some gruding attention from a UNIX programmer who can barely be troubled to scrape together a shitty
nmake makefile for a twelve year old version of Visual C++. It’s understood that without installing Cygwin and accepting alot of UNIX-on-Windows kludginess, you’ve basically a snowball’s chance in hell of building any moderately complex open source tool under Windows.
I finally managed to build Wireshark from sources using the instructions on the Wireshark dev site, but that’s only because Wireshark is one of the most Windows-friendly open source projects in the history of mankind. Ruby extensions, on the other hand, exemplify perfectly the my-way-or-the-highway conceit of the UNIX developer community. Allow me to explain:
If all you know about Ruby extensions you know from reading the Dave Thomas book Programming Ruby, then you can’t possibly understand what I’m saying. You know that building a Ruby extension in C is as easy as whipping up a few lines of
ruby to generate the
make install. Like so much else in Ruby, it Just Works.
Uh, yeah. If you’re running an operating system that the Ruby digerati have blessed as worthy, that’s probably how it works. Certainly under FreeBSD, most Linux distros, and Mac OS X, it’s that easy. I can only assume that us Windows users are such fuckwit n00bs we deserve to suffer as second class citizens; it’s what we get for betraying Le Resistance.
You see, in most UNIX environments, you can make a few simple assumptions. You can assume the system has a C compiler installed, that there’s only one compiler that could be used to build system binaries, that
cc are in the path, and that you can let the compiler more or less decide what runtime libraries to link with, knowing that whatever it chooses will run anywhere on the machine. You can further assume that you’re building binaries for your machine alone, since everyone knows how to do builds so why bother with binary tarballs?
Under Windows, each of these assumptions is wrong. Windows doesn’t ship with a compiler, and even if one is installed, there are several flavors of Visual C++ ranging from Visual C++ 6.0 circa 1999 through Visual C++ 2005 SP1 circa 2006. Though each version is similar, most have different runtime libraries that do not interoperate, and later versions have a bizarre manifest system whereby runtime libraries are linked at runtime. If you write an executable to use Compiler Platform A, and link with a DLL built with Compiler Platform B, any attempts to use runtime objects like
FILE pointers or memory management functions between the EXE and the DLL will end in tears.
This brings me back to the Ruby extension mechanism. The idea is simple enough: use a Ruby script to describe the headers, libraries, and various other bits an extension takes as input, and let Ruby itself generate a makefile custom tailored to your environment. This sucks for a couple of reasons: compiler version and runtime library.
It seems the one-click installer of Ruby 1.8.4 for Windows was built using Visual C++ 6.0, since the
config.h file in
c:\ruby\lib\ruby\1.8\i386-mswin32\config.h has this little nugget:
#if _MSC_VER != 1200 #error MSC version unmatch #endif
For those of you born after 1990,
1200 is the value for
_MSC_VER used to indicate code is being compiled under Visual C++ 6.0. So, right off the bat, any code that
config.h (that is to say, all Ruby extensions) will fail loudly when built with anything other than Visual C++ 6.0. Why, you might ask yourself, would the developer responsible for this code limit it to the oldest version of Visual C++ still in use today? I’ve nfi, especially since C/C++ purists have no better friend on Windows than Visual C++ 2005, and no worse enemy than Visual C++ 6.0. But, be that as it may, the decision was made, and we suffer for it.
You might be wondering ‘what would happen if I remove that code and build something with Visual C++ 2005?’. Great question; it turns out it can be made to work. First you must comment out those lines in
config.h. After you do that, and use
ruby extconf.rb to generate a
Makefile for your extension, you have to run
nmake (Microsoft’s version of
make). You get this not-at-all-reassuring output:
Microsoft (R) Program Maintenance Utility Version 8.00.50727.762 Copyright (C) Microsoft Corporation. All rights reserved. cl -nologo -I. -Ic:/ruby/lib/ruby/1.8/i386-mswin32 -Ic:/ruby/lib/ruby/1.8/i386-mswin32 -I. -MD -Zi -O2b2xg- -G6 -c -Tcrcapdissector.c cl : Command line warning D9035 : option 'Og-' has been deprecated and will be removed in a future release cl : Command line warning D9002 : ignoring unknown option '-G6' rcapdissector.c cl -nologo -LD -Fercapdissector.so rcapdissector.obj msvcrt-ruby18.lib oldnames.lib user32.lib advapi32.lib ws2_32.lib -link -incremental:no -debug -opt:ref -opt:icf -dll -libpath:"c:/ruby/lib" -def:rcapdissector-i386-mswin32.def -implib:rcapdissector-i386-mswin32.lib -pdb:rcapdissector-i386-mswin32.pdb Creating library rcapdissector-i386-mswin32.lib and object rcapdissector-i386-mswin32.exp
I like to think most competent programmers are conditioned to feel aversion and discontent when a build generates warnings, particularly warnings like “‘-G6’? What the fuck does that mean? Screw it, I’m ignoring it” and “Uh, nobody uses ‘Og-‘ anymore, what the hell is wrong with you?”. Apparently, I’m a naive little fuckwit, because you’re just expected to suck up those warnings if you want to build Ruby extensions under Visual C++ 2005.
Think you’re done yet? Fuck no. If you try to pull a
nmake install now, it’ll look like it worked:
Microsoft (R) Program Maintenance Utility Version 8.00.50727.762 Copyright (C) Microsoft Corporation. All rights reserved. install -c -p -m 0755 rcapdissector.so c:\ruby\lib\ruby\site_ruby\1.8\i386-msvcrt\rcapdissector install -c -p -m 0644 .\lib\field.rb c:\ruby\lib\ruby\site_ruby\1.8\rcapdissector install -c -p -m 0644 .\lib\packet.rb c:\ruby\lib\ruby\site_ruby\1.8\rcapdissector install -c -p -m 0644 .\lib\packet_element.rb c:\ruby\lib\ruby\site_ruby\1.8\rcapdissector install -c -p -m 0644 .\lib\protocol.rb c:\ruby\lib\ruby\site_ruby\1.8\rcapdissector
Hooray! But wait. Let’s try it out with
irb irb(main):001:0> require 'rcapdissector/rcapdissector' LoadError: 126: The specified module could not be found. - c:/ruby/lib/ruby/site_ruby/1.8/i386-msvcrt/rcapdissector/rcapdissector.so from c:/ruby/lib/ruby/site_ruby/1.8/i386-msvcrt/rcapdissector/rcapdissector.so from c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' from (irb):1
require and the
LoadError is a stern message box saying:
--------------------------- ruby.exe - Unable To Locate Component --------------------------- This application has failed to start because MSVCR80.dll was not found. Re-installing the application may fix this problem. --------------------------- OK ---------------------------
Bollox! How can it be missing
msvcr80.dll? That’s the Visual C++ 2005 runtime DLL, which is installed when Visual C++ is.
If the extent of your Visual C++ 2005 experience is with using the IDE to build stuff, you probably have no fucking idea what’s wrong. If, like me, you’ve had to deal with
nmake makefiles written by old
make curmudgeons who refuse to use a GUI, then you totally feel my pain.
You see, Microsoft thought it was bad that it was so easy to install the wrong version of a DLL and break a bunch of apps (so-called DLL Hell). To fix it, they borrowed an idea from the .NET world; dynamic linking based on a cryptographic hash of a file, not just a file name. This way, you can have multiple versions of
fuckit.dll installed side-by-side, and apps automatically load the right one. Brilliant. We’ll call it…side-by-side installation. Brilliant.
The catch is that apps that link to DLLs need to have some additional metadata descriptors to point to these side-by-side DLLs. This metadata is called a manifest, and it’s embedded in each executable as a resource. When you do a build with the Visual C++ IDE, it generates a manifest to link the C++ runtime DLL, and embeds it in the executable automatically. When you build something with
nmake, well, you’re on your own.
The Ruby extension build tools don’t know/care about this, because they’re written by a guy who sees no problem building software in 2007 using a compiler that was EOLed last century. You, however, do care about this, because you insist on building Ruby extensions with a compiler that was released after you were born. So, after you do your little
nmake, and before
nmake install, you need to do this:
mt.exe -manifest rcapdissector.so.manifest -outputresource:rcapdissector.so;2
.so extension; the Ruby developer responsible for the Ruby extension build system is a cultural imperialist who seems intent upon forcing his vision of the world upon users of other operation systems, like for example operating systems that use
.dll to denote dynamically linked libraries).
Now, do a
nmake install and try the shit out in
irb irb(main):001:0> require 'rcapdissector/rcapdissector' => true irb(main):002:0> shitoutofluck=CapDissector::CapFile.new('like i give a fuck') => #<CapDissector::CapFile:0x2e7ca50> irb(main):003:0>
Fuckin’ A. There you have it. How’s that for ‘Just Works’.