apocryph.org Notes to my future self

"Structured Wikis" – Seriously, is this the best we can do!?

Recently as part of my day job at AppAssure Software I migrated some never-used test cases from a Word document into our internal wiki, which runs FlexWiki. I wanted something more dynamic than a list of test cases with links to each one; I wanted to be able to group them as smoke tests, exhaustive tests, tests run on XP, etc.

It turns out FlexWiki has a surprising level of extensability. I added a field, Tags, to each test case topic, and used its quasi-functional programming language, ‘WikiTalk’, to build queries for all topics whose Tags attribute contained the tag I was looking for. The WikiTalk code is not pretty; it’s some sort of primitive object oriented environment that uses lamba functions to implement its functionality, and seems to be missing such things as logical operators.

Anyway, this experience got me to thinking about what more sophisticated Wiki packages that are designed as Structured Wikis might be capable of. I remembered from years ago playing with Twiki that it was the closest Wiki tool I found that actually thought it could replace simple database apps with Wiki technology. What I didn’t remember from years ago is how clunky it is.

First, Twiki is written in Perl. Those of you who know me understand that I’m a very tolerant, open-minded, enlightened person, but you also know that I hate Perl, and will probably hate any app written in Perl. Yes, I know, I’m no better than the Guilded Age nativist labor boss shitting all over the Irish because they were born in the wrong place. Too fucking bad. Perl was developed at a time when dynamic languages were still being explored, and before object oriented programming was well-understood. As a result, everything it does other than regular expressions feels lame and forced, with no clear idioms to aid a hapless developer desperately struggling to translate the line noise in his editor into code he can understand.

Python is better. Ruby is better. C is better. God help me, Java is better. If I need a dynamic language, I’ll use Ruby. If for whatever reason (performance, shitty libraries) Ruby isn’t available, I’ll use Python. If for whatever reason Python isn’t available (and I can’t imagine why it wouldn’t be), then I’ll give up. On the other hand, if a static-typed OO language is called for, C#, C++, or C are all options. The only reason I can imagine needing Perl is if I’m maintaining or extending a Perl tool, in which case I’ll just quit and get a job that doesn’t involve typing $, @, and ~ all day long.

With that rant behind me, you can probably imagine what’s coming. My sense of Twiki’s heritage is as one of those ‘simple’ tools, developed by people who don’t want to do anything ‘fancy’, who say ‘Keep it Simple Stupid’ at least eight times a day, and who bludgeon dissenters with the hubris of simplicity.

Don’t get me wrong. The quest for simple solutions is worthwhile, and I appreciate an elegant, simple solution as much as the next guy. However, after Einstein said ‘Make everything as simple as possible’, he admonished ‘but no simpler’. The simplists sometimes forget this last bit, perversely leading to more complexity. Twiki is, imho, a perfect example of that.

To start with, Twiki does not use a database engine to store its content. Each topic is a text file. I know what you’re thinking. “God, that must suck wind”. Pretty much. I can almost hear the rationale now. “You can run it anywhere you can run perl!” “You don’t need an admin to stand up a slow, resource-intensive database for you” “We’ll make it so everyone can run a wiki!” “Just extract the tarball and run!” “It’ll be so simple, yay!” Right. And you can see how it probably looked that way in the beginning.

Unfortunately, this imposes a significant architectural limitation that makes advanced searching or indexing or joining of content across topics either resource-intensive or impossible. Not a problem for a lightweight, content-oriented wiki platform storing a loose collection of unstructured content for beard-and-ponytail-wearing Perl hackers, but a serious problem if one is building a Structured Wiki.

The whole point of a structured wiki is to come at traditionally RDBMS-powered applications like knowledge bases and bug lists from a Wiki perspective. In other words, starting with no structure and imposing only enough to solve your problem, vs the RDBMS idiom of rigid pre-defined structure throughout the system. As a design philosophy I really like this idea, and if anyone ever actually gets the freedom/structure balance right they’ll have a kickass tool on their hands. Twiki, however, isn’t that tool.

Let me walk you through building your first Twiki application, from the perspective of an experienced programmer and all-around hacker with strong intellectual curiosity and a fairly average shit tolerance:

• Download Twiki tarball
• Extract Twiki tarball. You’re surprised and pissed that Twiki tarballs violate the long-standing tarball idiom of containing a single directory under which the actual artifacts are extracted, instead littering whereever you extracted the tarball with a bunch of files and folders.
• Create an .htaccess file in the bin directory so the damn perl scripts without a .pl extension will run when you access them. Bonus points if you figured this out without additional googling. Double bonus points if you didn’t use any expletives during the process.
• Run the bin/configure script in your browser. Experience fond memories of new Star Trek: DS9 episodes and the Spice Girls debut album as you wade through the el-cheapo late-90s configuration UI. Remember stupid, keep it simple!
• Mistakenly think that when you’re first prompted for a config password, you’re being asked to set one up, since after all you’re running this for the first time so there’s no existing config password. When you enter your password and click Save and it works, assume that means your password has been applied, even though in fact that’s not what’s going on and your config UI is wide open to anyone with the URL
• Plow through a bunch of hard-to-read config options looking for warnings and errors, fixing incorrect config settings all the while. You think this config UI was designed by Uday Hussein as a Father’s Day present for Saddam, but that’s because you’re not keeping your eye on the simplicity ball.
• At last, your Twiki wiki is set up.
• Navigate to the root Twiki folder. Hmm, that’s not right, this is the Twiki docs, not the Twiki install. WTF?
• Figure out the way to access Twiki is at the bin/view URL; create a .htaccess file in the root to implement this redirect automatically, and wonder aloud if this particular bit of simplicity was really necessary
• Click around the new Wiki trying to figure out what the fuck you’re supposed to do next. How do you create a new application? You discover your cat is a convenient whipping boy in lieu of the fucktard who came up with this whole Structured Wiki idea
• Through brute force clicking and sheer dumb luck, you find the TWikiForms page which explains how to create a TWikiForm, which apparently is where you define what a not-simple database programmer would call a schema.
• You create a new topic, FuckThisWikiShit, and admire the symmetry of a four-word wiki title composed entirely of four-letter words; it’s like a Wiki perfect square or something
• You create a table in the new topic, one row for each field you want in your application. After looking at the virtually unreadable table, with all of its |s, you wonder exactly what the fuck is simple about this.
• You wonder aloud what the fuck you do now. The TWikiForms topic says something about a template you add the form too.
• The TWikiForms page also says something about using FormattedSearch to find all the topics with the same form. Seriously. So in all the ‘simple’ sample apps, the table of records you can add to or edit was really just a query for all the topics associated with a given form. How’s that for granuarity. How can you search more specifically? Oh, wait, there’s no database, so any more granular search would have to trawl through every topic record looking at its form fields.
• You run across the DBContribCache add-on, which–wait for it–builds an in-memory cache of the TWiki content database. Hurray! We’ve arrived at a database-backed app through the back-door, and with a hand-rolled solution instead of something off the shelf. It’s so…simple!
• You try to install the DBContribCache add-on, only to find the tarball seems corrupted, and the installer is too stupid to create destination folders it needs before doing copy operations, so you manually create a handful of folders through the TWiki tree until the installer works
• You say ‘fuck this, I could’ve build this application in Rails by now, with less code and more docs’
• rm -rf twiki

That is more or less my journey from TWiki neophyte to TWiki hater. Your mileage may vary.

I’ve heard that JotSpot was a structured wiki built by, you know, professionals, but since they were bought by Google they’re closed to new users so I can’t tell for sure. The Wikipedia article on Structured Wikis lists only Twiki and JotSpot as structured wiki tools. Seriously, it’s 2007, this is where we are? I started writing notes to myself about how useful this type of functionality would be back in ’02, which was the first time I evaluated and disregard Twiki, and we’re in the same fucking place today.