apocryph.org

Today I’m trying to track down an elusive bug that only presents at a single customer site. We’ve narrowed it to something having to do with VSS, so now I need to force some VSS snapshots outside of our product. As it happens, the VSS SDK comes with just such a tool: vshadow. vshadow seems to have options to exercise just about every facet of the complex VSS API, which makes it a handy test probe. Just one problem; the version that ships with the VSS SDK is for Visual C++ 2005, and won’t build when put through VS 2k5.

Yes, yes, I know, binaries are included with the release. But what if you want to debug, or modify the code, or copy-paste it and get it to build in your own code? You’re screwed to the wall, that’s what. Here’s the secret handshake to get it building.

To start with, you go through the usual conversion wizard stuff. You build the XP version of the tool, which is missing some features. That builds fine, albeit with a torrent of scary security warnings. If you’re not willing to rewrite the code to use the new more-securer security APIs, add _CRT_SECURE_NO_WARNINGS to the list of preprocessor definitions.

That’s all fine, but let’s say you want to build the ‘Server’ version, which contains VSS features available only in Windows Server 2003. Well, in addition to the warnings you fixed above, you get this steamer:

1>revert.obj : error LNK2019: unresolved external symbol "long __stdcall ShouldBlockRevert(wchar_t const *,bool *)" (?ShouldBlockRevert@@YGJPB_WPA_N@Z) referenced in function "public: void __thiscall VssClient::RevertToSnapshot(struct _GUID)" (?RevertToSnapshot@VssClient@@QAEXU_GUID@@@Z)
1>C:\Program Files\Microsoft\VSSSDK72\TestApps\vshadow\bin\Debug-Server/vshadow.exe : fatal error LNK1120: 1 unresolved externals

Not. Helpful.

As it happens, I ran dumpbin /exports on vssapi.lib, and found that it does export ShouldBlockRevert, but thanks to C++ name mangling the mangled name is different. Why is it different? Because in vssapi.lib, the first argument to ShouldBlockRevert isn’t wchar_t, it’s unsigned short. “So what”, you’re thinking, “they’re equivalent”. And I don’t disagree, but the compiler treats them as different types for name manging purposes. What’s the fix? Well, disable the intrinsic wchar_t type in the C/C++ Language property page in the project properties (equivalent to the /Zc:wchar_t- switch if you’re one of the two people on the planet who build Visual C++ projects with makefiles).

Once that’s done, the LPCWSTR macro is defined to unsigned short, name mangling matches, planets align, and you can link. QED.

Tags: Migrated from Drupal, tech diary, visual studio, vshadow, vss

This entry was posted on Tuesday, September 4th, 2007 at 6:01 pm and is filed under Migrated from Drupal. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.