Recovering a Broken Fusion VM After an OS X Kernel Panic

Pursuant to the previous issue with kernel panics, the client reported that one panic had happened while she had a Fusion XP virtual machine running, now the VM wouldn’t boot. It errored out with a “Missing SYSTEM32 directory” error (or words to that effect),and admonished me to start from an installer disc and run the recovery console.

This requires either lightning reflexes, or editing the VM settings to get a longer POST delay. VMWare, conveniently enough, has an article on this. Getting the system booted from CD and running the discovery console didn’t help, but I did notice during the process that the VMDK file for the problem VM had an associated lockfile, and there was a VMEM file present. On a whim, I moved these files to the trash (actually, it was less a whim than an educated guess…lockfiles are a continual irritation when moving VMs between machines).

Et voîla (that’s probably the wrong accent mark), the VM booted, ran chkdsk, and recovered nicely. The moral of the story is: Look in your VM bundle, and dump lock and vmem files before assuming the VM is actually bad.


Cisco VPN Client, Ralink Wi-Fi Drivers and Random Kernel Panics

A client came in recently with a series of odd, apparently unpredictable kernel panics. A few minute of log trolling led to four panic logs. Knowing that she’d installed Ralink USB Wi-Fi device drivers on the machine under Snow Leopard, I initially suspect them as the root cause (I’ve never liked Ralink’s ugly, kludgy software, and tend to suspect it on principle.)

Two logs showed a crash on loading Airport extensions, and two on USB drivers, but the common link was the extension loaded just prior…in each case, a kext for Cisco’s VPN client. This surprised me not just because I’d expected to find the Ralink stuff, but because we switched from the Cisco native VPN client to AnyConnect over a year ago. Apparently, a significant portion of the old VPN client had remained on her system, and been carried forward through at least one upgrade cycle.

A check of the /private/var/opt/ subdirectory showed most of the client still around. Running the vpn_unintall script removed most of the bits (which I double-checked by visiting each location specified in the script. I then manually delete the cisco_vpn subdirectory in opt. I also removed the Ralink utilities, including a preferences file in /Library/Preferences and a directory in /Library/Frameworks. Ralink provides no uninstaller.

So far, no more kernel panics. Possibly my distaste for both Cisco’s VPN client and Ralink’s just is justified.

UPDATE: This situation is a good argument for keeping a copy of John Welch’s Kext Lister utility handy.

Creating a custom Sophos Anti-Virus for Mac package and deploying via script

First, some necessary explanation.

We use Sophos Anti-virus on Windows and OS X computers, managed by a central Sophos Enterprise Console (SEC). Clients get their settings from the SEC after installation, including the critical AutoUpdate settings. Now Sophos would have you believe, on the basis of this (woefully outdated) document that you can use Sophos Update Manager for OS X (login required) to simply inject your AutoUpdate (and other) settings and get a neat, distributable metapackage.

This, in our case at least, is a filthy lie. Or at least a half-truth in dire need of a shower and shave.

You can prepopulate all your settings in the metapackage, and it will run. Until you try to alter your AutoUpdate settings on the SEC and distribute them. For reasons unknown to us, the clients don’t get the message. We learned this the hard way, after shift in the address of our SEC left us directing clients to uninstall and reinstall the software just to get updated settings. Ugly.

After a few calls to technical support, it seems you have to distribute not a simple metapackage file, but rather an entire directory with several ancillary files that Sophos requires to correctly configure trusted communications with the console. Ugly.

So here’s our entire process for creating and distributing a custom installer for Sophos. This process assumes a reasonable level of familiarity with: Sophos Enterprise Console, basic shell scripting for OS X, Sophos Anti-virus for OS X, and the Casper Suite. If anything requires clarification, I’m happy to expand. As always, YMMV.

  1. Download and install Sophos Update Manager (SUM) on your OS X box. This requires a MySophos login.
  2. Launch SUM and authenticate. Provide your Sophos EM credentials in the usual and customary locations.The “Download Updates To” location is fixed to /Sophos Anti-Virus/ESOSX for inexplicable reasons, but can be modified using special instructions from Sophos. I’ve sanitized my copy of these directions a bit and pastebinned them. This is a total hack, for which I take no responsibility, but it may be useful to you since it lets you use a local CID and put it where you like. I prefer to copy the SEC’s authoritative CID.
  3. Continuing with my way, uncheck Check for updates from Sophos, then click Apply, then quit SUM.
  4. If you have rights to your SEC: Open SEC (either on a Windows box or a VM). Click Update Managers in the toolbar. Right-click on your update server, and select Update Now. Download status will change to “Downloading binaries”. When this process completes, your CIDs will contain up-to-date binaries, and you may exit SEC. If you don’t have SEC rights, ask for them, because you should have them. In the meantime, ask your SEC administrator to update the CIDs.
  5. Back in OS X, you need to get a copy of the CID for SAV Mac. If, like us, you have this location shared and you have access, just connect to your console’s fileshare and navigate to the CID subdirectory specified by your SAV subscription (usually ending in ESOSX or ESCOSX). Copy the entire contents of this this folder into /Sophos Anti-Virus/ESOSX/ on your Mac with SUM installed, replacing all contents of the target directory. Authenticate as required. You may find it easier to delete the contents of the target first, especially on 10.7 or 10.8. For reference, you should be seeing these files: cidsync.upd, customer_ID.txt, manifest.dat, master,upd, root.upd, sdf.xml and Sophos Anti-Virus.mpkg. If you don’t have rights or a convenient way to access the CID, ask an admin to send you a zipped copy of it, and proceed from there after (again) asking for a better setup.
  6. Relaunch SUM and authenticate. Switch to SAV Preferences. Configure all your preferences as you wish, but DO NOT TOUCH THE AUTOUPDATE TAB, or you won’t be able to save your settings! If you so much as open the AutoUpdate tab, SUM will demand that you fill it out, and refuse to save if you don’t (necessitating a restart from step 1). Apply, then quit SUM.
  7. Copy /Sophos Anti-Virus/ESOSX to your desktop.

At this point, you can redistribute the directory as a zip or DMG file and run the installer from it, but the entire contents of the directory must remain together for the installer to correctly configure its connection to the SEC.

Now to make this thing distributable via JSS (or, less desirably, DeployStudio). A package won’t work, since the ancillary files have to stay with it. Composer won’t work, because each install keys itself securely to the SEC, and redistribution breaks that link. Luckily, though, we already put the zipped installer in an HTTP-reachable location (for reasons outside the scope of this writeup). So rather than reinventing at least half of the wheel, I’ll use a script — distributed via JSS policy — to pull that zip file down to the client, run the installer silently, and remove itself. Here’s the commented script. If you use a DMG instead of a zip, you’ll need to change the commands and directories, of course, but the core process should still work. In DeployStudio, a postponed installation of a payload-free package using this script as a postflight should work.


# Make a working directory, after checking for and removing any leftover instances from a broken install

if [ -d /private/tmp/sophos/ ]; then
	rm -r /private/tmp/sophos/
	mkdir /private/tmp/sophos/
	logger "Sophos install temp directory created after removing old directory"
	mkdir /private/tmp/sophos/
	logger "Sophos install temp directory created"

# Get installer zip. Use whatever name you like. Our docs refer to Install Sophos, so that's what I'll use here.

curl > /private/tmp/sophos/Install\ Sophos\

# Decompress

cd /private/tmp/sophos/
unzip Install\ Sophos\

# Install. Normally, installer requires sudo, but the jamf binary runs with admin rights, and using sudo here breaks the script.

installer -pkg /private/tmp/sophos/Install\ Sophos\ Mac/Sophos\ Anti-Virus.mpkg -target /

# Cleanup

cd /
rm -rf /private/tmp/sophos

exit 0

Upload this script to your JSS, then set a policy to deploy it, and you’re done. Ours runs when a machine is first set up, then never again. Once a client is tied to the SEC, updates and even major version upgrades are typically very smooth.

I realize this seems like a lot of documentation for what may not be a very common situation. Still, figuring it out took long enough and was such a hassle that I figure if I save one person the headache, it’s worthwhile.