Jon’s Project Blog

Since my current laptop (2004) is pretty much on its last legs and I don’t feel like wasting or looking for my XP windows keys, I figured I would just migrate my laptop. Similarly, my desktop that was built in 1999 (Pentium 4 A – no hyperthreading) with 512 Mb of ram and non-functional USB ports and dead sound card, has some useful data on it I figured I might as well start with that and see how much of a hassle it would be to backup those systems. I figured the best way would be to some how ‘dd’ a hard disk and then run it in VirtualBox. However, I remember reading something about problems with Windows “memorizing” hard disk information and in general ‘dd’ is super slow.

Luckily I ran across this Lifehacker article via a quick google search. It doesn’t contain much information, except that it mentions this new (Oct 9,2009?) SysInternals tool called ‘disk2vhd’ which supposedly makes imaging windows drives a piece of cake. Some other Ubuntu googling searches resulted in similar recommendations but also noted that there could be some IDE/SATA driver issues and that you had to fix these driver issues “apriori”. I won’t go into the details since it seems all of this can be fixed in the image afterwards.

So I went about my way, running ‘disk2vhd‘ on 2 windows drives which totaled 50+ Gb. This went smoothely on my abandoned desktop and it took about an hour or two to create the VHD. I copy over the VHD, which took a few more hours (did it overnight). However, when I tried to run the VHD in virtualbox it said I had a grub error (error 21). I figure this has to do with the fact that the MBR points to grub, which lives on the hard disk I didn’t image. Anyways a quick google resulted in grabbing SuperGrubDisk. I just ran it, and selected the “Win” option as I had no intention of creating a dual boot VHD system. This fixed the grub problem.

Windows XP started booting up and then I ran into the infamous “0x0000007B” boot BSOD issue. It has something to do with windows not liking a difference in IDE hardware. At this point another quick google seemed to indicate I would have to do the “apriori” fix step. However, I wasn’t about to go spend another 8 hours trying to backup my computer as I already spent enough time trying to make it work. Luckily I ran across this confusing forum post, which hinted that the Ultimate Boot Cd for Windows (UBCD4Win) had something called “Fix_hdc” that could possibly fix the drivers issue. Luckily I had a copy of a UBCD4Win iso lying around. So I load up UBCD4Win iso and find ‘fix_hdc’ and just selected the “Usb option”.

Magically, it seemed to do the trick, and after a reboot of the virtual machine, everything worked flawlessly.

Summary:

1. Run “disk2vhd” from SysInternals on the drives you want to image. (Optionally, run mergeIDE and so forth to possibly avoid step 3.
2. If you had some multi-boot grub setup and grub doesn’t live on the same partition (which it probably doesn’t), use a grubrecovery disk such as SuperGrubDisk.
3. If you get the “0x0000007B” BSOD error during boot, just grab UBCD4Win, boot it up, and then run Fix_hdc. It should be in Start>Programs>Registry Tools>FIX_hdc>Fix Hard disk/USB

Overall, the process was fairly painless. Mostly just clicking a few “auto” buttons. Virtualbox seems to run well, and so far the experimental 3D drivers seem to play relatively nicely. I’ve watched some movies through VM and played a few games (none of them super resource hungry). Music seems to work fine. Ironically, when I ran Windows 7 earlier this year, the audio did not play smoothely, and video clearly didn’t work. Seems that windows media player plays video much more jerky than VLC which is less noticeable. I don’t remember off-hand what my old vlc settings were though, so I can’t comment much.

I’d say the conversion was a definite success, and I will do the same with my old computers so I will have access to software that I can no longer fine that I had installed on my old machines. At any rate, I now have a “working” backup of my old system which can run in several VM packages (VirtualPC, VirtualBox, VMware). I’m fairly certain that such formats will continually be updated (virtualization seems to be the future of everything), and thus it should be possible to keep a copy of my old machines with me on my new computers in the future.

Several times I’ve had to save authenticated wiki content to my phone to view it on long flights (across the US). I found wget + Firefox 3.0 (Export Cookies extension) to be quite sufficient. The extension page explains a bit about how to use wget.

The basic strategy is:

1. Login to website, and export cookies and session information to a text file (“cookies.txt”).
2. Run wget and try to figure out what a reasonable download rate is. Some websites obviously don’t like bots, and so you may need to look through the wget options to slowly go throught the site. You may also need to periodically update your cookies or session info via the text file so don’t be too greedy.

As usual, you may want to look into the following wget options:

-I,-r,-l,-p,-k,-np

You may need to play around with the timing as websites generally don’t like being spidered. It also may be be illegal should check the Terms of Service.

The following is a sample commandline used to save webpages and attachments to a directory for offline use. I’ve noticed that firefox on the mac has issues with lack of extensions from wiki sites.

Example:

wget --random-wait --load-cookies=cookies.txt --save-cookies=cookies.txt --keep-session-cookies -r -l 2 -p -k -I /pages/ http://***:8080/pages/viewpage.action?pageId=xxxxxxx download/attachments

Just a quick post on the simple greasemonkey script that rips UCLA Bruincast video links (obviously you need access). I made it for my brother, so he can download lectures for classes he would like to save. Here’s the short greasy script. (Remember, rename the urls from “http:” to “rtsp:”). To grab the DSL links simply replace ‘LAN’ with the text of the link. Enjoy!

// ==UserScript==
// @name           BruinCastPageLinks
// @namespace      hawflakes.unoc.net
// @description    Grabs Links on a page
// @include        http://www.oid.ucla.edu/webcasts/courses/*
// ==/UserScript==
 
// Greasemonkey Script written by Jonathan Wong, Copyright 2009
// Script is freeware (Use at your own risk!).
 
Links = findXPathNodes("//a[text()='LAN']/@href");
PrintString="";
 
for(i=0;i<links .snapshotLength;i++)
{
	PrintString=PrintString+Links.snapshotItem(i).value+"\n";
}
 
GM_openInTab("data:text;charset=UTF-8," + encodeURI(PrintString).replace(/&nbsp;/g,"&amp;nbsp;"));
 
function findXPathNode(xpath, start,doc)
{
	var result = (doc == null ? document : doc).evaluate(xpath,(start == null ? document : start), null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE ,null);
	return (result.snapshotLength > 0 ? result.snapshotItem(0) : null);
}
 
function findXPathNodes(xpath, start,doc)
{
	return (doc == null ? document : doc).evaluate(xpath,(start == null ? document : start), null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE ,null);	
}
 
function elem(tagname,content)
{
	var ret = document.createElement(tagname);
	ret.innerHTML = content;
	return ret;
}
</links>

So apparently mencoder only seems to work for asf,wmv,avi,mpeg streams, and does not like real media streams (I think this is purely due to the fact I do not have real media codecs on this machine and therefore -ovc/-oac options would not work anyway). So I’ve defaulted back to using mplayer for ripping the real media streams. Fortunately, I had remembered seeing this, which pretty much explains what one needs to do.

For UCLA, the linked .rm file is actually just plain text that gives you the actual url. However, in the case of UCLA (like stanford), one can simply replace “http” with “rtsp”.


mplayer -noframedrop -dumpfile out.rm -dumpstream rtsp://url.rm


The most interesting piece of information is that on windows, it is possible for mplayer to rip streams even if it can’t actually play it. For example, my Windows machine (what I’m using atm) does not have real player or alternative/hacked rm codecs on it, and therefore I can’t actually play .rm. However, I was able to rip it and someone else was able to play it. This does make sense.