Jon’s Project Blog

For those of you that use tecplot, you know that while the interface is fairly straightforward and the resulting visuals look pretty good, tecplot has one major flaw. The animations are generated in with essentially the MSRLE codec from a decade ago. The codec’s color palette generally cannot handle continuous color flooding  in tecplot; if you try to open it up on other machines or convert them to say flash video or mp4, you will find that the the colors are all messed up. Sometimes the frame size in tecplot causes the video to not display correctly.

So to fix this issue, I found out that the swf output actually works correctly. Modifying the previous pbox2avi script, I created a tecplot specific script to generate mpeg’s or whatever else ffmpeg can spit out. The script is pretty much a more rudimentary version of pbox2avi.py.

Requirements: swftools, ffmpeg

Download tecplot2mpeg.py.

As usual the code for the script is below:

# Copyright 2010 Jonathan Wong
# python script to convert SWF files generated from tecplot to mpeg
 
#requires swfextract
 
import commands,sys,re,math
 
def parse(filename):
#check filename for .swf extension
if not filename.find('.swf'): # not a comprehensive check
return
 
fileprefix = filename[0:filename.find('.swf')]
commands.getstatusoutput("rm %s.mpeg" % (fileprefix,))
 
#open file
status, output = commands.getstatusoutput("swfextract %s" % (filename,))
 
print output
 
#find "JPEGs: ID(s)"
slide_identifier = "PNGs: ID(s)"
start=output.find(slide_identifier)
slide_extract=[]
if start:
start += len(slide_identifier)+1
slide_end = output.find("[-f]",start)
print start,slide_end
slide_extract=output[start:slide_end-2].replace(" ","")
 
# now extract all the data
print "swfextract %s -P -p %s" % (filename,slide_extract)
 
slides = slide_extract.split(",")
digits = 1 + int(math.log10(int(slides[-1])))
 
for slide in slides:
print "Processing frame: %d" % (int(int(slide)/2)+1,)
print ("swfextract %%s -p %%s & mv output.png %s%%0%dd.png" % (fileprefix,digits,)) % (filename,slide,int(int(slide)/2)+1)
status, output = commands.getstatusoutput(("swfextract %%s -p %%s && mv output.png %s%%0%dd.png" % (fileprefix,digits,)) % (filename,slide,int(int(slide)/2)+1))
 
status, output = commands.getstatusoutput("ffmpeg -f image2 -i %s%%04d.png -b 600k %s.mpeg && rm %s*.png" % (fileprefix,fileprefix,fileprefix))
return slide_extract
 
if __name__=="__main__":
slides=parse(sys.argv[1])

Since my current laptop (2004) is pretty much on its last legs and I don’t feel like wasting or looking for my XP windows keys, I figured I would just migrate my laptop. Similarly, my desktop that was built in 1999 (Pentium 4 A – no hyperthreading) with 512 Mb of ram and non-functional USB ports and dead sound card, has some useful data on it I figured I might as well start with that and see how much of a hassle it would be to backup those systems. I figured the best way would be to some how ‘dd’ a hard disk and then run it in VirtualBox. However, I remember reading something about problems with Windows “memorizing” hard disk information and in general ‘dd’ is super slow.

Luckily I ran across this Lifehacker article via a quick google search. It doesn’t contain much information, except that it mentions this new (Oct 9,2009?) SysInternals tool called ‘disk2vhd’ which supposedly makes imaging windows drives a piece of cake. Some other Ubuntu googling searches resulted in similar recommendations but also noted that there could be some IDE/SATA driver issues and that you had to fix these driver issues “apriori”. I won’t go into the details since it seems all of this can be fixed in the image afterwards.

So I went about my way, running ‘disk2vhd‘ on 2 windows drives which totaled 50+ Gb. This went smoothely on my abandoned desktop and it took about an hour or two to create the VHD. I copy over the VHD, which took a few more hours (did it overnight). However, when I tried to run the VHD in virtualbox it said I had a grub error (error 21). I figure this has to do with the fact that the MBR points to grub, which lives on the hard disk I didn’t image. Anyways a quick google resulted in grabbing SuperGrubDisk. I just ran it, and selected the “Win” option as I had no intention of creating a dual boot VHD system. This fixed the grub problem.

Windows XP started booting up and then I ran into the infamous “0x0000007B” boot BSOD issue. It has something to do with windows not liking a difference in IDE hardware. At this point another quick google seemed to indicate I would have to do the “apriori” fix step. However, I wasn’t about to go spend another 8 hours trying to backup my computer as I already spent enough time trying to make it work. Luckily I ran across this confusing forum post, which hinted that the Ultimate Boot Cd for Windows (UBCD4Win) had something called “Fix_hdc” that could possibly fix the drivers issue. Luckily I had a copy of a UBCD4Win iso lying around. So I load up UBCD4Win iso and find ‘fix_hdc’ and just selected the “Usb option”.

Magically, it seemed to do the trick, and after a reboot of the virtual machine, everything worked flawlessly.

Summary:

1. Run “disk2vhd” from SysInternals on the drives you want to image. (Optionally, run mergeIDE and so forth to possibly avoid step 3.
2. If you had some multi-boot grub setup and grub doesn’t live on the same partition (which it probably doesn’t), use a grubrecovery disk such as SuperGrubDisk.
3. If you get the “0x0000007B” BSOD error during boot, just grab UBCD4Win, boot it up, and then run Fix_hdc. It should be in Start>Programs>Registry Tools>FIX_hdc>Fix Hard disk/USB

Overall, the process was fairly painless. Mostly just clicking a few “auto” buttons. Virtualbox seems to run well, and so far the experimental 3D drivers seem to play relatively nicely. I’ve watched some movies through VM and played a few games (none of them super resource hungry). Music seems to work fine. Ironically, when I ran Windows 7 earlier this year, the audio did not play smoothely, and video clearly didn’t work. Seems that windows media player plays video much more jerky than VLC which is less noticeable. I don’t remember off-hand what my old vlc settings were though, so I can’t comment much.

I’d say the conversion was a definite success, and I will do the same with my old computers so I will have access to software that I can no longer fine that I had installed on my old machines. At any rate, I now have a “working” backup of my old system which can run in several VM packages (VirtualPC, VirtualBox, VMware). I’m fairly certain that such formats will continually be updated (virtualization seems to be the future of everything), and thus it should be possible to keep a copy of my old machines with me on my new computers in the future.

I haven’t updated for awhile because of qualifying exams and research. Now that is done and over with, I will try to make updates more frequently. Several packages I would like to write reviews for soon and make little demos for are:

1. Meld (A Diff tool. So far I’ve been quite pleased with it’s unique way of displaying diffs.)
2. Paraview 3.6.1 – Now that I have more time I hope to take another stab at the new and improved version.
3. TurboVNC – Modified version of tightVNC that works very well for media over 54Mbps wifi and 100Mbit LANs. I’ve been able to watch videos, play tux racer, and starcraft over wine without noticing any differences. In fact, I’ve run several OS’s in virtualbox remotely using TurboVNC for some computer repair training sessions over G wifi and no one noticed any latency.

I’ve updated some of the SCPD code, and I’m also working on some small short term projects.

Several times I’ve had to save authenticated wiki content to my phone to view it on long flights (across the US). I found wget + Firefox 3.0 (Export Cookies extension) to be quite sufficient. The extension page explains a bit about how to use wget.

The basic strategy is:

1. Login to website, and export cookies and session information to a text file (“cookies.txt”).
2. Run wget and try to figure out what a reasonable download rate is. Some websites obviously don’t like bots, and so you may need to look through the wget options to slowly go throught the site. You may also need to periodically update your cookies or session info via the text file so don’t be too greedy.

As usual, you may want to look into the following wget options:

-I,-r,-l,-p,-k,-np

You may need to play around with the timing as websites generally don’t like being spidered. It also may be be illegal should check the Terms of Service.

The following is a sample commandline used to save webpages and attachments to a directory for offline use. I’ve noticed that firefox on the mac has issues with lack of extensions from wiki sites.

Example:

wget --random-wait --load-cookies=cookies.txt --save-cookies=cookies.txt --keep-session-cookies -r -l 2 -p -k -I /pages/ http://***:8080/pages/viewpage.action?pageId=xxxxxxx download/attachments