TV Shows Organizer Updated

i've updated the tv show management script.  it now checks against a list of prioritized regular expressions one after another until it finds one that works.  also, it grabs the name of the episode from thetvdb.com.  additionally, post-information-gathering hooks have been added(this allows me to grab the name of the episode as well as sanitize the filename for the filesystem).  finally, hooks have been added to fix show names automatically(eg. Brothers and Sisters becomes Brothers & Sisters so Boxee can find the right series from thetvdb).

i also rewrote my regular expression tool in c++.  a simple, one-test benchmark yielded:
-for the java version(the command was "time java -jar RegExpEval "(Hello)(World)" "Hello" "test""):
--real    0m0.446s
--user    0m0.107s
--sys    0m0.057s

-for the c++ version(the command was "time GetRegExp "(Hello)(World)" "Hello" "test"")
--real    0m0.065s
--user    0m0.002s
--sys    0m0.007s

it's not exactly scientific but the c++ is a lot faster than the java version

i also wrote a script to remove old episodes.  by default, the script keeps the latest 5 episodes in any series.  it removes old seasons as well.  it's possible to exclude a show, season, or specific episode from being deleted.

might i also mention this is all done in bash shell script?
due to the way i've written the hooks, there are major security problems(ie execute any code a malicious hacker wants) because the hooks are included into the shell script and then a function is executed.  same goes for regular expressions.  they are included and then tested followed by a processing function to allow the regexp to fix various things.

anyways, i'll post the code soon because i'm too lazy right now.

CUSEC '09

well, cusec finished saturday. never got around to my post-a-day that i wanted for the blog. hence, i'm condensing two posts into one.
forgot something on wednesday: Dan Ingalls talked about himself for the first half instead of talking about what he was going to talk about. in the end, he didn't have enough time to show off the Lively kernel. (sorry Dan! again. i don't know you so if i'm completely off-base, correct me in the (moderated) comments)
so friday, Joey deVilla from microsoft gave his keynote which turned out to be very entertaining and informative. Giles Bowkett also presented a ruby app called archaeopteryx. it basically generates midi events from a probability matrix or something like that. you can grab the code from http://github.com/gilesbowkett/archaeopteryx/tree/master if you want. the guy's funny as hell, no one's gonna fall asleep at his presentation.

on saturday, the only presentation i went to was the final keynote delivered by Richard Stallman. what to say, what to say? i've never met an activist(afaik) so this was an interesting experience for me. unfortunately, i was hung-over and tired so i half-nodded off a few times(sorry Richard!) simply to try to ignore my hangover. didn't work very well. anyways, rms talked about copyrights in the 21st century and how for many different types of media, a shorter copyright period is necessary and even crucial. after the keynote, he auctioned off a signed hardcover copy of his book and a gnu. the book sold for CA$85. when rms said the starting bid was 40, someone yelled "we're students!"! the gnu sold for $105 to Joey deVilla(remember, he works for microsoft!) Joey tells the story(EDIT: story is a link, my theme doesn't show it very well) better than me. EDIT: forgot to mention, Joey mentioned on his twitter feed that his boss let him expense the gnu on his ms corp card.

montreal is an amazing city! the people are really friendly(the ones i met) but the train station is a little hard to find if you haven't been there before(i looked for 1/2 hour because there weren't any signs).

overall, CUSEC '09 was an amazing experience and i can't wait for next year!

CUSEC '09

So I just finished my first day at cusec '09 (in Montreal) and so far, it's really awesome. I heard presentations from Leah Culver (if you ever get to see a keynote given by her, go for it. It's totally worth it.), Jói Sigurðsson (gave me a few really good ideas for my tagging project), and Dan Ingalls (unfortunately, I was getting really tired by that point of the day and half-nodded off a few times, sorry Dan!). Leah gave an awesome inspirational speech about using the tools you have available right *now* to do what you want to do(it was called Swiss Army Knives and Duct Tape). Jói gave an interesting presentation on client-side mass-market applications and the challenges of supporting as many configurations as possible with the most efficiency as possible. Dan talked about his life and involvement with physics and technology for about 30 minutes and then he moved onto the Lively Kernel which is very cool and there are imcredible possibilities in its future.
Later tonight is pub night at BENELUX. It's gonna be awesome!

Convoluted Tethering

so lately my mbp has been having problems with maintaining a wireless connection in any form(from non-encrypted to any form of encryption).  unfortunate because when i'm in class i can't really do much(although i probably shouldn't surf the net in class ;) ).  anyways, i arrived at the hotel for cusec '09 to find out there are no wired links for rooms, only wifi.  i thought to myself:  using my ipod touch for 3 days to surf the net.  painful.
in comes a program called iPhoneTunnel (not iPhone Tunnel Suite which is for windows, this one's for macs).  ipt opens an network connection through the usb cable connecting my ipod touch to the mbp.  it also includes preset commands for sftp, ssh, and tethering(they all employ ssh in one form or another, so you must have a jailbroken itouch for this to work).  fire up the tethering preset, set your socks proxy to 127.0.0.1:9999 and you're off to the races.

note for firefox: you must set network.proxy.socks_remote_dns to true for it to work.  same goes for any xulrunner-based app(thunderbird, songbird maybe)

fyi, i'm writing this post while tethered!

Moments

you know those moments in life where you go "it works!" and then you feel really proud of yourself (or hopefully whoever made it work), i just had one of those moments. several actually.

a bit of explanation: i'm currently developing a piece of software to allow me to tag files. yeah, i know a bunch of software already exists for that, but none of it works for me and my workflow. i've got to have things work the way i need them to, not the way people think i need them to. i also make things easy to do (for me) so i've got a highly customized mbp but most of the stuff is in the background so someone else can sit down and use it like any other mac without knowing the difference.

i was using a piece of software called Punakea which fit into my workflow but it wasn't as extensible or as flexible as i needed. there were other faults as well, but the important ones were that it couldn't build multi-level tag trees(ie tag something "hello/world" and end up with the file in "root/hello/world") and it cross-referenced related tags in the tag tree(a good idea in principle, but messy on the filesystem in practice).
the guys at Punakea have a wonderful product but it just doesn't fit my needs. i set out to write my own.
(note: any command to enter into the command-line will start with a # and then space and then end with a ;)
it will follow some of the same behaviours of Punakea but improve or modify them in some ways. there will also be several different ways to interact: command-line (ie # tag somefile sometag;), drag and drop onto a drop box, choose a file in finder and right-click it, etc.
i hacked Punakea's system a bit and did add command-line, but it wasn't enough for me.

i'm building the program in 2 steps: dependent functions and the core. the dependent functions are things like getting a directory listing, linking files, getting a file's tags, setting a file's tags, etc. i've put these functions in platform.c (platform because they'll probably change depending on whether you run mac, linux, win, etc) and osxutils.c (i've copied some code for supporting getting/setting tags, file aliasing). to port the program to another platform, all you need to do is fill in the functions in platform.c with platform equivalents. right now it's setup for osx 10.5 (maybe earlier, i don't have a system to test with).
the core has yet to be written because i was working very hard on the dependent functions for a while.

the coolest thing so far(and the "moment") is my make alias function. an alias in osx is simply an empty file with a resource fork describing the target(see http://en.wikipedia.org/wiki/Alias_(Mac OS) for details). note that the data fork(the one you see in finder and the terminal) is simply an *empty file*. a plain old empty file! now take that file, add some code to it, and now you have an alias that you can execute from the command-line to resolve the original location!
afaict, finder doesn't touch the data part, nor does the rest of the system, so the data fork will always be there (unless you move it to a non-hfs filesystem in which case the alias will no longer be an alias although i've seen that finder copies the resource fork into ._filename on non-hfs systems so it just might work).
you can do cool things like # ./aliasname; or # cd "`bash -c aliasname`"; or even put the alias in your $PATH and do # aliasname; if the original is executable!
the link function is smart enough to generate code for the type of file/folder it's linking. if it has it's execute bit set, it will output code that will execute the original (with any arguments) and also allow you to do # aliasname --resolve-alias; to get the real file. if the original file isn't executable, running # bash -c aliasname; will output the real file's location. if the file is a directory, you run # some_command_to_work_with_directory "`./aliasname`"; you can also add code to your .profile that will do the cd work for you transparently so you can just do # some_commane aliasname;.
right now, i'm using hfsdata from osxutils to grab the real file's location. bash code is inserted into the alias to run it. my plan is to integrate the resolving code from hfsdata into each alias created. it would be nice if i could hardlink data but unfortunately i can't so each alias will be slightly larger than they normally would be.
it's all very cool because now i can do stuff like tagging a file "Bin" or something like that and within seconds, it will show up in my $PATH under $HOME/bin as an alias to the original file and that alias will now execute that original file.
usage case: i build product-x.y.z in my Development directory but i don't want to install it in my system directory. in fact, i'm so lazy that i don't want to do anything with the built files. i'm not lazy enough to avoid tagging the executables though! i would tag one of the executables "Bin" and then do # executable; to run it because it would then be in my path!
it's freaking awesome!

sample alias code:
#!/bin/bash
if [ $1 ]; then
if [ "$1" = "--resolve-alias" ]; then
echo "`hfsdata -e "$0"`";
else
`hfsdata -e "$0"` $@
fi;
else
`hfsdata -e "$0"` $@;
fi
end_code
code to add to .profile for cd-ing into an alias(you can even do # cd somealias/some_dir_under_original_path; ! this will work for any alias that points to a directory, you only need hfsutil from osxutils. oh, and it's kinda slow only when dealing with aliases). For all intents and purposes, assume you are using a symbolic link when using this version of cd. What this will not do is resolve the alias and echo it back, that's what hfsutil is for.

If you don't want this to replace cd, change the line that reads "cd() {" to something else like "cda() {" or "idontwantcd() {"

begin_code:

string_substring() {
if [ $3 ]; then
local to=$3;
if [ $4 ]; then
to=$(($3 - $2));
fi;
echo "${1:$2:$to}";
else
echo "${1:$2}";
fi;
}
string_lastindexof() {
local ns="`string_length "$1"`";
local nf="`string_length "$2"`";
local -i tmp=$(($ns+1-$nf));
local -i k=$tmp;
local subs="";
while [ $k -ge 1 ]; do
Delay
subs="`string_substring "$1" "$k" "$nf"`";
if [ "$subs" == "$2" ]; then
echo "$k";
return 0;
fi;
k=$(($k-1));
done;
}
string_length() {
local tmp="$1";
echo "${#tmp}";
}
aliaspath() {
if [[ ("`hfsdata -e "$1" 2>/dev/null`" != "Argument is not an alias" && "`hfsdata -e "$1" 2>/dev/null`" != "") ]]; then
echo "$1";
else
local j="`string_lastindexof "$1" "/"`";
local f="`string_substring "$1" "0" "$j"`";
aliaspath "$f";
fi
}
trailingpath() {
local f="`aliaspath "$1"`";
local l="`string_length "$f"`";
l=$(($l+1));
local g="`string_substring "$1" "l"`";
echo "$g";
}
cdalias() {
local p="`aliaspath "$1"`";
local d="`dirname "$p"`";
local b="`basename "$p"`";
local t="`trailingpath "$1"`";
if [ "`cat "$d/$b"`" != "" ]; then
builtin cd "`bash "$d/$b"`";
else
builtin cd "`hfsdata -e "$d/$b"`";
fi
if [ "$t" != "" ]; then
builtin cd "$t";
fi
}

cd() {
if [ ${#1} == 0 ]; then
builtin cd;
elif [[ -d "${1}" || -L "${1}" ]]; then
builtin cd "${1}";
else
cdalias "$1";
fi;
}

end_code

if you want an explanation of the above code, comment on this post about it. don't comment about me re-implementing some of the bash functions (string_something). it's just to make it easier for me to use.

FileTaggingwith XTagRev(Extreme Tag Revolution)

Just recently I started using Punakea for tagging my files.  Before now, I hadn't tagged anything because I hadn't found any useful program to present me with tagged files.  Then Punakea came along.  The program is awesome, but it lacks in a few features that I absolutely require.  I must be able to create tags like "Hello/World" and then Punakea, when creating the tags folder, *should* create Tags/Hello/World.  Instead of doing that, it crashes(understandable).  Also, Punakea doesn't use aliases, but links, which means that if a file is moved, the entire tag folder needs to be rebuilt.  Finally, Punakea doesn't update the tags folder every x xseconds, it only updates when it has tagged a new file(or untagged one).
These are all showstoppers for me, so I heareby present XTagRev, short for Extreme Tag Revolution.

XTagRev is a series of bash scripts that provide the same core functionality as Punakea along with the showstoppers that I've listed above.  Out of the box, it runs on a mac but I've set it up so that you only need to change one file if you want it to work on other platforms(linux,windows).
There's no UI yet, just a few bash functions for getting/setting tags, rebuilding the tag folder, and a few others.
I was going to write it in C(or C++), but I wanted a quick prototype so I could see how it would work.  My plan now is to port it to C/C++ because bash is ssssllllllllloooooowwwwwwwww.  Also, it'll chew up the processor as it updates it's state.

You can get the code from my assembla account at http://code.assembla.com/rosskeschoolwork/git/nodes/master
Please ignore the foul language in some of the revisions' logs.
Save it in /usr/share/XTagRev, move .XTagRev to your home folder.  If you aren't running on a mac, it is imperative that you go through /usr/share/XTagRev/Platform.sh and change the code for your platform.  This means that the code for, say, FsGetTag needs to be replaced with code that will get a file's comment from your indexing system.  Most of it will stay the same for linux and other unix-types.  If you CAN'T set tags from the command-line with your indexing system, you are out of luck.  This WILL NOT work for you.

Personally, I linked /usr/share/XTagRev to /usr/bin/tagger and added it to launchd with the AbandonProcessGroup property(because Start.sh forks).  It now starts when I login.  Abort.sh must be called everytime you want to relaunch.  Abort.sh will take a while(even though it exits immediately)   Also, it must be run on logout(but you don't have to wait).

Have fun, and a C/C++ version is on the way!

EDIT: I haven't lost any data yet, but if you do, it's not my fault and remember, you were the one to try this.

new years eve

i spent the bulk of my new years eve quite unhappy. i came home for christmas and quickly setup the 2nd router in my house(it was sitting there, unused, and i had to give it a purpose) as a wds node. it was not fun trying to get it working and everytime i unplug the power, it *seems* i have to change one wifi-related setting to get them to connect.
i found out after an hour of tinkering that my brother was downloading several torrents and therefore was a) using all the available bandwidth, and b) consuming all of the router's resources(ram, cpu power, etc). several times during that hour, the two routers successfully initiated a connection, but never got past the initial connection because the main router couldn't do anything beyond the initial connection due to having to manage several hundred(at least) bittorrent peer connections. it royally screwed my enjoyment of the holiday having to troubleshoot a connection that wasn't working when it should have.
i'm kinda pissed off if you haven't noticed, but i did end up getting the connection working at around 1:30am today.
wds is an awesome technology, it even passes my laptop's dhcp requests off to the main router. i've connected my laptop with cat-5 to the seconday router, hence the main router sees it as being on the lan, not wifi, which is pretty cool(once it works).