Removing duplicate music files with different names from a large directory free

During a recent Mac migration, I hosed my iTunes library and unintentionally generated hundreds of duplicate songs (the actual media files). The problem was exacerbated by the fact that a numeric was generally appended to the duplicate filenames so that I had directories full of files like the following:
/artist/album/song.mp3/artist/album/song 1.mp3/artist/album/song 2.mp3
Apple wasn't doing me any favors by injecting a space in the new names either....what a mess!

I stole a few ideas from this excellent forum thread and cobbled together a script to remove files based on exact byte counts and md5sum signatures. Here's the good stuff:

#!/bin/bashfind . ! -empty -type f -printf "%s " -exec ls -dQ {} ; | sort -n | cut -d" " -f2- | xargs md5sum | sort | uniq -w32 -d | cut -c35- | while read filenamedo echo "Removing $filenamen" rm "$filename"done
Now here's the play-by-play:
  1. Prints out the size and filename of each file found on the path and sorts using the filesize as the key
  2. Trims off the file size in preparation for next stage
  3. Creates the checksum for the files of the same size and then sorts by result
  4. Strip out any checksums that are unique, leaving only the duplicates
  5. Strips out the checksum part, just leaving the duplicate filenames
  6. Loops through the results removing the first duplicate file of each set.
You'll need to run the script multiple times if you've managed to generate more than one copy of any given file. Good luck!
Share Comments

Running Synergy client automatically under Mac OS X 10.5

There is a bug in Synergy 1.3.1 causing it to crash when started as a daemon under Mac OS X 10.5.

Here's the fix you want.

Share Comments

avahi-daemon blocks nfsd on Ubuntu

I just spent a morning attempting to fix a NFS mounting configuration that had been working flawlessly for over 6 months. My daemon log was showing this:
nfssvc: writing fds to kernel failed: errno 98 (Address already in use)

After much angst, I finally stumbled upon this posting that mentioned a conflict between avahi-daemon and NFSd. After a quick "apt-get remove avahi-daemon", all is well with the world.
Share Comments

MathML support in OpenLaszlo

I'm happy to announce that my company (Tempest Strings Enterprises) has sponsored my development and release of a MathML library for OpenLaszlo. I'll be working through the OpenLaszlo contributor process, but imagine that it could take a considerable amount of time before you see this work in their repository since I've not yet made any effort to adhere to any coding guidelines. There is very little documentation in the source, no consistent style, and no support for any runtime except SWF. Nevertheless, I've already successfully integrated the library with a larger application and it is functioning well in our beta release.


This library is based upon the Flash Actionscript library that ionel-alexandru (Sourceforge username) of learn-math.info donated to the public domain. I make heavy use of Alex's object tree and MathML rendering logic (thanks Alex!). I've built upon OpenLaszlo's Drawview class from their SVN trunk and added support for rendering stand-alone MathML from a datasource as well as formulas placed inline with other text.


Note that the full set of MathML has not yet been implemented (ex: no support for <mtable>) but here are some examples of supported formulas:

<dataset name="example1"><math><mrow><msqrt>        <mrow>                <msup>                        <mn>2</mn>                        <mrow>                                <msup>                                        <mi>x</mi>                                        <mn>2</mn>                                </msup>                                <mo>+</mo>                                <mrow>                                        <mn>4</mn>                                        <mi>x</mi>                                </mrow>                                <mo>+</mo>                                <mn>4</mn>                        </mrow>                </msup>                <mo>+</mo>                <mrow>                        <mn>4</mn>                        <mi>x</mi>                </mrow>                <mo>+</mo>                <mn>45</mn>        </mrow></msqrt><mo>+</mo><mroot linethickness="3"><mrow>        <msub>                <mi>log</mi>                <mn>2</mn>        </msub></mrow><mi>5</mi></mroot></mrow></math></dataset>
renders with the library as:
Media_httpwwwjrollerc_nhdgt

<dataset name="example2"><math>        <mrow>                <mo> ( </mo>                <mfrac linethickness="0" color="#006699">                        <mi fontstyle="bold"> a + b +c </mi>                        <mi color="#ff0000"> b </mi>                </mfrac>                <mo> ) </mo>        </mrow>        <mfrac linethickness="2" fontstyle="italic" >                <mrow><mi color="#00ff00">3456</mi><mo>+</mo><mfrac>                        <mi> a </mi>                        <mi fontsize="8"> b + c + d</mi>                </mfrac>                </mrow><mrow><mn>3</mn>        <mfrac color="#ffff00">                        <mi> c + d </mi>                        <mi color="#ff0000"> d </mi>                </mfrac>        </mrow>        </mfrac></math></dataset>
makes use of node-level styling to render as
Media_httpwwwjrollerc_nhcbg

The next one really shows off some of the fancy rendering capabilities:

<dataset name="example3"><math>        <msubsup>                <mo><![CDATA[???]]></mo><mn>2</mn><mn>56</mn>        </msubsup>        <msup>                <mi>x</mi>                <mn>2</mn>        </msup>        <mtext fontstyle='italic'>dx</mtext><mo>+</mo>        <msubsup>                <mo><![CDATA[???]]></mo>                <mi>-<![CDATA[&infinity;]]></mi>                <mi><![CDATA[&infinity;]]></mi>        </msubsup>        <msup>                <mi>e</mi>                <mn>x</mn>        </msup>        <mtext fontstyle='italic'>dx</mtext><mo>+</mo>        <munderover>                <mo><![CDATA[???]]></mo>                <mrow>                        <mi>x</mi>                        <mo>=</mo>                        <mn>0</mn>                </mrow>                <mn>56</mn>        </munderover>        <msup>                <mi>x</mi>                <mn>2</mn>        </msup>        <mo>+</mo>        <munderover>                <mo><![CDATA[???]]></mo>                <mrow>                        <mi>x</mi>                        <mo>=</mo>                        <mn>0</mn>                </mrow>                <mi><![CDATA[&infinity;]]></mi>        </munderover>        <msup>                <mi>x</mi>                <mn>2</mn>        </msup>        <mo>+</mo>        <munderover>                <mo><![CDATA[???]]></mo>        <mrow><mi>x</mi><mo>=</mo><mn>1</mn></mrow>        <mi>n</mi></munderover><mrow>        <mi>x</mi></mrow><mo>+</mo>        <munder>                <mrow>                        <mi>lim</mi>                </mrow>                <mrow>                        <mi>x</mi>                <mo><![CDATA[???]]></mo>                <mn>0</mn>        </mrow></munder><mrow>        <mi>x</mi></mrow></math></dataset>
Media_httpwwwjrollerc_hjpgi

This last example is demonstrating inline rendering with text:

<dataset name="myFormula"><![CDATA[T
his is an example of pseudo inline MathML:<math> <mrow> <mi>x</mi> <mo>+</mo> <msup> <mi>y</mi> <mfrac> <mn>2</mn> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </mfrac> </msup> </mrow> </math>How do you like them apples?]]></dataset>
Media_httpwwwjrollerc_hdktp

All of the formulas above were displayed using the same MathML view using the datasets listed. Here's the view declaration:

<mathmlview font="sans-serif" fontsize="24" fgcolor="#000000" id="math" x="10" y="10" width="800" height="500" datapath="myFormula:/"/>
The current source is released under the MIT Open Source License and available on github. Enjoy!
Share Comments

Spring JMS Remoting support for async calls and methods with void return types

I've submitted a patch to the Spring Framework to give their JMS remoting feature support for async calls and methods with void return types. The associated Jira ticket (SPR-4427) could use a few votes if you are so inclined.
Share Comments

Hilight current post on threaded view in Fireboard 1.0.4

In the 1.0.4 release, there is no indicator of which post you are currently viewing within a threaded tree of posts. Here's my hack to implement such an indicator.

Under your chosen template directory (ex: components/com_fireboard/templates/default), edit the thread.php file. Look for the TR with class="fb_threadview_row" and modify it like so:

<tr class="fb_threadview_row" <? if($leaf->id == (int)$id){ echo "style='background-color:yellow'"; } ?> >
You can use any background-color value you desire. I used "yellow" in the above example.

Share Comments

Java webbrowser plugin for AMD64 Ubuntu

There is now (finally) a fix for Java applet support on AMD64 Ubuntu. Try the following:
sudo apt-get build-dep icedtea-java7-pluginsudo apt-get -b source icedtea-java7-plugin icedtea-java7-bin icedtea-java7-jre icedtea-java7-jdksudo dpkg -i icedtea-java7-*deb
Thanks to Fabien Lusseau for the tip in his comment here
Share Comments

Mounting an Ubuntu NFS share on RedHat Enterprise with firewalls on both

It can be tough making NFS play nice amongst different distros and firewalls. I'm sharing my Eclipse workspace on my Ubuntu (Debian) desktop with my CentOS (Redhat Enterprise) server. (That's not a mistake, my server is actually my NFS client in this scenario. You could easily swap these roles if you wish)
DEBIAN:I used port numbers and config instructions at http://wiki.debian.org/?SecuringNFSFollow those instructions exactly and you'll have a NFS server configuration that uses static ports (instead of random assignements)REDHAT:1. Create the file "/etc/sysconfig/nfs" and add the following contents:STATD_PORT=32765STATD_OUTGOING_PORT=32766LOCKD_TCPPORT=32768LOCKD_UDPPORT=32768MOUNTD_PORT=327672. (Probably not necessary for most, but here for completeness) Append the following to the file "/etc/services":rquotad 32769/tcp # rpc.rquotad tcp portrquotad 32769/udp # rpc.rquotad udp port3. Restart the nfslock service:/etc/init.d/nfslock restart4. Run /usr/sbin/rpcinfo -p and make sure all the ports above have changed and match their ubuntu numbers. (/usr/bin/rpcinfo on ubuntu)5. I run firestarter on Ubuntu which is exporting my NFS share. I added the following policy rules for inbound traffic:Allow 32767 (from my client machine's IP)Allow 2049 (from my client machine's IP)Allow 111 (from my client machine's IP)
Assuming you allow all outbound traffic on your client (I recommend this excellent iptables config file rc.firewall_023.txt), you should now be able to mount the NFS share on the Redhat box.
Share Comments

Migrated from JRoller

Entries prior to October 2007 are only available on my original Agile Answers blog hosted at JRoller here.

Share Comments