You've got two options I figure, either get the code to one (Azureus is FOSS, no?) and try to understand it, or work on it from the basics:
Break down the interface you see into sections, see what's repeated, find your basic GUI elements. Paper or screenshots and an image editing program should do.
Then try and build each section out of the standard Java GUI widgets, or (re)code your own ones for more control/effects.
Then put them all back together and Ta-Da!
Don't forget the old Model-View-Controller pattern, just have the GUI display the data and trigger actions like Pause/Stop but not directly be responsible for it. Goes well with making a command-line only prototype first before you add the nice interface.