I got it! On the first try too. Lucky me. Anyway here's how you do it:
When you walk out from castle town, there are two oddly placed, and seemingly pointless towers. On the one to your right, the happens to be a hookshoty-pad-star-thingy (no idea what those things are called). Hookshot to it, and fall down to a ledge, where there's a Spinner track. Go along said track, until the middle of it, where you can jump off to the half circle hole type thing in the ground that leads to the chest.
The chest contains 100 rupees.
How do you get the chest on the other tower? That I can't figure out.