I continue to battle with a similar problem, with all my controllers. Once a month (or whenever I think of it. lol), I clean them with some baby wipes. I know it sounds funny, but the baby wipes are pretty effective against controller grime.
Before babies came into my life, just a damp cloth worked well.