Huge fan. My personal favorites:
"Go To Extremes" - One of the best power pop songs ever written.
"Prelude/Angry Young Man" - Joe Jackson has been trying to write this song for almost thirty years now. It would also be the perfect theme music for the Age of the Fall, provided that Gabe wants something a tad more melodic than Lacey screaming bloody murder.
"Captain Jack" - Wonderfully depressing.
"Innocent Man" - I love how that song builds.
"Sometimes a Fantasy" - His attempt at writing a punk song. (No really... the "Glass Houses" was supposed to be his "punk" record. More on that in a second.) He doesn't quite it off (it's more "Billy Idol" than anything else), but it's an awesome song nonetheless.
"Don't Ask Me Why" - Billy Joel channels Harry Nilsson with fantastic results. (If you don't know who that is, shame on you.)
For a nice counterpoint on his "punk phase," I highly suggest hunting down the Weird Al rarity "It's Still Billy Joel to Me." It's insanely hard to find (live bootleg only), but it's well worth the listen. The lyrics are posted below.
What's the matter with the songs he's singin'
Can't you tell that they're pretty lame
After listenin' to a couple albums
Well, they all start to sound the same
So he tried to change his musical style
He tossed all his ballads in the circular file
Then he found the punk sound
Breakin' ground all around
It's still Billy Joel to me
What's the matter with the tune he's writin'
Well, you know it's gonna be a smash
It's so nice when you're a big name artist
Doesn't matter if it sounds like trash
Now everybody thinks the new wave is super
Just ask Linda Ronstadt or even Alice Cooper
It's a big hit, isn't it
Even if it's a piece of junk
It's still Billy Joel to me
Woah, it doesn't matter what the critics say about him
'Cause he doesn't worry how they feel
When you're record's sellin' millions and it's goin' triple platinum
You don't worry 'bout your next meal
'Cause money is no big deal
Maybe he should dye his hair bright pink
And stick a safety pin through his cheek
Then he'd really fit the new wave image
But he couldn't sit down for a week
Don't you know about the record business, honey
You gotta be trendy if you wanna make some money
Now everybody's sayin' that he sure sounds funny
But it's still Billy Joel to me
I can hardly wait 'til his next album
Well, I'll bet it's gonna be the rage
Buy a ticket to his next big concert
Well, I wonder what he'll do on stage
It might be disco and it might be the blues
Or maybe even somethin' like the B-52's
Just a handclap, finger snap
Even if it's mindless pap
It's still Billy Joel to me
Everybody's sayin' that he sure sounds funny
But it's still Billy Joel to me