Final Fantasy XIII is a good JRPG that wouldn't have garnered nearly as much hate if it hadn't borne the
Final Fantasy name. I don't recommend it as an entry point to the series, mainly because the story has relatively little in the way of traditional Final Fantasy concepts, but also because it has
two direct sequels.
Final Fantasy XII is a really interesting game, but it doesn't get truly good unless you play
Final Fantasy XII International: Zodiac Job System, which revamps several of the core gameplay systems, adds extra content, and most importantly, adds a job system in the form of additional license boards. This isn't available officially in English, of course, but if you can emulate PS2 games (shouldn't be too hard if you have an Intel processor made in the past five years or so) or have a modded or softmodded PS2, you can use a translation patch. I don't recommend this one as a starting point either; the gameplay is incredibly different from anything else in the series, and isn't a good indicator of whether or not you'd like
Final Fantasy games. Also, as The Chef said, it's set in Ivalice, the world of
Final Fantasy Tactics, and specifically set during the earlier part of the timeline with
Final Fantasy Tactics Advance and
Final Fantasy Tactics A2 (which features FF12 player characters Vaan and Penelo), so if you wanted the context of the world, you might want to play FFTA first, though it's certainly not necessary.
Final Fantasy X is a very easy starting point for the series and for JRPGs in general. The sphere grid's complexity builds slowly and eases you into full character customization, and the removal of the Active Time Battle system used throughout the SNES and PSX games means you have all the time in the world to figure out what you're going to do with its priority-based combat. The core plot is a love story between a traveling priestess and a time-traveling sportsball player, both of which are dealing with living up to the incredible legacies of their fathers, and the infamous laughing scene makes a lot more sense in context. It's also readily available on multiple platforms (all PlayStation, though), and not particularly expensive; you could pick up the PS3 version of
Final Fantasy X|X-2 HD Remaster, which includes the game and its incredibly divisive sequel in their optimal International Version forms, on PSN this week for $15, and from what I've seen, used copies on PS2 tend to stay in the $10 range.
Final Fantasy IX is...a game I need to play more.
Final Fantasy VIII has weird combat and stats systems, a main character you seemingly aren't supposed to like, and a card game you can easily spend more time with than the actual game. Not recommended as a starting point.
Final Fantasy VII is massively overrated. It's also incredibly difficult to approach today, because early PSX graphics look like butt. If you can get past the visuals, it'll make a decent starting point, but it's certainly not the best game in the series by any means. (Also, while
Final Fantasy VII is certainly the most milked
Final Fantasy entry, it's mostly through cameo appearances in other games, like
Ehrgeiz: God Bless the Ring.)
Final Fantasy VI is a game I don't care to play any time soon, and as such, I'm not qualified to speak on its qualities as a starting point for the series. I'd disregard the recommendation to play the SNES version, though! It's
full of bugs that were fixed in later releases. The GBA version is probably your best bet.
Final Fantasy V is wonderful. In-depth job system, great cast, fun plot, and the introduction of recurring boss Gilgamesh - what's not to like?
Final Fantasy IV is a clear formative point for the series, and makes a great starting point as a result. I've cooled on it a bit, though, and don't recommend it as strongly as I might have if you asked me about it in 2012, when I first played it (via the stellar PSP remake, which I'd recommend over the DS remake).
Final Fantasy III is a bad game. Don't bother. (I'll get some flak for saying that, but really, it's like playing the original
Street Fighter instead of
Super Street Fighter II Turbo. If you want job-based
Final Fantasy, play
Final Fantasy V instead.)
Final Fantasy II is BP's favorite. He can speak more for why it's a good game than I can.
By modern standards, the original
Final Fantasy is weird and difficult, seemingly for all the wrong reasons, but its remakes commit a far greater sin by removing any personality from the gameplay, leaving an incredibly generic JRPG. If you're going to play this one (which is an argument I'm not going to get into), do it right by playing the NES version.
Overall I'd recommend
Final Fantasy X or
Final Fantasy IV as a straightforward starting point, or
Final Fantasy V if you want something that shows a bit more of the experimental side of the series.
Final Fantasy spinoffs are a much stranger beast, and could fill their own thread.