As with much to do with the iPhone, there's a bit of a trick to getting this working smoothly so you can have control over your iPhone music, while keeping it as darn easy to work with as possible. The answer? Create a custom playlist for your iPhone music library.
Let me show you how to do it - it's pretty darn easy.
Pop into iTunes and choose File --> New Playlist. Now rename it to something logical: I use "iPhone Sync":

Now you can just drag and drop music from your iTunes library (in iTunes, needless to say) into this new playlist. When you're ready, you'll see a bunch of music happily ensconced in the playlist (remember, it's all just "pointers": the actual music files are still safely in the main music library, so if you delete something from a playlist , you haven't deleted the original music file itself Notice here on the very bottom of the iTunes window it shows you how many songs are in the playlist, along with the size and total playing time. For my current playlist, it's 140 songs, 10.1 hours, for a total file size of 851MB, no problem for my 4GB Apple iPhone.
At this point, you need to plug in your iPhone if you haven't already done so. It'll pop up the usual summary screen: