The first thing to confirm is that after unticking the Sync Music box is that also clicked the Apply (or Sync) button in the lower right corner of the iTunes window in order for the change to take effect. Did you do that?
Next, you mention unticking songs, which suggests to me that you have unticked individual songs. This is very different to the Sync Music option. There are two steps to unticking individual songs and causing them to not be added to your device:
- connect the device to iTunes and select the device icon that appears at the top of the iTunes window:
- that will display the Summary pane (which may look slightly different to the screenshot below):
- ensure that Manually Manage is not ticked
- ensure that Sync only ticked songs and videos is ticked
- Sync the device with iTunes
That will mean that if and when you untick a specific song in your iTunes Library, the next sync will remove it from your iPhone. To tick and untick a song, look in the Songs view of your library (that is, not on the device itself, but in your main library). Next to each song is the tickbox. (The song remains in your Library.)
Another option is to Sync selected playlists, artists, albums and genres. To do this, connect the phone to your library and look on the Device/Settings/Music pane. Beneath the Sync Music button is the option to sync either Entire music library, or Selected playlists, artists, albums and genres. Choose the selected playlists option and the various sections (playlists, artists, albums and genres) will appear underneath that. In the screenshot below notice the Playlists and the Artists headings at the bottom of the picture. It's all below those headings.
If you do use sync selected playlists etc. you then choose which of those items you want to add to the phone and then tap the Sync (or Apply) button. Having set up this option, whenever you change the content of a selected playlist, album etc., the next Sync will cause the phone to be updated with these changes.
