OS X has a function for playing notification sounds (Alerts) that parts of OS X and third party programs use. Once these are turned off in System Preferences it won't be necessary to turn them off in Applications to be able to listen to music without extra noise. To turn them off:

  1. Go to System Preferences (it's in the Apple Menu)
  2. Make sure you're on the main list (click Show All to get there if you aren't)
  3. Click on Sound (on the second row in Mountain Lion)
  4. Ensure that you're on the Sound Effects tab
  5. Drag the Alert Volume all the way to the left

There are two other types of sound you can turn off:

  • Play user interface sound effects (function unknown)
  • Play feedback when volume is changed (disables the clicking sound when changing the volume)