You can't turn off the music in the main menu because that's the main purpose of the application. The moment you install it, each time you hover the mouse on the drums it will play the corresponding sound. You can also type on the keyboard and the sounds will play as well.
When you press a key on the keyboard, it will also record what you're playing. Use the buttons on the upper-right area to play or pause the track.
Note: It's necessary to have Adobe Flash installed for ButtonBeats Virtual Drums to work properly.