I've installed the downloader and if you untick the browser where you want to keep Free Download Manager off, the download button won't appear. Here is a screenshot of the options that you have to disable.
I've started using this application on a daily basis since it helps me download things faster, including torrents and videos from video sharing websites.
