If you use Windows XP or below go to Start > Control Panel > Add or Remove Programs, click on the Internet Download Manager item from the Currently installed programs window and then press Remove.
If you have Windows Vista or above navigate to Start > Control Panel > Programs > Programs and Features, select the Internet Download Manager application and click on Uninstall.
Alternatively, you can use a utility from Software Informer database. Many of them are free of charge and will force the program to uninstall by wiping out all traces of its presence from the system.