You can completely uninstall the software(the 2007 and the 2010 version) and install it again(whatever suits you best 2007/2010). To do that, I suggest you use Revo Uninstaller. Even if it can't uninstall the application normally, it will force it to uninstall by wiping out all traces of its presence from the system.
After that, try running AdwCleaner to make sure that there are no leftover rogue registry keys or toolbars in your system.
However, you should be careful because Microsoft Outlook comes in a bundle, Microsoft Office. So if you want to uninstall it, you got to uninstall the bundle.