If the application was installed on a PC, navigate to Start > Control Panel > Uninstall a program, select the software from the list and press Uninstall. Alternatively, you can use Revo Uninstaller. Even if it can't uninstall the application normally, it will force it to uninstall by wiping out all traces of its presence from the system.
If you installed the program on your mobile device, go to Settings > Application Manager > Installed Apps, select the OVI Store app and remove it. Also, if the OVI Store installer is using SIS, SISX, WGZ, or JAR as file format, you can try uninstalling it by using the App Uninstall tool.