To check if the spyware was still installed, go to Settings > Security > Device Administrators. You should find it under a dubious name or something very familiar like Android Manager. Uncheck it and then install an anti-malware from the Play Store. Scan your phone and it should be deleted.
Unfortunately, it's impossible to say exactly when the application was installed.