I believe Dupli Find is the right application for this type of job. I say this because this application has "search-in-file" support which is different than all duplicate finders available on the Internet.
The process of removing duplicates is rather simple. Open the file in question using the Open button and the software will automatically find duplicate lines and content available in the file. Then with a single click you can delete them in a matter of seconds.
