If you have deleted someone, he/she will be removed from your phone. The messages might get deleted, but that will happen only on your part. This means that yes, even if you delete the other users, he/she should be able to access previous conversations.