Viber relies on the contacts stored on your phone, the real contacts you usually call using the mobile network. When you remove someone from the Contacts list and then refresh the same section, you see them removed, but in Viber, you will see the change right away.
This is the way most of these apps work (Viber, Line, Tango, WeChat, etc) when it comes to contacts management.
