Yes, the VSO Video converter can be used with NVENC because it already uses the available API provided by nVidia. However, many of these video converters are created to support NVIDIA CUDA processors to speed up the converting process and according to the official information, you can use the software with an nVidia graphics card as stated in the features list:
- Uses Hardware encoding: NVIDIA CUDA (including new NVENC API), Intel Quick Sync
For more information I recommend accessing the official page of the converter.