Do I need to have a computer plugged into the modem and router to use WiFi?
The answer is no because you can connect the Internet cable from the modem to the router and assign DHCP on the router, then you can connect the computer to the router on a normal port and use its WiFi.