I think that part is trying to use the Nvidia hardware encoding, which requires a modern Nvidia card with Windows drivers. That's not going to work on a Mac.
Quite true - for the Mac, there is limited access to Intel QSV hardware through the Video Toolbox interface, but not NVENC.
However, using the hardware encoders when one of the criteria is "not lose too much quality" would be a non-starter. x264 or x265 would be the way to go.