I've not set up a variable pitch quad. I recommend not using the setup wizard, and setting up each bit of functionality at a time.
First, on the hardware pane, set the radio type and reboot the FC. You can use the "Radio setup wizard" to configure the channels and ranges from your radio, then.
Next, on the output page, I would configure the output channels-- ranges and update rates. And confirm with "test outputs" that you can appropriately control them by moving the neutral positions on the slider. Obviously this should be done without propellers mounted.
Finally, you can configure the output mixer. This is the tricky part, and works by going to the vehicle setup pane. First, I would pick "quadcopter" for the mixer type, and then configure the 4 pitch servos to be the 4 "motor channels", and then save to get a starting set of values. Then, pick "custom" vehicle type. You want anything that communicates with ESC to be a "motor" type, and everything else to be a "servo" type.
I'm glad to offer help on the specifics --- just let us know where you get stuck.