Choke knob removal

Ok, this is gonna rank right up there with the dumb question award, but don't want to do more damage than necessary. I have a B&S model 20M314 engine on our Simplicity snowblower. I need to remove the shroud to replace a faulty ignition switch. In order to do that I have to remove the choke knob. Looks like it should just pull off it's shaft, but isn't coming up easily. I'm afraid of putting too much upward force on it in fear of breaking the shaft off at the carb. Is there a special tool to pop these off or should I just go for it and pray to the Gods as I pull? Perhaps getting 2 prybars under it?