The traditional use of selective yellow lighting was to reduce glare. Studies on the topic are mixed: some report yellow as better, some report white as better, and some have shown no real statistical difference, with various advantages and disadvantages more or less working against each other. Driver perception of forward vision is influenced by a combination of what we think our eyes can see (which is often an overestimation) and confirmation bias. Selective yellow lighting was traditionally very popular in Europe and in rally racing in particular. Though you don't often see true selective yellow used in such events anymore, tradition still plays a strong role in the thought process.
Blog Post | Fog Doesn't Play Favorites with Headlight Color | Car Talk
Selective yellow - Wikipedia
And, the authoritative source, Dan Stern:
Daniel Stern Lighting Consultancy and Supply
North American and European ECE regulations both allow for white or selective yellow for fog lamp situations (selective yellow is no longer allowed for headlamps in Europe, though it used to be popular there). If there was a clear safety advantage of yellow lighting, I think we'd see more of it at the OEM level. I'm sure a point could be made that many drivers wouldn't like the yellow lights, subjectively, and would want their fog lamps to match the color of their headlamps (thus, the reason for white from the factory). There may be just as many who prefer selective yellow and criticize auto makers for putting in white lighting.
One point remains valid, though. Fog lamps are NOT designed to allow you to continue driving 55 mph in pea soup. Fog lamps are intended to help you see the reflective lane lines on the side of the road so you can poke along when visibility is near zero. This is why fog lamps are (correctly) aimed low. They're not designed to supplement your headlamps in normal driving.
I think that's the best advice. Do what you think will work best for you, knowing that your results will be grounded more in your personal preference and
your set of eyes rather than a broadly held position that one color is universally "better" than the other. The post above about using fog lamps as a warming supplement to the white headlamps is an interesting idea.
@Fergie, I presume you've kept the fog lamps so that their aim is, at most, just under horizontal (measured from the vertical center of the lamp)?