Baking soda is very basic (9 on the pH scale) and vinegar is very acidic (3 on the pH scale), both of which would be detrimental to hair color. That's one reason that the BS/vinegar regime works so well together: by the powers combined, they neutralize each other. It's my favorite combination for cleaning an oven (well, either that or leaving a bowl of ammonia in the oven for 24 hours so the vapors break down all the nastyness).
Have you thought of trying just baking soda, or just vinegar, or just conditioner? After the One thing, I just used conditioner to wash as someone mentioned, and that works pretty well.