The caffeine will pass on, I know my daughter when she was small wouldn't allow me to drink even a sip of cola she would scream and refuse to nurse. That was one of the only things she ever reacted to in my breast milk.
Accordidng to Kellymom the potential problem with caffeine is that it can be passed on to baby, not that it affects the general nutritional quality of the breastmilk. How much the caffeine affects your baby depends on the amount of caffeince consumed, the age of the baby and the babies' sensitivety to caffeine. She says that recomendations vary, but that generally lactating women are told to consume no more that 500 mg (equal to 12.5 cans of Pepsi) of a caffeine a day.
If you have a particularily caffeine sensitive baby you may need to consume much less than this.