Can you have a similar intolerance for coffee as you do for milk?
What kind of milk intolerance do you have? I mean, what exactly in the milk are you intolerant to? Many people have a lactose intolerance, but there is no lactose in coffee. Yes, people can be intolerant to coffee, but it's usually not the same reason people are intolerant to milk.
Food intollerances usualy come from an inability of your body to process a thing. That's different to a food allergy, where you have a reaction to a thing. So you are more likely to have a coffee (or caffein) allergy than intollerance!
you can have an intolerance to anything.