It's possible but not all that likely. Latex is in tons of things besides condoms (Band-Aids, dishwashing gloves, balloons, sanitary pads) so if you were allergic to latex, you'd probably know already. What is more likely is you're allergic to another part of the condom--the spermicide or the lubricant. Try a different brand of condom or use one without a lubricant and supply your own, preferably a water-based one because they're way less irritating. That soreness and itchiness could also be a sign of an early-stage yeast infection or bacterial vaginosis (a bacterial infection of the vagina). These are usually also accompanied by discharge, so if your symptoms progress, see your gyno. She'll prescribe either an antifungal medication or antibiotic to help clear it up. Oh, and is this a new relationship? If so, and you ever have sex without a condom, it's also possible you're allergic to semen. While it's extremely rare, it happens. And it's a tough problem to solve. Some doctors try slowly increasing your exposure to the semen in hopes you'll no longer have a reaction. But this is tricky and needs to be done and supervised by a doctor.
From my book: