Waterproof bags are made from fabric that's coated or laminated with a plastic layer, usually TPU or PVC, then welded or glued at the seams. Fresh from the factory, the coating, the adhesives and any printing can all give off odor. The U.S. EPA notes that many products release organic compounds while you are using them and, to some degree, when they are stored. Its first recommendation for reducing exposure is more ventilation.
A waterproof bag makes that smell more noticeable for the same reason it works: air doesn't pass through the fabric or out the seams. Close it, seal it in a shipping box for a few weeks, and the first time you open it is usually the strongest it will ever smell.
Does the material matter? Palm Equipment, a paddling gear maker, says it uses TPU instead of PVC because PVC production relies on phthalates to make it flexible, while TPU doesn't need plasticizers. That's one manufacturer's position, and it says nothing about smell, so don't assume a TPU bag will be odor-free. For the bigger material differences, see TPU vs. PVC waterproof bags. If you're still shopping for something to carry a phone, wallet and keys, see how waterproof crossbody bags compare.