In a market study, more than 60 representative cylinder valves that are used in SCUBA diving were tested for burn-out safety according to the current standard ISO 10297 [2006, “Transportable Gas Cylinders—Cylinder valves—Specification and Type Testing,” International Standards Organization, Geneva, Switzerland]. This standard requires that oxygen cylinder valves have to be pressure-surge tested to prove their ignition resistance. After testing, the samples are disassembled and the nonmetallic materials shall not show any traces of ignition for the valve to pass the test. This is usually done by visual inspection. The existing pass–fail criteria, however, cause problems sometimes when the test results have to be interpreted or are not sufficient to ensure a safe use of the valve. This market study reveals what changes a non-metallic material may show after pressure-surge testing. It also shows the influence of the different design geometries in the seat area on the nonmetallic materials. After testing, some nonmetallic materials do not show any distinct traces of ignition or combustion. Only a change in color, notable surface deterioration, and loss of material is visible. Occasionally, the valve is no longer gas-tight, although no traces of ignition can be detected. However, according to the standard, the valve has passed the test. This paper provides various reasons why the pass–fail criteria of the oxygen pressure surge need to be more tightened. This applies not only to the current version of the standard ISO 10297, but for any standard that requires performance of the pressure-surge test for oxygen components.