The automatic ultrasonic test can be applied to welded joints of steel bridges instead of the radiographic test. However, there are not appropriate specifications for automatic ultrasonic testing. In order to establish specifications, a collaborative study was organized by the Ministry of Construction. A round robin test was conducted twice for this objective. This paper describes the details of the round robin tests.