Material degradation within taper junctions of modular total hip replacements remains of great concern. Imprinting and column damage are two damage modes that frequently occur on head taper surfaces. Both can cover large areas and therefore can be associated with significant material loss. It was the purpose of this study to determine the prevalence of imprinting and column damage on a group of retrievals collected at our medical center and to identify damage pathways on selected cases. We asked two research questions: (1) How do CoCrMo stems imprint into CoCrMo heads? (2) Does alloy microstructure influence the column damage pattern in CoCrMo heads? In order to answer these questions, we conducted a retrieval study on moderately to severely worn femoral head taper surfaces that were paired with stems of different materials. All components were viewed under a stereo-light microscope to determine the presence of imprinting and column damage. Selected cases were further studied by means of scanning electron microscope, interferometry, and metallography to determine damage mode and the potential role of alloy microstructure. Our results demonstrated that imprinting is independent of the stem material but highly dependent on its topography. The imprinting process is at least initially driven by fretting and the generation of oxide particles. Column damage on the other hand is highly dependent on the microstructure of wrought CoCrMo alloys, which can exhibit banding resulting from slight alloy segregations that were characterized by molybdenum depletion. Therefore, column damage may be prevented by avoiding banding of the alloy during the thermomechanical processing. This study demonstrates that it is important to consider differences among the occurring degradation mechanisms and to understand how they relate to material and design factors.