In this study, we perform retrieval and finite element analysis to better understand the role of trunnion and head taper surface topographies in the behavior of hip implant modular junctions. Using finite element analysis and the retrieval data, we evaluate contact mechanics and plastic deformation during implant assembly for head tapers and trunnions of different materials. Seventy-seven retrieved implants were analyzed. Trunnions were composed of CoCrMo, Ti6Al4V, Ti6Al7Nb, and Ti12Mo6Zr2Fe (TMZF); head tapers were made of CoCrMo and ceramic. The valley-to-peak height and peak-to-peak spacing between the machining marks of the trunnions and head tapers were measured with white-light interferometry and a profilometer. Other implant dimensions were obtained with a coordinate measuring machine and caliper. A global finite element model was created representing a half cross section of the femoral head, trunnion, and impactor. A local finite element model was created representing the topography of the trunnion and head taper. Dimensions were taken from median retrieval measurements. A 4 kN impaction force was applied to the global model femoral head over 0.2 s. The local model was driven by the global model displacements. Ti6Al7Nb trunnions had the highest and narrowest machining marks, while TMZF trunnions had shallow machining marks. Head taper machining marks were shallower and narrower than trunnions. The topography had implications on the surface damage score. In finite element analysis, CoCrMo head tapers had higher contact area and displaced further onto trunnions than ceramic head tapers. TMZF trunnions had the highest and Ti6Al7Nb had the lowest contact area. Ti6Al4V trunnions had the highest and CoCrMo had the lowest contact stress. Areas of plastic strain were largest in trunnions paired with ceramic femoral heads. Little plastic strain was present in Ti6Al4V and TMZF trunnions. Knowledge of contact mechanics and plastic strain is essential for predicting implant stability and preventing micromotion and subsequent fretting corrosion.