Multiple site damage (MSD) in aircraft structures can be defined by the presence of numerous minute cracks arising at different locations such as rivet holes in lap joints. One source of initiation can be due to residual stresses induced in the lap joints during the riveting process. This paper describes a finite element methodology of a quasistatic, displacement controlled riveting process to capture the local geometry features representative of service hole quality for accurate simulation of residual stresses in riveted lap joints. A three-dimensional finite element (FE) model that takes into account material, geometric, and contact non-linearities is developed using ABAQUS/Standard 6.4–1 to simulate the joining process. The interest of FE simulations is to conduct parametric studies to observe effects of varying hole clearance, misaligned holes, friction, and presence of debris on the residual stress state in the skin. Based on studies conducted, comments are made regarding any observed variation in tensile hoop stress, which is the primary source of fatigue cracks in riveted lap joints. A relation between the rivet-head deformation ratio and rivet-head height with applied displacement is developed using the finite element model, as a means of controlling the rivet-head deformation.