The main components of fragility analysis of structures can be listed as compilation of ground motion dataset, structural analysis and selection of ground motion intensity measures. In this study, epistemic uncertainty with respect to the selection of the ground motion dataset and intensity measure is evaluated. In this respect, the fragility curves of building type structural simulations for four different ground motion dataset and three selected intensity measures are developed by considering two different approaches and possible variations are investigated. The detailed evaluations which depend on the fragility curves and the deterministic earthquake scenarios indicate that the compiled ground motion datasets, selected intensity measures and fragility curve development approaches can considerably affect the analysis results. However, it can be concluded that the most influential parameter on the analysis results is intensity measure and its effective selection can considerably decrease the epistemic uncertainty in fragility analysis.