Digital technologies have been used for measurement purposes and whether the test medium influences the user is an important issue. The aim of this study is to investigate students' performances and duration differences between online and paper-pencil tests. An online testing tool was developed and administered in order to determine the differences between the traditional paper-pencil tests and online tests concerning students' performances and the duration on tests. This tool enables to add questions that utilize an online database and which are in the form of multiple choice (with 5 or 4 options), true-false, matching, filling in the blanks, with multiple answers, with short answers, with long answers, and it also enables to prepare tests and to turn them into paper-pencil test mode. Performance test was applied with both online and paper-pencil modes on junior students at one of the universities in Turkey. Besides, the online testing tool developed within the context of the study was evaluated by instructors with respect to usability, relevance to the purpose and design. Instructor and student questionnaires are developed to determine the opinions on the online testing tool and online tests. Results showed that there was no significant differences between the performances on online and paper-pencil tests. On the other hand, the time they spent on the online test has been longer than the time they spent on paper-pencil test. Students found the online testing tool easy to use and stated that online test medium is more comfortable than paper-pencil tests. However, they complained about external noises, tiredness, and focusing problems regarding the online examination mediums. Generally, instructors have also appreciated the online testing tool's design and they agree on the fact that it serves for its purposes.