theme 4 : before

Research paper:  Do human cognitive differences in information processing affect preference and performance of CAPTCHA?  ( LINK )

1. Which quantitative method or methods are used in the paper? Which are the benefits and limitations of using these methods?

This paper try to conduct a experimental research using computer collecting data aim to improve user experience in using CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) in order to make it more user friendly as well as preserve the security. Therefore , they conduct two user studies

1) explored the effect of users’ cognitive styles (Verbal/Imager) on user preference and task performance of two complementary types of CAPTCHA mechanisms; text-recognition and image-recognition. [1]

2) explores the effect of users’ cognitive processing abilities (speed of processing, controlled attention, working memory capacity) on task performance in regards with different levels of complexity [1]

The benefit of using quantitative is that they can gather a lot of data from many participants answering both of the two question they wanted to find out. For the sampling the first study has a total of 131 undergraduate Computer Science students participated (76 male, 55 female, age 20–25, mean 23) having recorded the same number of CAPTCHA sessions. The second study has a total of 125 undergraduate Electrical Engineering students participated (48 male, 77 female, age 18–28, mean 20). A total of 125 CAPTCHA sessions have been recorded. [1]

For the evaluation, they consider data about the task performed by the users. This includes the type of CAPTCHA (text or image), the level of complexity (baseline or higher), the total time (in seconds) and total number of attempts to successfully solve the CAPTCHA challenge. Based on the total number of attempts, the success rate (in percentage) of each session was also calculated. [1]

They reason why it is a good research finding using quantitative method is that it consists of observing and explaining each steps, collecting information, and analyzing the information

For the limitation, people cognition style could be difficult to define since each of people has different background and experiences. Therefore the result might be different in various different groups. more over, Since quantitation method is a collection of numbers they could overlook some of the factors each participant being test. Such as user eyesight , color-blind,  speaking language, participants surroundings etc.

      2. What did you learn about quantitative methods from reading the paper?

Each experimental study only focuses on one topic. They use two user-study to answer two questions. One study for the style test of the CAPTCHA, another one for the speed test. Their sampling group have an appropriate number of users participating in the study. One group of study, participants should have some similarity in their age, study background. The whole group is mix with both male and female. Same number of the testing material was put into each group therefore they can get a credible result.

3. Which are the main methodological problems of the study? How could the use of the quantitative method or methods have been improved?

The validity result is the main concern. they concern that the experimental design must be under the real-life context in order to get an accurate result. they also concern whether the result can be generalise to a wider extent.

Another limitation was the diversity of participants because in this study they only have undergraduate student. The sampling group could expand to more various background. They could have another study of people who are in working class or people from other specialized area included.

The result numbers that their experiment had collected need to be analyzed carefully because it is not only about user-friendly but also security. The result would identify that one type of CAPTCHA is more effective at the same time it have to also test not to be weak for security.

reference: 1. Belk, M., Fidas, C., Germanakos, P., & Samaras, G. (n.d.). Do human cognitive differences in information processing affect preference and performance of CAPTCHA? International Journal of Human-Computer Studies, 1-18.

---------------------------------------------

Read the following paper written by Ilias Bergström and colleagues. Reflect on the key points and what you learnt by reading the text. Also, briefly discuss the questions below.

• IEEE VR 2012 - Drumming in Immersive Virtual Reality

Keypoint of this research is to find if the form of body in virtual reality effect the behaviour using the immersive virtual reality at the first person perspective of people playing drum with different type of skin color and clothes.

1.    Which are the benefits and limitations of using quantitative methods?

Quantitative data can help collect more data compare to qualitative which is better for generalize the result. More data can be collect with more number of people per session. Instructions can be provide within the method material which make it easier for the researcher.For limitation, quantitative can overlook other contextual factors surrounding the participants by just collecting numeric result; missing something that cannot obtain by numbers or tick box. The interpretation maybe more difficult because you have to turn numbers into understanding.

2.    Which are the benefits and limitations of using qualitative methods?

Qualitative data is an in-dept analysis that researcher will have a close observation of the participants when they answering questions. Research an ask participants additional questions. Or participants can ask researcher to clarify when they don’t understands the process. For limitation, qualitative can be time consuming and more expensive than quantitative. The researcher has to be with the participants in each step of procedure. Research can conduct with less number of participants compare to quantitative which can be difficult to have a generalize result.