evaluation of InteliWeb in eServices context
scientific results / evaluation of InteliWeb eService
Sampling and Procedure
We have conducted experiments at the University of Cyprus and Athens. The students,
first created their profile using the online psychometric tests, and then they navigated
through the environment. Initially the environment was shown without any adaptation
taking place and then the content was adapted and personalized based on their profiles.
During their navigation we provided them with tasks regarding the content.
Sample: 89 students; 40% of the participants were male and 60% were female,
and their age varied from 19 to 23 with a mean age of 21;
The environment: An e-Commerce content on a series of laptop computers.
Dependent Variables
- Task accuracy (number of correct answers)
- Task completion time
- User Satisfaction (questionnaire)
Results
The most robust and interesting finding was the fact that users in the personalized
condition were more accurate in providing the correct answer for each task. The
same user in the raw condition had a mean of 0.6 / 3 correct answer, while in the
personalized condition the mean rose to 2.3 / 3.
Equally interesting is the fact that users in the personalized condition were significantly
faster at task completion. The mean aggregated time of answering all three questions
was 541 seconds in the raw condition, and 412 in the personalized (see diagram).
As it concerns the satisfaction questionnaire, 23 users leaned towards the personalized
environment, 17 had no preference while 13 preferred the raw.
All the results are demonstrating significance at zero level of confidence.
