Profiling users from users’ behavior

• Silvia Mirri. Dipartimento di Scienze dell'Informazione - University of Bologna - silvia.mirri@unibo.it
• Catia Prandi. Dipartimento di Scienze dell'Informazione - University of Bologna - catia.prandi2@unibo.it
• Paola Salomoni. Dipartimento di Scienze dell'Informazione - University of Bologna - paola.salomoni@unibo.it

1. Problem Description

Profiling users has been frequently exploited in accessibility context, to transcode content according to users’ needs. Usually, users have to explicitly declare their preferences (font size, luminance contrasts, media alternatives) and needs (assistive technologies they use), while repositories can be used to recover devices capabilities [1]. Then transcoding and adaptation are driven by categorizing device capabilities and users’ needs.

We propose the use of machine learning concepts to learn users’ preferences, understanding users’ experience and predicting users’ needs. We have designed a system which takes into account users’ behavior and automatically adapt Web pages (or just paragraphs). Our system profiles users by considering characteristics they have discarded and the ones they have preferred, modeling users with different needs: aging people, people with low vision, people with dyslexia, people with color blindness, etc. The more the user discards a characteristic the more the system learns to automatically adapt it, by substituting with the preferred one.

2. Background

This work is mainly based on different ideas: the idea of taking into account users’ experience so as to provide the best adaptation (also called “Experiential transcoding”), the idea of providing automatic adaptations on the basis of users’ previous behavior (just like recommendation systems), idea of improving Web pages legibility and readability by providing text characteristics (such as font size, font type, word and letter spaces, etc.) adaptations and the idea of profiling users’ preferences and some device capabilities by means of xml-based standards.

A very brief state of the art for each of these areas is listed below:

• Experiential Transcoding: providing adaptations on the basis of users’ experiences has been introduced in [2] and [3]. In [3] eyetracking has been used to understand users’ experiences.
• Recommendation Systems: Machine learning algorithms have been used in several works to provide recommendations according to learnt users’ preferences (in e-commerce Web sites, social networks, search engines filters, etc.) [4].
• Legibility and Readability: font type and size and other text characteristics (word and letter spacing, line height, color contrast, etc.) affect online reading. Several studies have been done so as to identify these characteristics effects and which characteristics better support users, according to their specific reading abilities [5, 6, 7, 8].
• Device and User Profiling Standards: xml and rdf-based standards have been used to profile users in adaptation systems to meet needs of users with mobile devices and users with disabilities. Most common ones are: CC/PP and UAProf (to profile device capabilities), IMS ACCLip and ISO PNP (to profile users’ needs in terms of accessibility). Some works have been done to combine and exploit these standards [9, 10].

3. Approach

We have developed a prototype which adapts typographic characteristics in Web pages paragraphs (i.e. font size, font face, luminance contrasts, colors). To understand user’s experience and to learn user’s preferences (predicting user’s needs) we have used a machine learning algorithm, based on the Reinforcement Learning concept and on the idea of reward/punishment [11]. No initial profile is provided, but this is automatically created and fed by our system, as the user explicitly requests for adaptations (by means of a contextual menu, as shown in Figure 1) and as he/she accepts or rejects adaptions the system proposes or automatically performs.

Our profile is a collections of characteristics gathered by the system, shared among all the devices the user exploits.

The system:

• updates the profile, adding new characteristics, updating reward/punishment values;
• learns user’s preferences: discarded and chosen characteristics;
• punishes discarded characteristics;
• rewards preferred characteristics.

The profile will be the more accurate the more the user asks for adaptations.

We have designed an xml-based profile, structured in different parts, according to devices the user exploits. In each part, the system stores:

• text characteristics (as tags) the user has requested or the user has discarded;
• the related value (the “v” attribute);
• a number (the “w” attribute) which states the reward/punishment.

The “w” value varies according to users’ behavior:

• If “w” is < 0: the user has discarded the “v” value characteristic;
• if “w” is > 0: the user has preferred such “v” value characteristic;
• if “w” is = 0: the “v” value characteristic has obtained the same quantity of rewards and of punishment.

The absence in the profile of a characteristic or a specific “v” value means that the user has never requested such characteristic adaptation or he/she has never discarded or chosen such a “v” value.

For each device the profile stores:

• type;
• id;
• display width;
• display height.

A fragment of profile is the following one:

…
<device type=”tablet” id=”2” display_width=”1024” display_height=”768”>
…
<font_face family=”sanserif” w=”5” v=”arial”/>
<font_face family=”serif” w=”-2” v=”times new roman”/>
…
<font_size w=”8” v=”18”/>
<font_size w=”-5” v=”9”/>
…
<line_height w=”5” v=”1.5”/>
<line_height w=”-2” v=”1”/>
…
</device>
…

The system works as follows:

1. When the user opens up a Web page, the system parses the text characteristics, taking into account the profile.
2. If there are some characteristics the user has discarded (with a negative “w”), the system computes if automatically adapting them, providing the preferred values (with the highest “w”) or if proposing adaptations.
3. The user exploits the page with adapted or proposed characteristics.
4. If the user ignores the automatic adaptations the reward is +1. If the user rejects such adaptations the reward is -1.
5. If the user accepts the adaptations the system has proposed the reward is +1. Else, the reward is -1.
6. If the user applies an adaptation to a characteristic the system assigns +1 to the requested characteristic and -1 to the discarded one.
7. Updated rewards and/or new characteristics are stored into the profile.

4. Challenges

Major difficulties are related to the need to keep data not only about users’ preferences but also about characteristics the users have discarded: it is necessary to frequently update the user’s profile, which can become wide. On one hand this means a more complex computation with waiting times (to update the profile and to wait for automatically adapted pages), on the other hand this means more adequate and user-centered adaptations.

5. Outcomes

We have designed and developed a prototype which adapts Web pages, as a Firefox extension. Users can activate a contextual menu to set the adaptations, then the system suitably changes the HTML and/or the CSS code, on the client-side. In the meanwhile, the prototype tracks users’ behaviors, learning their preferences and needs (feeding the xml-based profile) without asking the users to explicitly declare them. Then the system automatically applies or proposes adaptations.

The prototype has been tested on laptops and on Samsung Galaxy Tab 2 devices.

6. Future Research

A user testing phase is needed and would involve several users with specific needs (i.e. aging people, users with low vision, users with dyslexia, users with color blindness, etc.) equipped with different devices.

Further work is needed to develop extensions and/or adds-on for the most commonly-used browsers (such as Chrome, Internet Explorer, etc.) and for other kinds of documents viewers and readers, letting the system adapt not only HTML pages, but also other markup documents (i.e. LaTeX ones).

An interesting idea which needs further investigation is to combine the proposed profile with other well-known ones, such as IMS ACCLIP or ISO PNP (which describe users’ needs in terms of accessibility) and CC/PP or UAProf (which describes device capabilities), with the aim of providing a more complete user’s and device profile.

References

1. WURFL (2013) Wireless Universal Resource File Library. Available: http://wurfl.sourceforge.net. Last accessed 6^th June 2013.
2. Brown, A., Jay, C., Harper, S. (2010) Audio access to calendars. In Proceedings of the 7^th International Cross-Disciplinary Conference on Web Accessibility (Raleigh, NC, USA, April 26-27, 2010). W4A’10, ACM New York, USA. DOI:10.1145/1805986.1806028.
3. Yesilada, Y., Harper, S. and Eraslan, S. (2013). Experiential Transcoding: an eye tracking approach. In Proceedings of the 10^th International Cross-Disciplinary Conference on Web Accessibility (Rio de Janeiro, Brazil, May 13-15, 2013). W4A’13, ACM New York, USA. DOI:10.1145/2461121.2461134.
4. Swapna Raj, P., Ravindran, B. (2008) Personalized Web-page Rendering System. In Proceedings of the International Conference on Management of Data (Mumbai, India, December 17–19, 2008 ). COMAD 2008, pp. 30-39.
5. Bernard M., Liao C. H., Mills M. (2001) The Effects of Font Type and Size on the Legibility and Reading Time of Online Text by Older Adults. In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (Seattle, Washington, 31 March-5 April, 2001). CHI 2001, pp. 175-176. DOI:10.1145/634067.634173.
6. Bernard M., Mills M., Peterson M., Storrer K. (2001) A Comparison of Popular Online Fonts: Which is Best and When? In Usability News, 2001. Available: http://usabilitynews.org/a-comparison-of-popular-online-fonts-which-is-best-and-when/.
7. Beymer D., Russell D. M., Orton P. Z. (2008) An Eye Tracking Study of How Font Size, Font Type, and Pictures Influence Online Reading. In Proceedings of the 22^nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction 2008 (Liverpool, UK, September 1-5, 2008). BCS-HCI '08, pp. 15-18.
8. Biswas, P., Langdon, P. (2011) Investigating the accessibility of program selection menus of a digital TV interface. In Proceedings of the International Conference on Human Computer Interaction (Orlando, FL, USA, July 9-14, 2011). HCII 2011, pp. 425-434. DOI:10.1007/978-3-642-21619-0_52.
9. Ferretti, S., Roccetti, M., Salomoni, P., Mirri, S. (2009) Custom E-learning Experiences: Working with Profiles for Multiple Content Sources Access and Adaptation. Journal of Access Services, Taylor & Francis. 6(1-2), pp.174-192. DOI:10.1080/15367960802301093.
10. Salomoni, P., Mirri, S., Ferretti, S., Roccetti, M. (2008) A Multimedia Broker to support Accessible and Mobile Learning through Learning Objects Adaptation. ACM Transactions on Internet Technology, ACM Press. 8(2), pp. 9-23. DOI:10.1145/1323651.1323655.
11. Mirri, S., Prandi, C., Salomoni, P. (2013) Experiential adaptation to provide user-centered documents personalization, submitted to the Proceedings of the 6^th International Conference on Advances in Human oriented and Personalized Mechanisms, Technologies, and Services (Venice, Italy, October 27 - November 1, 2013). CENTRIC 2013.