W3C Workshop on the Future of Social Networking, 15-16 January 2009, Barcelona
John G. Breslin 1,2, Stefan Decker 1, Manfred Hauswirth 1, Gearoid Hynes 1, Danh Le Phuoc 1, Alexandre Passant , Axel Polleres 1, Cornelius Rabsch 1, Vinny Reynolds 1
Sensors have begun to infiltrate people's everyday lives. They can provide information about a car's condition, can enable smart buildings, and are being used in various mobile applications, to name a few. Generally, sensors provide information about various aspects of the real world. Online social networks, another emerging trend over the past six or seven years, can provide insights into the communication links and patterns between people. They have enabled novel developments in communications as well as transforming the Web from a technical infrastructure to a social platform, very much along the lines of the original Web as proposed by Tim Berners-Lee, which is now often referred to as the Social Web. In this position paper, we highlight some of the interesting research areas where sensors and social networks can fruitfully interface, from sensors providing contextual information in context-aware and personalized social applications, to using social networks as "storage infrastructures" for sensor information.
A few weeks before the recent 7th International Semantic Web Conference, a social networking application was set up in order to let attendees define and maintain their social network and discuss particular topics. One feature provided by that application was the ability to let one define people not only as friends, but as people they would like to meet. While it was obviously an interesting idea to foster and extend one's social network, there was no other way to facilitate this apart from sending e-mails at the conference in order to set up meetings at particular times in particular locations. In that case, it would have been interesting to have had the benefit of a system that could identify people's precise locations (using for instance GPS-enabled phones, Wi-Fi positioning techniques or Bluetooth devices interacting with sensors in each conference room), letting people define their availability and then allowing them to send (or receive) alerts to or from people that wanted to meet them while they were in a shared place, e.g. a coffee room.
Going further, other sources of social data available on the Web could be used to minimize the required user input. While in the aforementioned cases, social networks have been explicitly defined by the users, they could also have been implicitly defined from online activities and footprints of users. For instance, one might receive an alert on their mobile phone as soon as someone that they exchange messages with on a blog or on some mailing list is in the same room. Alerts might also be sent when people with a shared interest are in the same place. Obviously, considering shared but niche interests might be more accurate, especially in technology-oriented conferences where many people share widely-common interests as "Web" or "Social Network". Since this step requires interoperability between various social networking applications and data sources, we will emphasize in the next section the role of the Semantic Web has to play.
A use case could be to find a calm place based on "noise logs" recorded by one's social network. In order to find a calm place for a business meeting or in a work environment, a user could query his or her social network to find such information. This would include both sensor information, for example, obtained as in the NoiseTube experiment where a small application provides noise level readings from standard mobile phones, as well as ratings provided by users. Smart phones can be leveraged in order to identify these suitable spaces. Each phone within a given area could periodically have its microphone polled to get an estimation of how noisy a particular location is. This data can be augmented by scanning for both Wi-Fi and Bluetooth devices nearby to provide an approximation for how calm a particular location is. Once again, the location of the device can be gathered via GPS if available or via Wi-Fi positioning techniques. In the ideal case where microphone, Bluetooth, Wi-Fi and location data is available from all smart phones within a building then a simple query can be sent to the smart phones in order to find an appropriate "calm place". However, in situations where there is information missing we can treat people as sensors and issue a query to them. For instance if we know that John is in a quiet place but we don't have location details for him, we can send him an SMS to retrieve his location. To actually find a calm place, a social version of "expanding ring search" could be used, i.e. query your direct network, and if a satisfying answer is not found, then expand the search perimeter to people via one indirection in the social network (probably at the cost of less accurate information because of access rights), etc. There is an incentive to expand the social network and the sensor network access to retrieve better data and more qualified query answers.
Along the lines of this scenario, the sensors could also be viewed as a "social network" on a technical level. Sensors become part of the social network, as they are just one more set of paths in the graph. A trust relationship similar to the one in a standard social network could form the basis for this. The idea boils down to the question: can a sensing network be extended through the people who own the sensors? For example, person A owns sensor A and person B owns sensor B and person A knows person B: that implies that sensor A "knows" sensor B.
Social networks and sensor networks can also be combined to support independent living and health support for elders. By deriving semantic presence based on context from sensor-enabled social networking devices, we can carry out useful tasks for the elderly. For example for daily living purposes, we can check the status of the friends and find shopping or walking buddies to promote the mobility of elders. By using semantic representations of information from sensors, we can build on the idea of connecting people through shared activities and interests [1]. More importantly, we can send alerts based on abnormal activity patterns. Through sensor readings of body position or health measurements, we can issue requests for attention not just to carers or clinicians but to nearby friends in the elder's social network.
Collaborative rehabilitation is also possible using sensor-enabled portable devices. More and more portable devices are supporting sensor-based interactions, from peripherals (Nike+iPod) to integrated sensors (the original iPhone made good use of its accelerometer, while the latest iPhone 3G has added various proximity and light sensors). We can make use of the Social Web and Sensor Networks to create collaborative applications for portable devices to encourage exercise, à la the Wii. As an example of how this could be done, we could begin by finding contacts on the social network with similar interests or by GPS location (e.g. using FireEagle). This social network of friends can then be used to power collaborative applications (CAPTCHAs, the ESP game, quizzes) where progress can be made by the group when a certain level of exercise has been achieved. Then, as a final step, the resulting sensor data is sent to physicians for analysis.
While the use of Social Web applications is now a common practice, either personally or professionaly, it raises many issue of interoperability. Integrating data from different networks, or finding all related content about a particular topic is generally a complex task. Yet, the Semantic Web, and lightweight models such as FOAF and SIOC - the latter being described in detail in another paper of that workshop - can help to provide a unified layer on the top of existing applications. Using a common semantic, data from the Social Web can be seen as an interlinked graph of social activities and interactions on the top of heterogeneous and distributed services. It hence ease the process of data integration between application, as well as querying. Considering our previous use-case of acquaintance location for implicit social networks, social networks might be automatically mined from this global graph by applying Social Network Analysis techniques, a task which cannot be done considering isolated applications.
Moreover, that Social Semantic Web stack can be efficiently integrated with other efforts from the Semantic Web community. For instance, data extracted from Wikipedia (thanks to DBpedia) can be used to model interests of a user. Then, applications can identify that two person share common interests because one states he enjoys The Clash and the other one Joe Strummer, and there is a short semantic distance between both, that can once again be computed by analysing Semantic Web data available on DBpedia, for instance.
Although sensors have been used in embedded, manufacturing and control systems for many years, it is only recently that sensors have become an area of research within their own right. Advances in the research of wireless sensor networks in the last decade have led to their deployment in application domains as diverse as environmental monitoring, logistics management and smart spaces. In all of these domains, the sensors themselves and the application, running either on the individual sensor or built on top of the sensor network, is embedded within the physical environment, and is independent of any one individual. A second area of research, pervasive computing, has also gained traction within the last decade and is more oriented towards that individual. Research areas, such as context-aware computing and personal area networks within pervasive computing, are driven by the users themselves, and sensor devices that may be either on their person or associated with them, often by proximity.
Sensors are becoming more prevalent in mobile devices in recent years. Both the Apple iPhone, and the Nokia N95 now contain GPS and accelerometer sensors. Coupled with bluetooth and wifi communication stacks, the mobile phone has now become a sensor gateway for the individual. A wide range of bluetooth sensors, such as heart monitors, and environmental monitors can now be associated with these mobile phones enabling a new paradigm, the personal sensor network, in which the individual becomes the sensor hub.
In open context-aware systems ontologies are the modelling method of choice due to their high level of formality thus enabling easier interoperation with loser coupling between systems. These ontologies are focused on ubiquitous computing scenarios, in particular smart-home and smart-office scenarios, and create custom classes for their scenarios. However, of late there has been several notable cases where sensor models have been combined with social semantic data, the Annotation CReatiON for Your Media ontology is an example of this. The ACRONYM ontology's purpose is to lend context from the real-world to the creation of resources in particular it is integrated with other ontologies from social networking, multimedia, geography and context-awareness (FOAF, iCal ...) to aid the semi-automatic annotation of media.
There have been also several efforts (NASA/JPL Sensor Webs Project, 52°North, SANY IP, European Space Agency, etc) using these specifications (SensorML, Observation Mesurement schema, etc.) in creating applications, platforms, and products involving Web-connected devices such as flood gauges, air pollution monitors, stress gauges on bridges, mobile heart monitors, Webcams, and robots as well as space and airborne earth imaging devices. Sensor Web XML-based specifications were created in consideration of Semantic Web technology. There are some works annotating sensor markup encodings, for example, GRDDL in SensorML. Semantic Sensor Networks workshop in 2006 showed the trends of leveraging Semantic Web technology in Sensor-based applications.
As explain in the first section, we envision a lot of scenarios which will rely on information from sensor sources along with information from social networks and "traditional" Web information sources and classical information sources. However, to date this integration and interaction is still decoupled in a general setting, i.e., they are not immediate. This problem is the driving idea behind DERI's Semantic Reality effort, which targets the information-driven online integration of the physical world and the Web. Similarly, as the Internet has changed the way people communicate in the virtual world, Semantic Reality extends this vision to the physical world, enabling novel ways for humans to interact with their environment and facilitating interactions among entities of the physical world (Internet of Things). The physical world will be represented in cyberspace and information on our environment will become ubiquitously available on the Internet. We believe that the same vision also applies to the integration of sensor technologies and social networks.
From a practical point of view, as both Social Networks and Sensors information can be modeled using Semantic Web technologies, they can be efficiently connected in an interoperable and straightforward way (see Figure 1). The flexibility of RDF indeed allows to link together data from various sources, while using shared ontology offers common semantics for this data.
Figure 1: Interoperability between Social Networks and Sensor Networks using Semantic Web technologies
For instance, the ConServ system developed in DERI benefits from Semantic Web technologies to integrate existing data sources, such as geographic information from Geonames, FOAF and iCal with sensors data. ConServ allows users to provide ubiquitous applications and context-aware web-services controlled access to their context data. ConServ can be leveraged to provide a service which, for instance, can inform someone about the presence (or not) of a colleague in a building a few minutes before a meeting, so that it can be postponed if the collegue is still far away. By using RDF modeling, that application could be extended to take into account the Social Networking aspects we identified in the two use cases. The only requirement is the use of RDF to model those activities as stated before, which can easily be done thanks to the numerous exporters available for existing applications.
By combining Social Network and sensors, applications can provide an extension of social activities through sensors, as user activity is modeled not by voluntary user input, but can be automatically generated by sensors. It hence enhances the idea of ubiquitous social networking, that can be observed on microblogging services such as Twitter, where some people tend to publish simple updates containing only their current location as GPS coordinate.
The integration of sensor networks with social networks leads to applications that can sense the context of a user in much better ways and thus provides more personalized and detailed solutions. Applications are deeper integrated into the daily life and context-awareness through social network and sensor network data becomes a key enabler for these emerging services. Obviously, different issues will have to be taken into account, especially regarding privacy and personal-data access, as we exposed in another position paper of that workshop. We hope that this convergence can be as fruitful as the convergence of the Semantic Web and social software, and can be beneficial for both communities, by extending and improving (1) social networks by using sensors and (2) sensors networks thanks to social networks. Furthermore, we think that some interaction between the Semantic Web and the Mobile community within a W3C group could be benefical to this convergence.