Overview

Sentient Graffiti enables mobile users to profit from the benefits of Ubiquitous Computing in uncontrolled environments, only requiring in exchange, the participation in a community of users interested on publishing and consuming context-aware empowered annotations and services. Users annotate objects and spatial regions with multimedia data or web services which are only made available to other users when those match the contextual attributes (location range, period of time, and so forth) previously assigned to the resources.

A virtual graffiti can be thought of as a virtual post-it note in the form of an XML document which combines some content (multimedia presentation or web service front-end) with some keywords summarising the content, and some contextual attributes (profile of creator, location, time interval and so on) which restrict who, when and where can those annotations be seen

Virtual Graffitis are edited through a PC web browser or, on the move, by means of the Sentient Graffiti mobile client and then published on the back-end. We currently support two types of graffiti content:

  1. Multimedia presentations in SMIL format (including video, audio and images)
  2. URLs pointing to web service front-ends
Before a graffiti can be published edited it must be associated to:
  1. a spatial region (GPS)
  2. a Bluetooth location
  3. an object using its identity given by a barcode (TRIP) or
  4. a RFID tag

Sentient Graffiti presents a client/server architecture (see figure below) where users run a SG client in either their mobile device or a computer’s web browser, whilst a server-side component, namely Sentient Graffiti Server, stores, indexes and matches user annotations against user’s current context published by SG clients.

The generic SG Mobile Client provided enhances user to environment interaction by enabling the following four interaction mechanisms (Multi-modal Interaction):

  • Pointing – the user can point his camera phone to a bi-dimensional visual marker and obtain as result all the graffitis associated to such marker which are relevant and can be seen by him.
  • Touching – the user can use a mobile RFID reader bound to a mobile through Bluetooth to touch an RFID tag and obtain all the relevant graffitis associated to that tag. We used the ID Blue (http://www.cathexis.com/) pen-like Bluetooth-enabled RFID reader to enable this interaction mode.
  • Location-aware – mobiles equipped with a GPS in outdoor environments can obtain all the relevant nearby graffitis in a certain location range which are relevant to the user. The Bluetoothenabled GPS device is used to test the location-aware graffiti retrieval capacities of Sentient Graffiti.
  • Proximity-aware – the SG Mobile Client can retrieve all the graffitis published in nearby accessible Bluetooth servers when the device is in Bluetooth range from the server. A user by means of proximity interaction scans for nearby Sentient Graffiti Bluetooth servers, finds one installed in a near digital information booth and browses through the private and public notes available for him in that context.

NFC (Near Field Communication)

An interesting technology which has emerged lately and which seems as an ideal candidate to help on providing a more natural way of interaction between the user and the environment is Near-Field- Communication (NFC). This technology is a combination of contact-less identification and interconnection technologies that enables wireless short-range communication between mobile devices, consumer electronics, PCs and smart objects. NFC offers a simple, touch-based solution that allows consumers to exchange information and to access content and services in an intuitive way. The emergence of NFC should simplify human to environment interaction, giving place to the Touch Computing paradigm, where users have only to wave their representing mobile devices in front of everyday objects augmented with RFID tags and visual markers or other NFC-aware devices in order to trigger the smart services offered by them. In fact, the combination of RFID and visual tagging of everyday physical objects and NFC-enabled devices will foster the UW/IoT vision where every resource that surrounds us and its associated services are available through some kind of networking (Bluetooh, Wi-Fi, GPRS/UMTS) and programming infrastructure (web service, semantic web service, RSS feed and so forth), easily discovered and consumed through our mobile devices. Sentient Graffiti, as an UW/IoTenabling platform, also leverages on the promising NFC technology as it will be later reviewed.

In order to further improve the support for human-to-environment interaction offered by Sentient Graffiti we have adopted NFC technology. The adoption of this technology on the mobile client has brought out improvements on the way users retrieve public and private virtual graffitis associated to surrounding resources. Our initial implementation of the Sentient Graffiti Mobile Client supported only non NFC-aware mobiles. The adoption of NFC technology and the implementation of a Touch2Launch service as part of the Sentient Graffiti MIDlet for the Nokia NFC 6131 device (using the Nokia NFC SDK ), has provided the following improvements for the touching and proximity aware interaction modes:

  • Touching interaction through NFC. So far, enabling touching interaction required the complicated scenario of pairing a Bluetooth RFID reader such as the ID Blue with a mobile device, and then force the user to carry both the mobile and the RFID reader, to be able of touching the RFID tags identifying intelligent service enhanced surrounding objects and so obtain the graffitis associated to them. This cumbersome scenario is now simplified for users with a Nokia NFC 6131, since the device itself is able of reading smart object identifying RFID tags. The combination of the Push Registry MIDP 2.0 and NFC technology prevents a user from even having to have the Sentient Graffiti application started in their mobile before undertaking an interaction with an augmented object. As soon as the user approaches his device to an RFID tag attached to an object, the Sentient Graffiti Mobile Client is started thanks to the MIDP 2.0 Push Registry functionality and configured to retrieve all the virtual graffitis associated to the read RFID tag. A user initiating a session of Sentient Graffiti Mobile Client on his Nokia NFC 6131 by touching an RFID tag.

  • Proximity-aware interaction through NFC. An important drawback is that the user had to wait until the Bluetooth discovery process concluded (about 10 seconds) to figure out whether that information point contained a Bluetooth SG server providing virtual graffitis of his interest. The use of NFC-technology has simplified this scenario considerably. Now, the binding between an NFC-enabled device (Nokia NFC 6131) and the Bluetooth server is carried out by simply touching with the mobile an RFID tag attached to the information point. Such tags offer Bluetooth connection details such as the Bluetooth MAC address of the server and the port where a service is waiting for mobile client requests. Moreover, thanks again to the combination of the Push Registry capacity of MIDP 2.0 and NFC, it is not even necessary that the user has the Sentient Graffiti Mobile Client started. By pointing his device to the information point tag, the application is automatically initiated, and most importantly, correctly configured so that it automatically retrieves virtual graffitis for the user from that Bluetooth Sentient Graffiti Server.

In conclusion, incorporating the Touch2Launch service into the Sentient Graffiti MIDlet for the Nokia NFC 6131 has simplified both touch and Bluetooth proximity-aware interaction with nearby smart objects. Thus, NFC makes Sentient Graffiti even more suitable as a platform to enable the discovery and consumption of services within UW/IoT by further simplifying two of its natural (pointing, touching) interaction supported modes.

Download

The contents of every graffiti are stored in the DSG Server. There are two ways to access that data:

  1. Through the website available at this location

  2. Installing the DSG mobile application on your mobile phone.

    Select your mobile phone model:

There's also available a demo application that uses the SG's API. BilboBus Alerter

User Guide

Table of Contents
  1. WebPage
  2. Mobile Client
  3. Application Examples

WebPage

1. Login to the Web

  • To access the DSG website, follow this link

  • When you a login dialogue is prompted, enter your user data or login as guest:

    • Username: guest
    • No password

  • You can check the "Remind Me" option if you want your user information to be remembered.

  • Press the Login button

  • We encourage you to become a registered user to fully enjoy the website.

    In the login page there's a link to register a new user, where you enter your personal data in order to create your account and start taking part on the DSG Community.

2. Web

Once you've logged in, you'll be redirected to the Main Page where you can view the available graffitis.

Users can add new graffitis by clicking on the map location where the new graffiti should be placed or browse through the graffitis already created by other users. Advanced filtering functionality is provided in order to reduce the set of graffitis returned.

You'll find all the available actions in the upper menu bar:

  • Main Menu
  • User
    • Edit Profile
    • Filter
    • Create New TRIP
    • Rss Feed
  • Tag List
    • Tag Cloud
    • Graffiti Domain List
  • Logout


Mobile Client

1. Install

The installation process is very easy. It only takes a short while. Just follow these steps to get the application running:

  1. Send the .jar file to your mobile phone by means of Bluetooth or infrared.
  2. Find the .jar file and launch it in your mobile's menu.
  3. The Installation will start. An information screen will appear showing you that installation is taking place.
  4. Accept the security warning, and select the installation options.
  5. Wait until the installer finishes.
  6. The application will be now ready to be used in your mobile phone.

These images show the complete installation process.


2. Login to the Application

Once the DSG Mobile client has been installed on your device, you're ready to start:

  1. Open the DSG Midlet.
  2. After the splash screen, the login screen is displayed.
  3. Enter your user data or login as guest:
    • User: guest
    • No password
      • Installing the DSG mobile
    • Select the Login option.
    • If the login information is correct, the application main menu will be displayed.

These images show the complete login process:


The following snapshots depict the Sentient Graffiti system in action: