Google boost Jabber + VOIP, Skype releases IM toolkit, Jabber for P2P SPARQL?

Interesting times for the personal Semantic Web: “Any client that supports Jabber/XMPP can connect to the Google Talk service” Google Talk and Open Communications. It does voice calls too, using “a custom XMPP-based signaling protocol and peer-to-peer communication mechanism. We will fully document this protocol. In the near future, we plan to support SIP signaling.”

Meanwhile, Skype (the P2P-based VOIP and messaging system) have apparently released developer tools for their IM system. From a ZDNet article:

“Skype to wants to embrace the rest of Internet,” Skype co-founder Janus Friis said during a recent interview.

He did offer hypothetical examples. Online gamers involved in massive multiple player mayhem could use Skype IM to taunt rivals and discuss strategy with teammates. Skype’s IM features could be incorporated, Friss suggests, into software-based media players for personal computers, Web sites for dating, blogging or “eBay kinds of auctions,” Friis said.

I spent some time recently looking at collaborative globe-browsing with Google Earth (ie. giving and taking of tours), and yesterday, revisiting Jabber/XMPP as a possible transport for SPARQL queries and responses between friends and FOAFs. Both apps could get a healthy boost from these developments in the industry. Skype is great but the technology could do with being more open; maybe the nudge from Google will help there. Jabber is great but … hardly used by the people I chat with (who are split across MSN, Yahoo, AIM, Skype and IRC).

For a long time I’ve wanted to do RDF queries in a P2P context (eg. see book chapter I wrote with Rael Dornfest). Given Apple’s recent boost for Jabber, and now this from Google, the technology looks to have a healthy future. I want to try exposing desktop, laptop etc RDF collections (addressbooks, calendars, music, photos) directly as SPARQL endpoints exposed via Jabber. There will be some fiddly details, but the basic idea is that Jabber users (including Google and Apple customers) could have some way to expose aspects of their local data for query by their friends and FOAFs, without having to upload it all to some central Web site.

Next practical question: which Jabber software library to start hacking with? I was using Rich Kilmer’s Jabber4R but read that it wasn’t unmaintained, so wondering about switching to Perl or Python…

Return to LambdaMOO

                          ***************************
                          *  Welcome to LambdaMOO!  *
                          ***************************
PLEASE NOTE:
LambdaMOO is a new kind of society, where thousands of people 
voluntarily come together from all over the world.  What these 
people say or do may not always be to your liking; as when visiting
any international city, it is wise to be careful who you associate 
with and what you say. 

The operators of LambdaMOO have provided the materials for 
the buildings of this community, but are not responsible for 
what is said or done in them. 

It’s a long time since I went back to LambdaMOO. Experimenting with the visually lush Google Earth application this week reminded me of nothing more than my first explorations of LambdaMOO. Despite the visual differences and the passing years, both applications offer a virtual globe that can be collaboratively annotated and extended by users, both are a taste of things to come, and both leave a lot unsaid on the topic of Bristol. When Google engineers ponder where to go with KML (mappings to GML, inclusion of style and UI-related markup, etc), I’m sure they’ll be giving some thought to non-graphical interfaces to such data. LambdaMOO, to me, suggests that non-visual (including voice) interfaces could be every bit as compelling as a 3D flyover.

*** Connected ***

The Coat Closet
The closet is a dark, cramped space.  It appears  to be very crowded in here;
 you keep bumping into what feels like coats,  boots, and other people
 (apparently sleeping).  One useful thing that you've  discovered in your
 bumbling about is a metal doorknob set at waist level into  what might be a
 door.  Next to it is a spring lever labeled 'QUIET!'.
There is new news.  Type `news' to read all news or `news new' to read just
 new news.
Type `@tutorial' for an introduction to basic MOOing.  If you have not already
 done so, please type `help manners' and read the text carefully.  It outlines
 the community standard of conduct, which each player is expected to follow
 while in LambdaMOO.

open door

You open the closet door and leave the darkness for the living room, closing
 the door behind you so as not to wake the sleeping people inside.
The Living Room
It is very bright, open, and airy here, with large plate-glass windows looking
 southward over the pool to the gardens beyond.  On the north wall, there is a
 rough stonework fireplace.  The east and west walls are almost completely
 covered with large, well-stocked bookcases.  An exit in the northwest corner
 leads to the kitchen and, in a more northerly direction, to the entrance
 hall.  The door into the coat closet is at the north end of the east wall,
 and at the south end is a sliding glass door leading out onto a wooden deck.
 There are two sets of couches, one clustered around the fireplace and one
 with a view out the windows.
You see Welcome Poster, a fireplace, the living room couch, Helpful Person
 Finder, Cockatoo, The Birthday Machine, and lag meter here.
neural (dozing), lilakay (dozing), Evil (out on his feet), Fred_Smythe
 (dozing), and Ultraviolet_Guest are here.

north

The Entrance Hall
This small foyer is the hub of the currently-occupied portion of the house.
 To the north are the double doors forming the main entrance to the house.
 There is a mirror at about head height on the east wall, just to the right of
 a corridor leading off into the bedroom area.  The south wall is all rough
 stonework, the back of the living room fireplace; at the west end of the wall
 is the opening leading south into the living room and southwest into the
 kitchen.  Finally, to the west is an open archway leading into the dining
 room.
You see mirror at about head height, MOO population meter, Edgar the Footman,
 an antique suit of armour, and a globe here.

enter globe

You step into the globe...
Earth
A big blue-green planet.
This is Mapgrrl, Sparklebunny, Audrey, Boreal, Alista, tiny_ant, Zeddie,
 Pandemonium, mscope, Pasha, and entropygatherer's hometown.
Within Earth you see: Africa, Asia, Australia, Europe, North America, South
 America, and Antarctica.

enter europe

Europe, Earth
Small continent, many countries.
Within Europe you see: England, Italy, Scotland, France, Germany, Russia,
 belgium, Netherlands, Ireland, Norway, wales , Gibraltar, Sweden, Austria,
 Spain, Bulgaria, and Praha.

enter england

England, Europe
Heritage UK plc--purveyors of fine shortbread and Princess Di Memorial Plates
 to the rest of the Globe.
Within England you see: London, Kingston, Rainhill, bradford, Winchester,
 Derby, Knebworth, Stratford-upon-Avon, Oxford, Hereford, Newcastle,
 Southampton, Birmingham, Southend-on-Sea, Reading, Devon, Bristol, Watford,
 Rickmansworth, Croxley, and Warwickshire.

enter bristol

Bristol, England
This place isn't very interesting. Perhaps you should describe it, or go
 someplace more interesting.

leave

England, Europe
Heritage UK plc--purveyors of fine shortbread and Princess Di Memorial Plates
 to the rest of the Globe.
Within England you see: London, Kingston, Rainhill, bradford, Winchester,
 Derby, Knebworth, Stratford-upon-Avon, Oxford, Hereford, Newcastle,
 Southampton, Birmingham, Southend-on-Sea, Reading, Devon, Bristol, Watford,
 Rickmansworth, Croxley, and Warwickshire.

enter london

London, England
Earth has not anything to show more fair: dull would he be of soul who could
 pass by a sight so touching in its majesty...
Within London you see: Soho, Westminster Bridge, Camden, West Kensington,
 somerset house, Leytonstone, and Brixton.

enter Brixton

Brixton, London
Whatever you want, you find it here, mate...

GIS and Spatial Extensions with MySQL

GIS and Spatial Extensions with MySQL.

MySQL 4.1 introduces spatial functionality in MySQL. This article describes some of the uses of spatial extensions in a relational database, how it can be implemented in a relational database, what features are present in MySQL and some simple examples.

I’m hoping to understand the commonalities between this and PostGIS. PostGIS follows the OpenGIS “Simple Features Specification for SQL“. As do the MySQL extensions, apparently. The MySQL pages summarise the extensions as follows:

Data types. There needs to be data types to store the GIS information. This is best illustrated with an example, a POINT in a 2-dimensional system.

Operations. There must be additional operators to support the management of multi-dimensional objects, again, this is best illustrated with an example, a function that computes the AREA of a polygon of any shape.

The ability to input and output GIS data. To make systems interoperable, OGC has specified how contents of GIS objects are represented in binary and text format.

Indexing of spatial data. To use the different operators, some means of indexing of GIS data is needed, or in technical terms, spatial indexing.

I’m currently working on some ideas to prototype a new project (to fill the gap that the completion of SWAD-Europe leaves in my schedule). I’ll be revisiting my Gargonza plan to add a basic SemWeb RDF crawler to personal weblog installations, initially prototyping with Redland addons to WordPress. Ultimately, pure PHP would be better, unless Redland finds its way into the default PHP installation. Since WordPress requires MySQL anyway, it seems worth taking a look at these geo-related extensions. A more thorough investigation would take a look at reflecting GIS SQL concepts into RDF, perhaps exposing them in a SPARQL query environment. But that’s a bit ambitious for now.

What I hope to do for starters is use a blog as a personal SW crawler, scooping up RSS, FOAF, calendar, and photo descriptions from nearby Web sites. It isn’t clear yet exactly how photo metadata should most usefully be structured, but it is clear that we’ll find a way to harvest it into an RDF store. And if that metadata has mappable content, whether basic lat/long tags, richer GML, or something in between, we’ll harvest that too. My working hypothesis is that we’ll need something like MySQL spatial extensions or PostGIS to really make the most of that data, for eg. to expose location-specific, app-centric RSS, KML, etc. feeds such as those available from the flickr-derrived geobloggers.com and brainoff flickr.proxy sites. See mapufacture.com for one possible client app; Google Earth as KML browser is another.

That’s the plan anyway. So the reading list grows. Fortunately, OGC’s GIS SQL spec at least has some nice diagrams…

GIS datatype hierarchy

Profiling GML for RSS/Atom, RDF and Web developers

I spent some time yesterday talking with Ron Lake about GML, RDF, RSS and other acronyms. GML was originally an RDF application, and various RDFisms can still be seen in the design. I learned a fair bit about GML, and about its extensibility and profiling mechanisms.

We discussed some possibilities for sharing data between GML, RSS/Atom and RDF environments. In particular, two options: RDF inside GML; and RDFized GML.

The possibility of embedding islands of RDF inside GML (eg. the GML for a restaurant might use RDF for restaurant-review or menu markup) is interesting, as would allow GML documents to use any RDF vocabulary to describe the features on a map. Currently, such extension data typically requires the creation of a custom XML Schema. The other option, “RDFized GML”, is to explore the creation of an RDF vocabulary that allows some useful subset of GML data to be used in RDF. I’ll come back to this in a minute.

While GML comes from the world of professional GIS, its influence is being felt more widely: Google Earth (formerly Keyhole) uses something called KML, which bears a great many similarities with GML. Meanwhile in the RDF and RSS/Atom world, the very basic addition of “geo:lat” and “geo:long” tagging (sometimes using the W3C SemWeb IG WGS_84 namespace) has got a number of toolmakers interested. This year has seen the release of Yahoo! Maps, Google Maps, Google Earth and most recently Microsoft Virtual Earth. We’ve also seen the release of the excellent Mapping Hacks book, and increasing interest in this area from Web developers.

Although the experimental SWIG RDF vocabulary only deals with points described in WGS_84, there have been various discussions on possible extensions (eg. RDFGeom-2d from Chris Goad). These are intriguing, but we should be careful to avoid re-inventing wheels. Basically, I think we have all the ingredients for a hybrid approach: an RDFized GML subset designed for use by Web developers alongside RSS/Atom, FOAF and other public-facing XML formats. GML serves well as a data format in the GIS community, but some work is needed to find a subset that will find adoption in the wider Web.

The tiny W3C SWIG vocab, and related geo:lat/long tagging of “geo”-RSS feeds has shown that there is real interest in a lightweight XML-based mechanism for sharing map-related markup. GML shows us (via a 600 page specification, for GML 3.1) quite how rich and complex a problem space we’re facing, and KML demonstrates that a medium-sized “GML lite” subset can get traction with webmasters and developers, when backed by useful tools and services.

There are two pieces of work to do here (setting aside for now the topic of RDF islands within GML documents). Let’s first find a strawman profile of GML. From my limited knowledge and discussion with others, something “GML 2-ish” but profiled against GML 3.1, is the area to explore. Then we try getting those data structures into RDF, so it can mix freely with other information.

I understand from Ron Lake that profiling is something that is actively encouraged for GML, and there are even tools to support it that come with the spec: have a look at subsetutility.zip. These files (thanks Ron!) show a pretty easy path for experimentation with profiles. In addition to the schema subsetting utilities, the .zip also includes (just as an example to help me understand GML) an example application schema CommonObjects.xsd, showing how to define things like ‘Building’, ‘River’, and a sample instance .xml file that uses it.

To use the profiling tool, just put the unzipped files directly in the base/ directory of .xsd files that ships with GML 3.1, then run an XSLT processor to generate a GML subset.

xsltproc depends.xsl gml.xsd > _gml.dep

xsltproc GML3.1.1Subset.xsl _gml.dep > _gmlSubset.xsd

…and that’s your profile. The scripts take care of all the dependencies (ie. they’ll read the 29 XML Schemas, so you don’t have to :)

The bits of GML you want are specified as parameters in GML3.1.1Subset.xsl. The default in this .zip is: gml:Point, gml:LineString, gml:Polygon, gml:LinearRing, gml:Observation, gml:TimeInstant, gml:TimePeriod

I’m no GML expert, but if someone can help get some instance data matching such a profile, I’ll have a go at RDFizing it. Also, of course, it will be useful to debate how many facilities from full GML would find use in the Webmaster (RSS, KML etc) scene.

Disclaimer: for now this is purely an informal collaboration. If we make something interesting, it might be worth investigation of something more formal between W3C (home of RDF, and where I work) and OGC (home of GML). For now, let’s just try out some ideas…

geobloggers: “Network Link” in Google Earth

This is the hidden gem of Google Earth. Adding a “Network Link” allows you to fetch KML data from remote servers. It does this in two ways, Time Based or Location Based. So *anyone* can add dynamic data to Google Maps.

Apparently KML is based on GML. I don’t know Keyhole/Google’s work differs. There seems to be a role here for something simple enough for Google Earth, WorldWind, and other viewer apps to use, when consulting a remote server for info about some area of interest. Maybe it’s GML Web Feature Servers, maybe KML, maybe geo-extended RSS/Atom, or perhaps generic query interfaces like the SPARQL protocol. SOAP, WSDL and REST fit in the picture somewhere. Probably, various things will be used in different environments, depending on application emphasis. We might be looking up the opening-hours of a shop, contact information for an organization, or jobs, events, photos, blog posts, FOAF profiles etc in a certain area, … it isn’t clear where the line is drawn between ‘geographic’ data and the wider unbounded collection of information about the world. GML has strengths at the geographical end of the spectrum, RDF (and its query system, SPARQL) has strengths at the generic, domain-neutral end. RSS/Atom is serving well as a generic carrier for data syndication. It isn’t clear to me yet where KML fits (or SVG, for that matter), but work on the relationship between GML and RDF would seem timely.

The geobloggers post has examples and links to flickr and del.icio.us-based services that expose this interface. I’m going to try making such a service on top of SPARQL…

geo/photo raw materials from a walk around Bristol

I’ve just been for a walk around town, taking a few photos on the way, with the camera’s clock sync’d to that of my new GPS. I’ve not yet figured out the best tool to use (other than write my own) for hooking up the photo and location metadata. It’s easy enough to get the data off of the device using GPSBabel; for now I’ve saved tracks and waypoints in GPX format. GPSBabel offers a wealth of other options, none of which I’ve explored yet.

The first thing I had to hand that consumed GPX was Google Earth on my WinXP laptop; it plotted the tracks happily enough:

2004-07-24 walk around town, plotted via google earth

I’m sure there are other, more open-data and hackable systems out there that I can feed this data to, but for a quick test, Google Earth (formerly ‘Keyhole’) is quite fun.

For Windows users, there are WorldWind scripts that convert GPX into World Wind Layers, available as an online service. I’ve just tried that; it produced me XML document that references a PNG image. I need to read the instructions again a bit more carefully, but nice to see that there’s more to the world than Google. There are also some notes on the OpenStreetMap site on MakingGpxTracks.

The next thing I should try is getting the location and photo metadata integrated, and then I can play around with associating images on maps. There is also a growing geo-tagger community using Flickr (by creating odd ‘tags’ that embed lat/long data). For example see various of jip’s photos, and the geobloggers website (which seems to use HTTP REFERER to plot geotagged flickr images with Google maps).

Update: as I suspected, the Locative guys have already done the work :)

After CPAN-installing the Perl dependencies Image::EXIF and RDF::Simple::Serialiser, Time::Piece, LWP::UserAgent, XML::Simple, Data::Compare and Geo::Track::Log, it looks like geoloc_media.pl is just what I’m after (thanks!).

geoloc_media.pl 0 http://rdfweb.org/people/danbri/media/2005/07/ ../../mydata/2005-07-24-Tracks.gpx    danbri@foaf-project.org 2004-07-24-BristolWalk/*jpg

The resulting RDF looks about right, from a quick look.

Here’s a quick query of the data using the online Redland SPARQL service. The query I ran was as follows (basically echoing back the core data structure emitted by the perl script above):

PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX locative: <http://locative.net/2004/packet#>

SELECT DISTINCT ?image, ?lat, ?long, ?email 
WHERE {
  ?x rdf:type locative:Packet .
  ?x locative:media ?image .
  ?x geo:lat ?lat .
  ?x geo:long ?long .
  ?image foaf:maker ?who .
  ?who foaf:mbox ?email . 
}
ORDER BY ?lat

See the Locative wiki for more uses of the data, including some nice SVG hacks. Or the OpenStreetMap view of the area (seems to be bitmap imagery there but no streets etc yet).

Conspiracy Theory?

There are two kinds of people in the world, those who believe there are two kinds of people in the world and those who don’t.

–Robert Benchley, Benchley’s Law of Distinction

From a recent Guardian article, “Police scrutinise extremist Islamist websites“:

Among the sites causing concern is Jihadunspun (JUS), a highly professional website which claims to present “a clear view of war on terror”. It has been widely criticised in the US by agencies including the state department.

Nearby in the Web, Google News carry the site, causing much grumbling amongst those who’ve noticed its iffy content.

Digging around a bit more unearthed a bit of controversy around the site, specifically, claims that it is a CIA front. For example, see commentary on the Wisdom Fund site, or Information Clearing House. Both of which draw attention to the Canadian hosting of the site, its use of PayPal to sell video content, etc.

Who to believe? :)

Rummaging on the JUS site itself, there’s a lengthy and peculiar story from its “owner and publisher”:

My name is Khadija Abdul Qahaar and I am the owner and publisher of Jihad Unspun. I am Muslim, who like many others, embraced Islam as the result of 911. Many of our viewers will know me as Bev Kennedy and Paul Morris-Read, the pen names I have used on this portal, a necessary precaution particularly in the early days of publishing this portal, due to the nature of our work. [...]

Most intriguingly…

For the curious, a copy of my CV is located here.

That article mentions “Bev’s” prior work in the Web industry, and her work for the “aerospace market”. The CV includes 1999-2003 as president/director/shareholder of DataCrafters Inc. Their website is now blank, but much of it is available through the Internet Archive. The Client Kudos section is interesting, as is the Before and after page, which features happy customer Royell Manufacturing, makers of aerospace parts. Back in the pre-9/11 copy of the site, the “spinbin” section had a press release announcing the release of the “eSYS Online Estimating System”, including a glowing quote from a Robin Crabb of Aeroalliance, (whose website is full of pictures of scary military jets):

“We are delighted that eSYS is now in beta testing. With feedback from high caliber Aeroalliance members like Royell, we are confident that we will have captured the ease of use and familiarity of process that aerospace manufacturers find appealing.”

The company seems to have had a lot of dealings with the military-tinged aerospace industry, before Bev switched to working on JUS. Bev’s autobiographical note on the JUS goes into some detail describing her change of heart, as – if you believe twf.org – does this lengthy letter, which also discusses the “CIA front” accusation. Whatever the truth, it’d make for a good movie.

I could go on, there’s plenty more out there on the site. It’s hard to know really what the Guardian guys should’ve done. Ten minutes of digging around the background to DataCrafters does suggest a remarkable change of heart, and one that’s not documented heavily on the JUS site.

Is the “this is a CIA front” accusation purely an exercise in tin-foil hattery? Who knows!

Another quote from the JUS site, puffing up the warmongering ‘clash of civilizations’ agenda…

I named the portal as a reference to Benjamin Barber’s best seller, Jihad vs. McWorld, an excellent work that explains the clash of two civilizations, but that entirely omits the reason why. The result was Jihad Unspun.

Update: there’s a huge and often silly, offensive etc thread on the libertyforum.org site. See also jihadunspun unhinged on the Internet Haganah site; they seems to take JUS seriously and quote from Bev’s JUS-hosted story:

Late in 2002, the Jewish vigilante groups struck with a vengeance. Through a stupid error on our part, we had a portfolio gallery showing off some of our client work on our web company’s website and after putting two and two together, they managed to access some of our client list. They of course went after the aerospace companies and I was forced to sell of my aerospace application at a significant loss and eventually I had shut down the company entirely in order to protect my clients.

I wonder what happens next…