Imagemap magic

I’ve always found HTML imagemaps to be a curiously neglected technology. They seem somehow to evoke the Web of the mid-to-late 90s, to be terribly ‘1.0’. But there’s glue in the old horse yet…

A client-side HTML imagemap lets you associate links (and via Javascript, behaviour) with regions of an image. As such, they’re a form of image metadata that can have applications including image search, Web accessibility and social networking. They’re also a poor cousin to the Web’s new vector image format, SVG. This morning I dug out some old work on this (much of which from Max, Libby, Jim all of whom btw are currently working at Joost; as am I, albeit part-time).

The first hurdle you hit when you want to play with HTML imagemaps is finding an editor that produces them. The fact that my blog post asking for MacOSX HTML imagemap editors is now top Google hit for “MacOSX HTML imagemap” pretty much says it all. Eventually I found (and paid for) one called YokMak that seems OK.

So the first experiment here, was to take a picture (of me) and make a simple HTML imagemap.

danbri being imagemapped

As a step towards treating this as re-usable metadata, here’s imagemap2svg.xslt from Max back in 2002. The results of running it with xsltproc are online: _output.svg (you need an SVG-happy browser). Firefox, Safari and Opera seem more or less happy with it (ie. they show the selected area against a pink background). This shows that imagemap data can be freed from the clutches of HTML, and repurposed. You can do similar things server-side using Apache Batik, a Java SVG toolkit. There are still a few 2002 examples floating around, showing how bits of the image can be described in RDF that includes imagemap info, and then manipulated using SVG tools driven from metadata.

Once we have this ability to pick out a region of an image (eg. photo) and tag it, it opens up a few fun directions. In the FOAF scene a few years ago, we had fun using RDF to tag image region parts with information about the things they depicted. But we didn’t really get into questions of surface-syntax, ie. how to maker rich claims about the image area directly within the HTML markup. These days, some combination of RDFa or microformats would probably be the thing to use (or perhaps GRDDL). I’ve sent mail to the RDFa group looking for help with this (see that message for various further related-work links too).

Specifically, I’d love to have some clean HTML markup that said, not just “this area of the photo is associated with the URI http://danbri.org/”, but “this area is the Person whose openid is danbri.org, … and this area depicts the thing that is the primary topic of http://en.wikipedia.org/wiki/Eiffel_Tower”. If we had this, I think we’d have some nice tools for finding images, for explaining images to people who can’t see them, and for connecting people and social networks through codepiction.

Codepiction

HTML Imagemap authoring tool for MacOSX?


I’ve searched around in vain for one. I want to annotate my FOAF spec diagram with mouseover text and links into the documentation. Most of the tools I find are a decade or more old, or pay-to-play. I remember the Gimp image editor can do imagemaps, but it crashes on startup on my MacBook. I did find a nice Javascript editor the other day, but it had no “undo” function, which made complex work impossible. Maybe I’ll try Amaya. Suggestions very much welcomed! Are imagemaps *that* uncool these days? They’re just SVG and image metadata in another notation… (and imho one of the the more interesting scenarios for HTML-based microformattery). That last link has missing photos and out of date SVG, but might still be of interest. A surviving screenshot included here.

Nearby: (from SWAD-Europe hacking days) an imagemap2svg.xslt thanks to Max Froumentin.

Update: I installed Amaya 9.99 for Intel MacOSX. Sadly it couldn’t even display my JPEG properly, although it did do a better job showing OmniGraffle’s SVG output than Firefox.


FOAF diagram (day 2)


FOAF diagram (day 2)
Originally uploaded by danbri

Another revision, after feedback from Ivan.

The original had “Thing” in italics (a convention I tried before adding in doap: dc: and sioc: references), to indicate it was from another namespace. I’ve now made that heritage explicit (although I suspect it might confuse, the idea is pretty central so worth explaining).

Originally I had both “tipjar” and “mbox” drawn as if they were literal properties, when both are relational. The later layout I’m using allows tipjar to be drawn without crossovers, so now only “mbox” is an oddity. I’ve used italics as a hint of that, but without explicit explanation.

A lot of the redundancy in the diagram comes from the property inverses, but I want to leave them in since they’re important to explain. The expanded key in the bottom-left now has a nerdy little explanation of “inverse property”. I also use the word “subClass” explicitly on the fat arrow, to tap into an OO heritage that might be floating around in people’s minds. In other words, I want people to realise that “a Person is an Agent is a Spatial thing is a thing” is what we’re getting at with those fat little arrows.

On Ivan’s suggestion I trimmed the property lists a little. Removing the non-jabber IM properties, and first/last name. The space earned from the latter was immediately spent by adding in bio:olb since it’s useful and I’ve the impression it’s widely used.

I think that’s about it for changes. Ivan suggested removing the DOAP and SIOC partys, but to my mind they are important because those vocabularies (and projects) elaborate on parts of FOAF which are important but otherwise neglected: the description of Projects, and of Users.

OK one more version. This has coloured blogs for the core FOAF classes. I think I made a mistake choosing light blue, since that is the colour code I’ve assigned to mean “inverse functional property”, nearly. Perhaps a light yellow?

Anyway here it is for now. Guess I should stop using Flickr for this really!

with colours

Update: I’ve put the files in svn (original OmniGraffle XML is the .graffle file; also some SVG output although I’m unsure the quality), and made another slight revision, this time focussing on the layout of the ends arrows for better readability; previously they were cluttered and the property directions were therefore hard to read.


FOAF spec diagram


experimental foafspec diagram
Originally uploaded by danbri

I’ve been trying to capture the core terms of FOAF in a diagram. Here’s version two.

Notes: there are a few terms I’ve missed out, to avoid massive clutter: workInfoHomepage, geekcode, myersBriggs, currentProject, pastProject, dnaChecksum, membershipClass, sha1, fundedBy, theme, Online*XyzAccount, logo, phone. These are all underspecified, underused, or boring. Or all three :) The properties mbox and tipjar are the main quirks; they’re drawn like string-valued properties but are really relations. Some of the non-FOAF namespaces are done the same way too. Maybe it needs a footnote or notation for those?

I’ve included a few nods towards external namespaces; obviously these aren’t supposed to be complete.

Even this kind of diagram is a bit meta; there should be a companion diagram with instance level data using these terms. Work in progress. Suggestions and feedback would be much appreciated.

Symbol languages and the Semantic Web

OK so I just stumbled upon this…

Bomb in Baghdad

…via Jonathan Chetwynd’s ever-inventive and SVG-happy Peepo.com.

The “Car Bomb in Baghdad” story is from a site created by Widgit Software, and explains itself as follows:

Symbolworld has been set up to provide a web site with material suitable for symbol readers of all ages. The internet is an important medium which many people really like to use. Sadly there is very little material that is appropriate or accessible by people with learning difficulties.

The copyright statement for Symbolworld says “symbols on this page are copyright of the commercial owners. They may not be copied or used in any other format without the written permission from the owner.“, which initially struck me as a potential challenge to any use of this particular symbol-set for online communication. But I don’t really know this scene, and I guess this copyright could be just the same as the way eg. fonts are copyrighted.

So I don’t know much about this particular project/company/product, but it reminded me of some similar work I heard about a few years ago. Back when the EU, in their occasionally infinite wisdom, funded SWAD-Europe to run around talking to interesting people about standards and the Semantic Web (and giving them t-shirts) , Chaals organised a great developer workshop in Madrid on Image annotation. We had the usual fun with SVG and RDF (which btw I’m still betting on) and I got to learn a bit about CCF. Seeing the Baghdad example this morning reminded me of all this. I’ve been clicking around and trying to gather my sprawling thoughts.

CCF, the Concept Coding Framework, is kinda image annotation in reverse. Instead of focussing on the description of the content, concepts etc associated with images, the emphasis is on the use of images to illustrate some enumerated set of concepts. From Chaals’ workshop report re outcomes, I’m reminded that we discussed…

How to use Creative Commons and similar vocabularies to determine whether a particular symbol can be freely used (typically in commercial systems the symbols themselves are proprietary, which can be a major barrier to communication between people who have different systems).

CCF was using some variant of SKOS (another SWAD-Europe activity). This found its way into SKOS itself, where we now have prefSymbol and altSymbol relationships that associate a skos:Concept with a dcmitype:Image. Borrowing an illustration from the SKOS Core guide here:

skos symbol diagram

The guide also notes a distinction between symbolic labelling and “depiction” in the FOAF sense; some symbols are purely symbolic, and have no representational content.

So, catching up with this area of work a little, I find the Bliss Symbolics, the WWAAC project final report, and various other accessibility-related efforts. But I’ve not really figured out where I’d start if I wanted to build something simple using a freely available symbol-set, nor what the main options/projects currently are. But there’s plenty of reading, including pages from a recent Bliss “think tank” meeting.

The latest I can find on CCF is that it has moved sites and that there are some Web interfaces available, but “sorry – there are no downloads for this project yet.”. Ah, apparently the SYMBERED project is continuing development of CCF (aside: Bliss with swedish translation; doubly incomprehensible to me!). There is a nice example on their site showing the multilingual aspect to this work, as well as contrasting Bliss with a more representationally-oriented symbol set; see their site for details.

Here’s a simple example just in English:

I want coffee and milk and cookies.

In case anyone thinks this whole exploration is a bit niche and obscure, take a look at how people use MSN and other IM systems sometime. And of course the Jabber/XMPP guys have been exploring specs for standard emoticons. Chaals also points out some connections to VoiceXML where “there are a handful of options available in an interaction designed to be through voice, and developers will define assorted ways of recognising from a user’s speech which of the relevant concepts is being matched.”

We’re also only a few clicks away from the survey conducted by the dreamily named W3C Emotion incubator group into use cases and technologies for the description of emotion using markup languages.

If an RDF-ized Wordnet is also thrown into the mix, assigning URIs to Synsets, WordSenses, Words, I think we might actually be getting somewhere. The version of Wordnet in RDF published at W3C doesn’t currently use SKOS, although this was discussed, and of course there are other representations that make more use of RDF class hierarchies for nouns (at the expense of linguistic lossyness and losing non-noun content). Princeton’s original English-language Wordnet has spawned many related projects and translations, but as far as I know, there is little integration amongst them, and not all of the data is public or freely re-usable.

I once had a silly dream of taking a photo to go with each and every noun term in Wordnet. A kind of SemWeb I-Spy, a cousin to Immuexa’s FOAF bingo. Or better, of doing that with friends. The rise of Flickr and tagging means that we’d probably do this now by aggregate using Flickr and similar sites. But it seems conceivable to me that such an “illustrated wordnet” could be made, using either photo-oriented or symbol-oriented illustrations.

OK what might that buy us? Let’s try these two samples.

Not perfect, … but imho Wordnet gives a nice set of common and identifiable concepts that can be used as a hub for all kinds of different projects. And all we’d need is a huge pile of shared data shaped like this:

wordnet:word-cookie skos.prefLabel wikicommons:Explosion.svg .

OK it’s not going to bring about world peace and the return of esperanto, and of course there’s much more to language and communication than nouns and verbs (the easiest part of wordnet to turn into visual symbols), but it does strike me as a fun little (big) project…. Wordnet is too huge to be useful in every context where we’d want a modest-sized symbol set (eg. IM emoticons), … but it is nicely searchable, and would provide a framework for such subsets to evolve and be interconnected.

Google Earth touring via KML

While I’m writing up old hacks, here’s one that I really enjoyed, even if it was a bit clunky. A couple of years ago Mikel Maron implemented (on my urging in irc.oftc.net #geo IRC :) a PHP-based Google Earth touring service, which interconnects a “tour guide” user with “tourists”.

This site facilitates collaborative, realtime exploration of Google Earth. As the “tour guide” navigates, “tourists” will automatically follow along.

When the tour guide’s Google Earth installation is at rest, a specially installed KML network link sends the server an HTTP request, showing the coordinates of the visible area of the globe. This same service is periodically polled (every second) by “tourists” whose Google Earth will dutifully fly to the appropriate spot.

The system seems to be offline currently, but was quite evocative to use, even if tricky. You never quite knew what the other party could actually see, since the picture can load quite slowly when moving around a lot. And the implementation didn’t do anything about angle of view (although this became possible in later versions of KML). I had experimental tours of Dublin guided by Ina (Skyping at same time), and of various places in Iran by Hamed Saber.

I expect in due course (if not already, I don’t track these things) Google Earth and similar products (Worldwind, or the Microsoft thingy) will offer social map browsing, it’s such a nice feature, though it really needs an audio channel open at the same time. Last week I tried to do the same without such a link, my mum talking me thru finding a small village in France. Much harder! “Take the road north out of Chabanais … past a small farm, past the swimming pool…”.

Here is Mikel’s “how it works” writeup:

The web interface generates KML files, which are loaded into Google Earth and create Network Links. The tour guide has a “View Based Refresh” Network Link, which sends the bounding box of the current view to the specified URL whenever the camera stops. That position is stored. Tourists receive a “Time Based Refresh” Network Link, which requests every 10 seconds and receives the last stored position of the guide.

Right now only location and altitude are transmitted. A future release of Google Earth may enable tilt and rotation. Integrated chat would be nice as well.

The fact that they’ve hidden a full flight simulator within Google Earth might make this worth revisiting. And of course there is infinite fun to be had from playing with photos etc on the globe, although my last attempts in that direction (preparing for 3 months in Buenos Aires by studying geo-tagged photos instead of Spanish) tailed off. Everyone was putting pics on maps, I got a bit bored, even though it’s still a worthwhile area with much still to be done.

Some ideas are not meant to be combined though: who really needs a collaborative realtime photo-navigator implemented with Google Earth flight simulator? :)

Flickr’d

Just renewed my Flickr-Pro account for 2 years, ensuring an irregular supply of pigeon, fish and other misc depictions.

I wasn’t 100% happy with the wording of their terms though.

To participate in Flickr pro, you must have a valid Yahoo! ID and, solely if you have not received a free offer or gift for a specific number of days of Flickr pro (“Free pro Period”), you will also need to provide other information, such as your credit card and billing information (your “Registration Data”). If you do not have a Yahoo! ID, you will be prompted to complete the registration process for it before you can register for Flickr pro. In consideration of your use of Flickr pro, you agree to: (a) provide true, accurate, current and complete information about yourself and (b) maintain and promptly update the Registration Data to keep it true, accurate, current and complete. If you provide any information that is untrue, inaccurate, not current or incomplete, or Flickr has reasonable grounds to suspect that such information is untrue, inaccurate, not current or incomplete, Flickr has the right to suspend or terminate your account and delete any information or content therein without liability to Flickr.

The “provide true, accurate, current and complete information about yourself” is only contextually limited to “credit card” and “billing information”; it could also plausibly be read as covering the more general Flickr user profile, on which I’ve every right to omit various bits of information (Missing isn’t broken). The billing system also let me have the choice of storing credit card info or re-entering it again next time it’s used. So it isn’t really clear what they’re asking for here. If my buddy icon doesn’t show enough grey hair, is that inaccurate? :) I guess they’re really focussed on contact details, in which case, it’s best to say so.

I signed up anyways. The Flickr API and the RDF-oriented Perl backup library make it a more reliable option for my photos than my own little Ruby scripts ever were. Back 2-3 years ago I maintained the fantasy that I’d manage my own photos and their metadata; the big reason I switched to Flickr was the commenting/social side. It’s just too hard for per-person sites to maintain that level of interactivity and community (unless you’re super famous or beautiful or both). And a photo site without comments and community, for me, is kind of boring. For decentralists … perhaps some combination of extended RSS feed plus OpenID for comments could come closer these days; but before OpenID, I couldn’t ever see a way for commenting and annotation to be massmarket-friendly in a decentralised manner. And, well, also no need to be grudging: Flickr is a great product. I’ve definitely had my money’s worth…

Bottles and teargas in Piazza Navona

Bottle thrower and teargas

I’ve just posted 100 or so photos from the anti-Bush protest in Rome last night. Zac and I were on the way to a restaurant when we stumbled into the protest in Piazza Navona, just by the Senate where Bush apparently was in meetings. We wandered around the square taking photos of the protesters, then suddenly the mood changed from carnival to street fighting at the southern end of the square.

Anti-Bush crowds watching protesters fight police

I tried geo-tagging some of the photos (see flickr map) but it is hard to be precise: I don’t know the square well, my memory isn’t perfect, and the tool doesn’t allow me to be clear about position, direction or subject of the image. But it helps. I didn’t see the fighting start, but I ran to the front to see what teargas tasted like (“sort of like lemon sherbet, with an initial zing then a slow, well-rounded plummy finish”; but doesn’t sit well with mild asthma), then hung around as the square was eventually cleared.

Shield & Sky

Main thing I take away from the experience: nothing says “body language” like street-fighting Italians…

GIS and Spatial Extensions with MySQL

GIS and Spatial Extensions with MySQL.

MySQL 4.1 introduces spatial functionality in MySQL. This article describes some of the uses of spatial extensions in a relational database, how it can be implemented in a relational database, what features are present in MySQL and some simple examples.

I’m hoping to understand the commonalities between this and PostGIS. PostGIS follows the OpenGIS “Simple Features Specification for SQL“. As do the MySQL extensions, apparently. The MySQL pages summarise the extensions as follows:

Data types. There needs to be data types to store the GIS information. This is best illustrated with an example, a POINT in a 2-dimensional system.

Operations. There must be additional operators to support the management of multi-dimensional objects, again, this is best illustrated with an example, a function that computes the AREA of a polygon of any shape.

The ability to input and output GIS data. To make systems interoperable, OGC has specified how contents of GIS objects are represented in binary and text format.

Indexing of spatial data. To use the different operators, some means of indexing of GIS data is needed, or in technical terms, spatial indexing.

I’m currently working on some ideas to prototype a new project (to fill the gap that the completion of SWAD-Europe leaves in my schedule). I’ll be revisiting my Gargonza plan to add a basic SemWeb RDF crawler to personal weblog installations, initially prototyping with Redland addons to WordPress. Ultimately, pure PHP would be better, unless Redland finds its way into the default PHP installation. Since WordPress requires MySQL anyway, it seems worth taking a look at these geo-related extensions. A more thorough investigation would take a look at reflecting GIS SQL concepts into RDF, perhaps exposing them in a SPARQL query environment. But that’s a bit ambitious for now.

What I hope to do for starters is use a blog as a personal SW crawler, scooping up RSS, FOAF, calendar, and photo descriptions from nearby Web sites. It isn’t clear yet exactly how photo metadata should most usefully be structured, but it is clear that we’ll find a way to harvest it into an RDF store. And if that metadata has mappable content, whether basic lat/long tags, richer GML, or something in between, we’ll harvest that too. My working hypothesis is that we’ll need something like MySQL spatial extensions or PostGIS to really make the most of that data, for eg. to expose location-specific, app-centric RSS, KML, etc. feeds such as those available from the flickr-derrived geobloggers.com and brainoff flickr.proxy sites. See mapufacture.com for one possible client app; Google Earth as KML browser is another.

That’s the plan anyway. So the reading list grows. Fortunately, OGC’s GIS SQL spec at least has some nice diagrams…

GIS datatype hierarchy