Tutorial: TM Semantic Visualisation (Part IV)

(continued from Part III)

Last time I left off with showing you how individual documents would be blended into a landscape which is computed from a topic map.

http://kill.devc.at/system/files/synth-hover-small.jpg

So far I have ignored the topological structure of the topic map itself and computed the landscape only from the terms within the documents. But my ultimate goal is/was to visualize whole topic maps, not just the text corpus.

Topic Maps Impact

For that I compute a measure how strongly a particular document (inline data or content within a URL-referenced document) is attached to a topic. Via occurrences and associations not only the closest topic, but also all neighboring topics have so an impact. And all that is weighted by the entropical value an association has within one map.

This measure then I use to strengthen the topic names in the convergence process.

To see how this works let us return to the bipolar topic map, which only contained the topics AAA and NNN. Both have documents attached, but they are completely disjoint (although identical in internal topology).

http://kill.devc.at/system/files/bipolar-doc-small.jpg

When only looking at these documents, then the highest intensities are at WWW and JJJ, respectively. As there is no overlap in terminology, the mountain ranges are disjoint. But if you look closer you see strong similarities between the two: YYY corresponds to LLL, OOO to BBB, XXX to KKK, etc.

Now let's turn on the topic map impact:

http://kill.devc.at/system/files/bipolar-doc-tm-small.jpg

Having pushed up the topic names, they now dominate large areas. Only the originally strong JJJ/WWW or XXX/KKK hot spots survived the change. Underneath the surface the internal topology remained mostly the same, although it sharpened a bit, aggregating the documents into a somewhat closer area.

2nd Experiment

The second topic map I experiment with is a variation of the first, namely with some common documents shared between the two topics. At some stage a document-only rendering looks like this:

http://kill.devc.at/system/files/bipolar-linked-doc-small.jpg

You will notice some small elevations around RRR and AAA, both of which are created by shared documents being quite intense there. The rest is the usual two-continent landscape.

Now let us turn on the topic map impact:

http://kill.devc.at/system/files/bipolar-linked-doc-tm-small.jpg

Again the overall structure remains the same, and again you can see how the topics not only dominate the picture, but also that they gently aggregate the documents, deepening the divides.

One can even increase this effect by turning on topic mappishness from the very beginning:

http://kill.devc.at/system/files/bipolar-linked-tm-small.jpg

The reason for this to work is that the network is more amenable to learning when it is young and less biased (go figure).

And So What?

For what it's worth, I have proven to myself that one can visualize a topic map, respecting both, the topological aspect in the map, and the text information from the involved documents.

Of course there is much to say about the practical value of this.

AttachmentSize
bipolar-doc.png375.48 KB
bipolar-doc-small.jpg31.3 KB
bipolar-doc-tm.png335.46 KB
bipolar-doc-tm-small.jpg29.34 KB
bipolar-linked-doc.png480.51 KB
bipolar-linked-doc-small.jpg36.36 KB
bipolar-linked-doc-tm.png448.34 KB
bipolar-linked-doc-tm-small.jpg34.38 KB
bipolar-linked-tm.png452.22 KB
bipolar-linked-tm-small.jpg34.21 KB
Posted In

For What It's Worth?

One use that occurs to me is that once we move away from hand authored topic maps and start to see large scale aggregation topic maps, such visualizing techniques will be important for recognition of patterns in data by human observers. (Pattern recognition remaining a difficult subject for our inbred digital cousins.)

Exploration could be particularly effective if an interface offered interactive selection of dimensions for re-casting of the map as the user explores it. Along with the ability to "save" or "bookmark" views so that they could be overlays on other views or perhaps displayed orthogonally to them.

Within the limits of hard copy technology it is understandable that maps, for the most part, are presentations of a fixed point of view. Topic maps have the potential to move beyond that limitation, assuming we have the will to make it so.

Hope you are having a great day!

Patrick

Patrick Durusau (not verified) | Wed, 11/04/2009 - 17:10

Re: For What It's Worth?

One use that occurs to me is that once we move away from hand authored topic maps and start to see large scale aggregation topic maps, such visualizing techniques will be important for recognition of patterns in data by human observers.

Right. But even with large hand-written maps it is amazing what you find how they look in this rendering.

Exploration could be particularly effective if an interface offered interactive selection of dimensions for re-casting of the map as the user explores it.

Such interface is indeed in the making (albeit slow progress). But as the dimensions are complex to control such a feature would only for the map creator, rather not for the end user. A stable map, i.e. one which looks always the same is paramount.

rho | Thu, 11/05/2009 - 08:53

Only for map creator?

Robert,

Sorry for the delayed comment: Why "..such a feature would only be for the map creator, rather not for the end user."?

I recently finished the edits on data pilot tables in ODF, the prime purpose of which is to take structured data and to organize it differently than its source. To permit different analysis of the data.

Granting that for processing reasons it might be useful to restrict the dimensions that can be selected or who can select them but I am not getting the rationale for restricting such a feature to the map owner.

I suppose that may be my prejudice against authorities who want to impose a "correct" view, whether than be a Bible translation, a data store or even a topic map. Some "views" are more useful than others, for some purposes, according to some individuals. Beyond that I am really not willing to go.

I suppose a "stable" map would be useful, much like one of the mapping services that starts us all off with some given view of an area. But that is merely a canvas that we can then zoom in or out and place other markers onto.

Or do you have a client that "knows" the true map that should be displayed? That makes a difference, deliver the "truth" as they see it. Cha-ching!

Patrick

Patrick Durusau (not verified) | Tue, 11/17/2009 - 17:54

Re: Only for map creator?

Sorry for the delayed comment: Why "..such a feature would only be for the map creator, rather not for the end user."?

I certainly agree that maps will be useful for "end users". I hope I have not written anything to the contrary.

I suppose that may be my prejudice against authorities who want to impose a "correct" view, whether than be a Bible translation, a data store or even a topic map. Some "views" are more useful than others, for some purposes, according to some individuals. Beyond that I am really not willing to go.

Having views (and different resolutions and contortional displays) are on my agenda. But they will have to wait for another funding round. Or me loosing interest, whatever comes first :-)

I suppose a "stable" map would be useful, much like one of the mapping services that starts us all off with some given view of an area.

Stability is a primary objective. Having the landscape rotating, shifting, warping is incredibly annoying.

Or do you have a client that "knows" the true map that should be displayed? That makes a difference, deliver the "truth" as they see it. Cha-ching!

Well, my hidden, insidious and satanistic agenda is to enable everyone to develop his/her own truth.

rho | Wed, 11/18/2009 - 11:40