Organizing an index organically

Classifying content in Drupal requires creating several taxonomies of various types. One of the useful types for rapid and informal classification is free tagging, where arbitrary, unrelated keywords can be attached to nodes. It is more general than, say, the index we find at the end of a book, where the terms are explicitly found in the text. but I'm getting ahead of myself: let's start at the beginning.

Now an organic infohacker starting to organize his information on Drupal will likely begin with one tree-like, hierarchical index dimension, which represents his initial classification of the future content of the system. In many cases, the classification will be topical, covering the areas of interest of the user, to the required level of detail corresponding to his own depth and specialization of knowledge.

The aim of this exercise, it is worth to remember, is to be able to retrieve content (by querying the system) faster and more precisely. No indexing at all would require a full-text search every time, and would cause much information to become inaccessible or lost on the medium or long term.

As the infohacker enters content and attempts to classify it, he will keep refining this one-dimensional structure to best support his content on one hand, and to best represent his own implicit understanding of the structure of the topic at hand, on the second. This will mainly adding new terms at various levels of the hierarchy, with more specialized terms towards the leaves, and sometimes pruning whole branches. This operation is still tedious in Drupal because the terms are simply dropped but the user is not given the opportunity to reorganize the content that is relying on those terms. It would be worth affording a user interface to rearrange the index terms of the affected nodes, either individually or in bulk. A view with form elements in front of each node (e.g. the terms) and bulk operation checkboxes would do it.

At some point, the user might feel that some content requires additional terms that are not easily integrated into the static structure being built. Rather, arbitrary terms are required, corresponding to points in the total indexing space, which is of course an infinite-dimensional, semantic and hierarchical space (much like R^n). This index is therefore doomed to always represent 0% of the potential content, since the index is infinite many times over. Nevertheless, it can momentarily capture some clusters of information that are of practical use to the user. We can now visualize the current index in use: a one dimensional tree surrounded by a cloud of points, with possibly some correlation between the positions of the two.

With time, some points in the free tag cloud will gravitate around a specific branch or term of the tree. It would be useful at that point to afford the ability to "solidify" those free tags into index terms in the tree hierarchy. This operation will help in further refining the index.