Tech Update Software Infrastructure

 Add your opinion

Forward in

Format for

Mining legacy data presents three challenges: determining where those bits are stored, what stored them there (for example, e-mail clients and servers and their largely proprietary repositories), and how (format, security, etc) they're stored. The amount of corporate data and knowledge that is resting on PC's hard drives (where it's virtually undiscoverable and inaccessible) and that could be beneficial to other users, accounts for the vast majority of an organization's information assets. The remaining information, which is found on servers, may be more accessible, but is still just as difficult to relate and mine.

So difficult is relating, mining, and even replicating existing information to other repositories that Microsoft is creating a new file system (WinFS) for Windows that promises not only to divorce data from the applications that create it, but also provides a metadata framework for speeding the information's discovery, retrieval, and marriage to other related information. WinFS, which is based on Microsoft's next version of SQL Server (code-named Yukon), is expected to make its appearance in the next version of Windows (code-named Longhorn), which isn't do until 2006 at the earliest. So far, metadata functionality is not a part of the public roadmap for any competing operating systems -- including Linux.

		What's the meta, data? Metadata is widely known as data about data. For example, the title of a document, who created it, when it was created, and the language it is written in are all forms of metadata. Even corporate data, such as the data generated in the course of a transaction, may have some metadata attached to it. A call center, for example, may generate a lot of transactional data relating to incoming calls, but attaching an operator's name (or code) to each of the transactions qualifies as metadata much as an author's name that is attached to a document. Another form of metadata that could be assigned to document or a transaction is its category or classification. This is where taxonomy comes into play. Taxonomies are generally hierarchical and come in handy for assigning knowledge and data to categories. They involve the selection of the standard vocabulary that represents categories, sub-categories, sub-sub categories, and so on. Taxonomies also are thesauri because they also map synonyms to that terminology. As more data, documents, and knowledge are categorized according to any given organization's taxonomy, rich connections can be made between dissimilar data types that were previously related, but impossible to connect technologically because there wasn't a taxonomical superstructure sitting across or between the various stove-pipes for making those connections.

In the meantime, there's no shortage of vendors with solutions to wrap some or most of an enterprise's data and knowledge, regardless of where it lives or what's creating it in a metadata layer.

Getting going
"For humans, it's obvious how to organize things. But, learning to be explicit so that computers know what it is so we can find it is a tremendous challenge," according to Gartner analyst Rita Knox. The three challenges to programming explicitness into our information technology include understanding the differences between metadata and a taxonomy and what they're each good for; getting executive buy-in for metadata and taxonomy-related projects; and properly anticipating the need for the specialized talent required for such projects.

Getting executive buy-in can be difficult. According to Knox, enterprises have frequently cited complexity and ROI as issues that scare them away from taking on such ambitious projects. Though taxonomies can improve employee productivity (less time searching, more time on the front lines) and ultimately customer satisfaction (especially if they're working directly with external facing systems), quantifying those gains in terms of ROI can be difficult. Knox recommends tying taxonomy projects to very specific business goals such as improving response rate or customer acceptance. One application mentioned by Knox that could be a hot button for CEOs and corporate attorneys has to do with regulation compliance and the ability to mine archives for e-mails that might be relevant to the discovery phase of a particular investigation.