Compiling Logical Mappings to Executable Code

Overview

A frequent problem in data exchange is the need to map between alternative formats for data. The AutoMed platform provides a repository for BAV transformations that can be used to specify the logical mapping between these alternative formats of data. The aim of this project is to write a compiler to read BAV transformations from the AutoMed repository, and to produce applications (in Java, C#, and/or SQL) that are capable of processing large volumes of data (meaning data sets containing thousands of rows of data, as opposed to small demonstration files).

Detailed Description

Data exchange between companies and individuals is often inhibited by differences in data formats used by those companies and organisations. Examples include (taken from the problem domain of exchanging parts listings between computer spare parts suppliers) include:

Part numbers may be stored in difference lengths, with one organisation using 7 characters, the other using 9 characters, where the 7 character part numbers are perfixed with "00".
Part numbers may store type information using extensions (-E for exchange, -C for compatible) or using a separate type column to define "exchange" or "compatible"
The type of a product may be identified by a number in one organisation (1, 2) whilst another uses character fields ("Motherboard", "Power Supply", ...)
The type of a product may be expressed in different languages (English, French, Italian, ...)
Prices may be expressed in different currencies (GBP, EUR, SEK, ...)

The objective of this project will be to develop a set of transformation patterns that can be used to solve the general problem of data exchange between databases with conflicting representations of data, to implement those patterns as tools that create BAV transformations in the AutoMed API, and to then compile those transformations into code that can be used in routine data processing (eg Java with JDBC, or SQL with views on remote tables). The project should also illustrate the application of those transformation patterns and compilation tools in a case study.

There will be a PhD award available for students in the general area of Data Integration from October 2011, in which work on this project could be continued to PhD level.