VesselAI Data Harmonisation Services
The VesselAI Data Harmonisation Services is a set of services provided through an User Interface for harmonising Maritime Domain-related data sources with Big Data characteristics. This harmonisation process entails the mapping of raw data schemas/formats to harmonised and unified data schemas provided by the Data Harmonisation Services, for data types such as AIS data, Weather data and Noon Report data. Other data types may be harmonised by the Services through the upload of custom harmonisation schemas to the services.
The VesselAI Data Harmonisation Services address the following technical requirements:
(i) able to deal with batch and streaming raw data in many formats and sizes;
(ii) assure data quality;
(iii) efficient big data harmonization;
(iv) being able to address interoperability at the data level, enabling the development of additional value added services for users;
(v) inclusion of custom schemas in order to transform and harmonize data into standardized or proprietary schemas;
(vi) and a robust schemaless intermediate storage system, that can tap into different, structured and unstructured data sets, in order to process data from data sources.