IIC Technologies develops data exchange system for Korea Hydrographic and Oceanographic Agency (KHOA)
posted in development by admin
How does a complex organisation that uses a number of discrete and differently structured databases reduce overlaps and eliminate manual transfer operations which are inefficient and prone to error? A solution recently achieved on behalf of KHOA by IIC Technologies comprised the design and development of a RESTful API middleware solution that has now automated much of the process.
KHOA uses three independent information systems to manage their key geospatial data holdings. Since there is a sizable overlap between the data content of these systems, the respective departments must ensure the consistency of all data and resulting data products at all times. Historically, all such data exchange operations were carried out manually.
Figure 1: Manual Data Translation
As the underlying data structures are completely different in each database, this was a very challenging and time-consuming task to do manually and KHOA contracted IIC’s Innovation Center to help automate the process.
Figure 2: Automated Data Translation
The IIC Technologies solution was a developed, web-enabled middleware solution that allows KHOA to configure all data translation operations, so as to exchange data between the systems automatically with just a few clicks. To achieve this a powerful software toolset allows operators to map the individual feature objects between the databases holding the underlying Notice to Mariners (NTM), Hydrographic Production Database (HPD) and S-100 information.
Figure 3: Data Exchange System Interface
The emphasis was on interconnecting these systems to fully support lossless and two-directional data transfers using RESTful API middleware. The objective of the middleware approach was to make it possible for KHOA to create their own customized user interface so that KHOA officers could easily convert and exchange the data on their own as required. The middleware provides the data exchange capabilities via a set of RESTful APIs implementing a selection of Create, Read, Update and Delete (CRUD) database operations. IIC Technologies also used the Swagger platform to formally document the complete API design, all endpoints, data models, and system response codes.
Figure 4: Swagger RESTful API
KHOA was also provided with a sample front-end client to showcase a representative subset of functionality, enough to demonstrate system inner-workings and all key functionalities.
The development of this middleware proved to be a very challenging project for the IIC team, as it required not only the understanding of the three different database platforms used for the NTM, HPD and S-100 systems, but also the very different feature object models employed by each of them. This required the IIC Team to closely investigate all database tables, stored procedures, and complex relationships to achieve a complete understanding of all underlying feature types, their nested attributes, and relationships. The NTM database was relatively straightforward, and the main challenge was the mapping between the NTM database and the HPD database to support transfer of the features. Understanding the other database models, in particular the new S-100 structures and the mapping between the other two databases, was a big learning experience for the team.
The outcome, the robust functionality, and the ability to configure all future data conversions based on KHOA requirements has been well received and it allows the operators to automate ongoing data exchanges between the systems and ensure data consistency with far less manual effort.