Single View Challenges: Ensuring Timeliness in Data Systems

As we continue our breakdown of the key challenges organisations face when creating a Single View solution, we reach number 4 in our series; Timeliness.

We have already addressed how to tackle the sheer volume of data and how to use Micro Service Design and Taxonomies to address standardisation and governance, however, what about the timeliness and speed of accessing the data?

Using traditional approaches to moving data from the source systems (ELT, ETL) is effective and does work in many scenarios. The challenge, however, is when real-time or near real-time changes need to be seen within the Single View solution.

The concept of event-driven architecture, described for the logical decoupling of systems, can also support ‘closer to real-time’ data movement. Having events update allows representation  of information in ‘near real-time’ to interested components. One interested component would be the aggregated store, but some caching layers to help manage the insert/update load on our data stores for fastmoving transactional data may also be required. A cache is able to listen to events and provide a ‘speed’ layer to the data (ideally through segmented microservices).

Read more about our Single View; The Challenges Series on the Insights Page

‍