Data in transit raises security risks

Keeping data secure can be as daunting as herding cats, unless data governance is approached strategically.

There is no doubt data proliferation is presenting a challenge to organisations. IDC predicts the data created and shared every year will reach 180 zettabytes in 2025; and we can expect much of that data to be in transit a lot of the time.


This means it will not be securely locked down in data centres, but travelling across layers throughout enterprises, across the globe and in and out of the cloud. This proliferation of data across multiple layers is raising concern among CIOs and businesses worldwide, particularly in light of new legislation coming into play, such as the General Data Protection Regulations for Europe, due to be implemented next year.

Where traditionally, data resided safely within enterprises, it is now in motion almost constantly. Even data on-premises is transported between business units and among branches within the enterprise, presenting the risk of security chaos. The bulk of the core data is always in movement – these are enabling pieces of data moving within the local domain. At every stage and every endpoint, there is a risk of accidental or deliberate leaks.

When data is copied and transmitted via e-mail, porting or some other mechanism from one location to another, this data is not always encrypted or digitally signed. To enable this, companies require the classification of data assets against the due security measures each would require, and this is not evident in most companies today.

At the next layer, commonly described as the ‘fog’ just below the cloud, data and information travelling between applications and devices off-premises are also at risk. A great deal of data is shared in peer-to-peer networks, connected appliances or by connected cars. If this data is not secured, it too could end up in the wrong hands.

In an ever-growing data ecosystem, enterprise systems should be architected from the ground up with compliance in mind.

Most companies have data security policies and measures in place, but these usually only apply on-premises. Many lack effective measures when the data physically leaves the premises on employee laptops, mobile devices and memory sticks. These devices are then used in unsecured WiFi areas, or they are stolen or lost, putting company IP at risk. Data on mobile devices must be protected using locks, passwords, tracking micro-dots, and encryption and decryption tools.

Finally, at the cloud layer, data stored, managed and processed in the cloud is at risk unless caution is exercised in selecting cloud service providers and network security protocols, and applying effective cloud data governance.

While large enterprises are becoming well versed in ensuring data governance and compliance in the cloud, small and mid-sized enterprises (SMEs) are becoming increasingly vulnerable to risk in the IOT/cloud era.

For many SMEs, the cloud is the only option in the face of capex constraints, and due diligence might be overlooked in the quest for convenience. Many SMEs would, for example, sign up for a free online accounting package without considering who will have access to their client information, and how secure that data is.

Locking down data that now exists across multiple layers across vast geographical areas and is constantly in transit demands several measures. Data must be protected at source or ‘at the bone’. In this way, even if all tiers of security should be breached, the ultimate security is in place on the data elements at cell level itself throughout its lifecycles. Effective encryption, identity management and point in time controls are also important for ensuring data is accessible only when and where it should be available, and only to those authorised to access it.

Role and policy-based access controls must be implemented throughout the data lifecycle, and organisations must have the ability to implement these down to field and data element level.

In an ever-growing data ecosystem, enterprise systems should be architected from the ground up with compliance in mind, with data quality and security as the two key pinnacles of compliance. In addition, compliance education and awareness must be an ongoing priority.

All stakeholders, from application developers through to data stewards, analysts and business users, must be continually trained to put effective data governance at the heart of the business, if they are to maintain control over the fast-expanding digital data universe.


The wielder, not the axe, propel plunder aplenty.

By Mervyn Mooi, Director at Knowledge Integration Dynamics. (KID).
Johannesburg, 25 Sept 2014

Business intelligence is a fairly hot topic today – good news for me and my ilk – but that doesn’t mean everything about it is new and exciting. The rise and rise of BI has seen a maturation of the technologies, derived from a sweeping round of acquisitions and consolidations in the industry just a few years ago, that have created something of a standardisation of tools.

We have dashboards and scorecards, data warehouses and all the old Scandinavian-sounding LAPs: ROLAP, MOLAP, OLAP and possibly a Ragnar Lothbrok or two. And, like the Vikings knew, without some means to differentiate, everyone in the industry becomes a me-too, which means that’s what their customers ultimately get. And that makes it very hard to win battles.

Building new frameworks around tools to achieve some sense of differentiation achieves just that: only a sense of differentiation. In fact, even when it comes to measurements, most measures, indicators and references in BI today are calculated in a common manner across businesses. They typically use financial measures, such as monthly revenues, costs, interest and so on. The real difference, however, comes in preparing the data and the rules that are applied to the function.


A basic example that illustrates the point: let’s say the Vikings want to invade England and make off with some loot. Before they can embark on their journey of conquest they need to ascertain a few facts. Do they have enough men to defeat the forces in England? Do they have enough ships to get them there? Do they know how to navigate the ocean? Are their ships capable of safely crossing? Can they carry enough stores to see them through the campaign or will they need to raid settlements for food when they arrive? Would those settlements be available to them? How much booty are they likely to capture? Can they carry it all home? Will it be enough to warrant the cost of the expedition?

The simple answer was that the first time they set sail they had absolutely no idea because they had no data. It was massively risky of the type that most organisations aim to avoid these days. So before they could even begin to analyse the pros and cons they had to get at the raw data itself. And that’s the same issue that most organisations have today. They need the raw data but they don’t need it, in the Viking context, from travellers and mystics, spirits and whispers carried on the wind. It must be good quality data derived from reliable sources and a good geographic cross-section. And in preparing their facts, checking they are correct, that they come from reliable sources, that there has been case of broken telephone, that businesses will truly make a difference. Information is king in war because it allows a much smaller force to figure out where to maximise its impact upon a potentially much larger enemy. The same is true in business today.

Before the Vikings could begin to loot and pillage they had to know where they could put ashore quickly to effect a surprise raid with overwhelming odds in their favour. In business you could say that you need to know the basic facts before you drill down for the nuggets that await.

The first Viking raids grew to become larger as the information the Vikings had about England grew. Pretty soon they had banded their tribes or groups together, shared their knowledge and were working toward a common goal: getting rich by looting England. In business, too, divisions, units or operating companies may individually gain knowledge that it makes sense to share with the rest to work toward the most sought-after plunder: the overall business strategy.

Because the tools and technologies supply common functionality and businesses or implementers can put them together in fairly standard approaches as they choose, the real differentiator for BI is the data itself and how the data is prepared – what rules are applied to it before it enters the BI systems. Preparation is king.

These rules ultimately differentiate information based on wind-carried whispers or reliable reports

Tougher than rocket science.

By Mervyn Mooi, Director at Knowledge Integration Dynamics. (KID).

Data scientists have become among the most prominent professionals in companies today. Their promise is epic and growing daily.

There are many definitions for a data scientist; IBM notably defines this character as follows:

“A data scientist represents an evolution from the business or data analyst role. The formal training is similar, with a solid foundation typically in computer science and applications, modelling, statistics, analytics and maths. What sets the data scientist apart is strong business acumen, coupled with the ability to communicate findings to both business and IT leaders in a way that can influence how an organisation approaches a business challenge. A good data scientist will pick the right problems that have the most value to the organisation.”

A data scientist is therefore highly skilled at overcoming data and information challenges.

It’s not surprising, considering that data scientists at Internet giants such as Google and LinkedIn have shown the potential impact they may have. For example, LinkedIn data scientists figured out how to recommend people you may know, a feature that boosted page views by millions, and therefore LinkedIn’s marketability. When the feature was tested back in 2006, it achieved click-through rates 30% higher than other prompts to visit additional pages on LinkedIn. It shifted the organisation into a higher gear, and since 2011, the company’s share price has increased 65%.

But behind the façade of the data scientist demigods, enshrouded in near mythic qualities, lie pragmatic realities that must be accounted for by responsible organisations seeking their employment – and the business results they promise. Desired results range across a vast expanse of business issues, but some include dialogue with consumers; accelerated product development; regulatory, reputational and operational risk analysis; data security and compliance; new revenue streams; reduced production maintenance costs; personalised Web site experience to propel marketing campaigns and customer engagement; tight financial and operational performance monitoring and results analysis; maximised customer profitability; minimised customer churn; fraud detection; and increased marketing campaign effectiveness.

First reality: teams

When interrogating the functions data scientists must perform in order to promote these business goals, their demigod status becomes earthly palpable. It quickly becomes clear that the list of skills is so karmic it would require several lifetimes to fully acquire. These include statistics, mathematics, predictive analytics, computer science, data engineering, programming, information management, integration, data warehousing, virtualisation (Hadoop, MapReduce and more), data quality, business analysis, data mining, visualisation, content management, collaboration, presentation authoring and executive communication – among others.

The first reality then is that no one person can extensively fulfil this function. There must be a team of people the size of which depends on the scope of the company and the projects it intends.

Second reality: scarcity

The next reality that hits home is the scarcity of these people, which is a contributing factor in their prohibitive expense. They are so scarce that they are often referred to as unicorns. Data scientists are routinely offered positions in SA with salaries averaging half a million rand per annum – this without much experience. Experienced scientists go for up to triple that amount. Regardless, a data scientist possessing all of the above skills at once is yet to be found.

Although the data scientist came to prominence with the advent of big data, data science as a discipline is an age-old concept that incorporates a wide variety of ICT skills. And this is where data scientists find their place, by quickly bringing together the necessary skills and knowledge to produce information and insights that would result in value for a company.

Third reality: foundations

With that in mind, the third reality is the necessity of adequate infrastructure and maturity before hiring even one data scientist. No lucid executive should afford the expense of employing data scientists without first ensuring the fundamentals are sound.

[Data scientists] are so scarce that they are often referred to as unicorns.

There should be a data management strategy, good data quality, good data governance, a platform and architecture, and operational data processes. These outcomes rationally precede statistical and predictive analyses and the entirety of the interrogative and operational functions companies seek to perform with their data.

Fourth reality: toolbox

Thereafter, the fourth reality is that these people require a toolbox. They must have the analytical tools to perform data quality experiments and roll-outs as well as the statistical tools such as Statistical Package for the Social Sciences, the R programming language and software environment for statistical computing and graphics, PIG and general purpose Python programming languages for anywhere access and quick high-level programming, and MATLAB (Matrix Laboratory), which is useful for linear algebra, solving algebraic and differential equations, and numerical integration.

Larger South African organisations already, to some extent, have the foundation they require and even some of the skills to launch the data scientist function. Some have already acquired the requisite skilled personnel and have operational projects.

Many smaller companies, however, are gingerly dipping a toe into the waters, because they know and understand the business benefits. However, they do not know and understand what lies behind the façade of benefits, they do not have the resources to retain experienced people indefinitely, they do not have the fundamentals in place, and they struggle to grasp the scope of their intentions. These companies are not, however, to be left in the cold. The scarcity of skills has resulted in the acknowledged data consultancies investing in personnel to fill the breach.

Professional Data Management Services


Founded in 1999, KID is a comprehensive and successful data management business. Our expertise and skills provide leading-edge business and data solutions, ensuring a consistent and cohesive view of your business. At KID we use established methodologies and have a proven approach to deploy appropriate technologies, ensuring successful and sustained implementations and project deliveries that are on time and within budget.

KID is a pioneer in data solutions – we are at the fore-front of EIM development and implementation with the objective of economizing on all the necessary architectural components, processes and procedures.

KID offers comprehensive professional consulting and resource services; specialising in business intelligence solutions, data warehousing, data integration and data management. Our expertise and skills provide leading-edge business and data solutions, ensuring a consistent and cohesive view of your business.

Since its inception, KID, has delivered solutions and services to over 100 corporate clients and has by far the most experience in this industry complemented by a reputable history of successful deliveries and implementations.