Big data has irrevocably changed the game and you need to be in it
By Mervyn Mooi, director at Knowledge Integration Dynamics
What are the business benefits?
An information architecture framework requires proper upfront and sustained planning of how data and information systems will work together to support the business strategy. With the advent of big data and the digital economy IT systems (and how it integrates) information architecture becomes more complex. Architectures generally have a three to five-year relevance lifecycle, so it is important to continually revise and refine the information architecture to support the business and enable leading-edge capabilities that cater and scale to all types of data and information and render resource economy.
There are many business benefits you can expect from an organised architecture (there may be more based on your business):
- Data/information quality and single consistent views of business entities (e.g. customer)
- Common data models and processes
- Leaner operational resources and increased efficiencies
- Reduced risk and overall costs
The above benefits translate into other benefits, such as:
- Narrower customer and market segmentation
- Even better decision making (through more sophisticated analytics)
- More valuable business insights as deduced from quality data/information
- Product and service development
- Consumer dialogues
- New revenue streams
- Tailored services and products
Why is it even necessary?
Big data has changed the rules that govern data. Data used to be placed in databases on dedicated servers and shared over networks in a rather regimented time-controlled manner. Big data however is dynamic – it is sourced from and propagated to many different server locations (or clusters of servers) across the globe, at a rate and scale that is difficult to comprehend.
The data and information are primarily in the public domain and your company doesn’t own it all. Some cannot be placed in traditional databases or data warehouses because it includes different types of unstructured data. It requires specialised database and file systems e.g. NoSQL and Hadoop. The data may sometimes be placed in a cloud system because it may be cost effective to do so, it may offer functionality your own systems don’t offer, or it may only be required for a short period of time.
Image Credit: Duncan Hill
The upshot is: data of different types, will be sourced from many different locations, worked with in many different analytics tools, and fed through to many different end points.
The information architecture must specifically manage data security and information privacy to also comply with governance processes and standards. Security outside of your organisation’s domain (off-premise) remains a challenge.
The data environment has changed
The new data environment, which includes hardware and software – plus the policies that govern processes, will probably use more open source software and commodity hardware. Commodity and open source mean cheaper to buy but there are other costs to consider such as the people who can use the software tools and maintain the hardware. The challenge is to trust open source to deliver against standards-based requirements, that you can support the software for business sustainability, and that it still requires on-board expertise to implement and maintain.
Many tools people used to work with data in the past no longer work. What can you do about it?
- Database and data software vendors are extending the capabilities of their tools. New tools or new functionality means teaching existing employees new skills, which requires time and money, or finding new people who can use the new tools.
- Some companies (companies that use data tools) are adding to their traditional data tools themselves so that the tools become capable. It saves them retraining employees and means they can continue to capitalise on existing skills and experience. One way this is being done is to write a SQL abstraction (or virtual) application and data layer on top of the environment.
- Others are creating dashboards specially coded to shield their users from having to know R (a programming language) used for statistical analyses or the SQL language.
It’s clear that there’s no one way to deal with this because every environment is different based on the investments companies like yours have made in skilled data people, software and hardware.
The future is in the framework
However you and your company choose to deal with this shifting landscape, you’ll need a proper framework to deal with the complexity of data sources, tools, hardware, and end users. Slapping something together on the fly will result in much future heartache and a whopping great bill to match.
But, no matter your chosen solution, the fact remains that companies are beginning to get the benefits of big data so you can’t afford to ignore it. Yet, with what is a relatively immature field, you need to give yourself every opportunity at success. A proper information architecture framework is just the ticket.