Subscribe to our Newsletter

Guest blog post by John P. Stevens

For a few years now Big Data has drawn the attention of many firms and CIOs. The rewards are major –new business insights, new customer segments, speed (faster time-to-answer, faster time-to-decision, and faster speed-to-market) to driving operational cost efficiencies and even corporate strategies. 

In the past few months I have been connecting with senior managers who want to start their Big Data efforts this year. You will need to take the necessary time to focus on some upfront planning activities before you start your first Big Data initiative.

Although there have been many improvements in the Big Data arena recently, companies still need to be focused on the scope of effort it will take to harness these benefits.  An executive of mine in 1993 once told me ‘If you haven’t made a million dollar (technology) mistake you haven’t tried hard enough’. With Big Data it may even easier to make a ‘Big Dollar’ mistake if you don’t do your homework and plan accordingly. If your firm is already in the petabyte range of data capture, or has a variety of disparate unstructured data sources to assess, and you have technology leadership’s approval to embark on your first Big Data initiative – start planning now.


Development of your Big Data strategy

Always start with the creation of your Big Data strategy.  It’s interesting to note that McKinsey & Co report that many firms new to Big Data lack one.  I have also seen this as well. It’s important to provide a Big Data strategy that will not only look at Big Data as a new technology direction for your firm, but it must be able to define how it will provide benefits and advances to your lines of business. Creating a Big Data strategy for your firm will also ensure that you have a strong foundation that will guide your subsequent Big Data implementation and long-term planning activities. There are several key items you should focus on when developing your Big Data strategy:

  • Create your Big Data vision statement.  This is a powerful view of Big Data and its contribution to your firm’s future success.  Make a statement that will capture the attention of executive management and will resonate with your organization.
  • Develop your Big Data strategy with your business stakeholders.  This will ensure that the business strategy will drive your Big Data strategy and be closely aligned. Many times your business stakeholders will provide valuable insights to their business plans and goals that will potentially uncover and highlight additional hidden opportunities.
  • Determine your Big Data business value proposition.  Identify how Big Data in your firm either greatly improves existing business value propositions or creates new business value propositions. Again this component is driven in collaboration with your business stakeholders.
  • Address how Big Data will transform your business and organization.  This is key component of your Big Data strategy since you must be able to identify and target specific improvements as a result of its integration to the business unit and organization structure.
  • Develop your Big Data solution roadmap. Show how your Big Data strategy will be implemented over time and its eventual integration in the Information Technology environment.  Include the basics of Big Data technology integration and address items such as data implementations, tools introduction, infrastructure expansion, governance, business goal horizons, etc.
  • Identify the Return of Investment (ROI) of Big Data to the firm.  Many times a technology strategy will ignore this part and only do a cost analysis on an initiative by initiative basis.  Set your assumptions and figure out the approximate ROI and when it can be realized.  Determine the ROI timeline for Big Data and its break-even point.  If you don’t do it here you will, more than likely, be asked for it in your executive review of your Big Data strategy.
  • Determine any Big Data risks in your strategy.  As with any new technology implementations there are risks associated.  Identify Big Data risks and what would be a remediation of the risk in your Big Data strategy.  Look at current Big Data adaptors and assess concerns uncovered and how these are being addressed.  Investigate future Big Data product trends, vendor product maturity, market share, tool restrictions, data security issues, and overall competitive analysis of existing solution models.

If your firm has already established a technology roadmap or overall enterprise technology strategy plan in place, make sure you are able to align your Big Data strategy to it.  It is important that your Enterprise Architecture group and Information Technology Governance department review your strategy so you gain any necessary approvals to proceed as well as to get any questions or concerns reviewed and resolved prior to starting any Big Data effort.  Once this is done make sure you go through any and all of your firms required review processes already in place.  This is especially true for your executive management review that is usually augmented with a slide presentation.


Initial Big Data planning

The Big Data plans I have seen are often too high-level and tend to concentrate on the data itself or infrastructure, or just the vendor product and tools implementation.  But you need to start planning before you start looking at any specific products and infrastructure and break down all relevant activities from creating a ‘plan for the plan’ to post implementation metrics and reviews.  The following are activities you should consider when planning for Big Data:

  • Define a scope and objectives statement. In the scope definition identify Big Data volume, velocity, variety, variability and complexity.
  • Create specific Big Data use cases that are driven from your Big Data strategy and objectives
  • Identify all executive checkpoints and stakeholder reviews
  • Develop a Big Data conceptual solution model
  • Perform a vendor product analysis and market assessment to the conceptual solution and objectives statement
  • Assess any buy versus build components and clearly provide justifications
  • Determine vendor management approach and product support requirements
  • Identify all potential licensing, contract and other product and infrastructure agreements necessary
  • Develop a resource plan and determine recruitment and training requirements (technology, data management and business personnel)
  • Determine all source data requirements and associated storage estimates
  • Determine all infrastructure and requisition requirements
  • Identify implementation approach, proof of concepts and business management trials
  • Provide financial estimates and a cost benefit analysis
  • Develop the Big Data solution integration model
  • Develop the Big Data logical data model and schema
  • Identify all metrics reporting requirements for solution assessment and evaluation
  • Develop the analytics model
  • Develop the Big Data solution point of failure and recoverability model
  • Identify all tool configurations and access requirements
  • Identify all product software and tools staging and implementation requirements
  • Determine product testing methods for all functional areas and defect remediation approach
  • Identify the product and data certification process
  • Determine the overall Big Data software release management approach
  • Determine production and product maintenance schedules
  • Provision for pre and post-production scheduling and support
  • Schedule post-mortem reviews 


Of course there are many more items you could include in the overall initial planning of Big Data for your firm.  The more planning items you can define prior to getting your Big Data solution in place the better prepared you will be for a smooth implementation.  Once you assess which planning activities are required you can build your own specific detail Big Data implantation plan which will keep you focused and becomes the communications vehicle of your Big Data progress.


Long-term Big Data planning

You need not to just create a plan to get Big Data in-house – you also need to decide your long-term plan for Big Data as well.  How does the Big Data long-term plan align with your firm’s technology strategic plan and does it readily be incorporated in the overall technology roadmap?

For the Big Data long-term plan, you need to make sure that post implementation of your first initiative you focus beyond basic volume and growth demands on data and infrastructure.  Some items to consider when developing a long-term Big Data plan would include:

  • Determine and define your long-term Big Data goals (be specific and add any target business lift or impacts and financial horizons to achieve)
  • Identify your multi-year Big Data timeline (start from initial Big Data assessments to levels of solution maturity).  Make sure you have a clear timeline aligned to your Big Data roadmap. If you already are looking to extend a relational data warehouse solution, then perform a gap analysis from your current state (relational data warehouse) to your future state (Big Data solution extension) and include in your timeline.
  • Determine your Big Data governance implementation and checkpoint its maturity. You should start this activity very early either in alignment to your Big Data implementation plan or soon thereafter.
  • Introduce Big Data standards and policies and determine issuance and communication targets and integrate into your Big Data governance implementation
  • Determine your data stewardship and ownership alignments as you mature your governance process
  • Identify Big Data lessons learned check points and determine best practices to publish and communication within the organization and utilize in your Big Data governance process
  • Establish your Analytics tool usage targets and user growth rate projections for data consumption
  • Identify expansion of Big Data Metrics reporting and metric horizons to achieve at specific checkpoints
  • Identify yearly Data volume growth targets (include introduction of any new data sources)
  • Determine your yearly infrastructure expansion estimates and target implementations
  • Align resource growth and additional training targets for all impacted organizations
  • Provide rolling, quarterly financial estimates and measurement against projected ROI
  • Evaluate data security and risk exposure controls and provide implementation targets
  • Identify any and all software licensing and maintenance contracts additions, changes and product releases on a yearly basis


Having a year-end review of your long-term plans with your stakeholders is recommended.  You and your stakeholders can assess where you are with Big Data and its maturity in your firm.  It also provides a forum for open discussion that may influence changes to your long-term Big Data plans or even a revision to the Big Data strategy as a result.


As with any new technology change the planning methodologies to introduce that technology into a firm will take time to assess and apply.  Big Data is no different.  Even the best attempts at planning does not guarantee ultimate success, but if you start with a Big Data strategy to drive your implementation and long-term plans your risk of failure could be greatly diminished.

E-mail me when people leave their comments –

You need to be a member of Hadoop360 to add comments!

Join Hadoop360

Featured Blog Posts - DSC