Big Data Insights on Transforming Healthcare

Buy, don’t build, for faster time-to-value

Healthcare organizations (HCOs) have been, and continue to be, invested heavily in IT to improve outcomes and lower costs. While much of this investment continues to support the implementation and optimization of mandated electronic medical record (EMR) systems, the post-EMR implementation world is on the horizon, and all of those EMR systems will be generating tremendous amounts of data. According to PwC’s 2016 Annual Global CEO Survey, 95% of healthcare CEOs are already looking beyond EMR and say they are exploring better ways to harness and use big data.

Why are they so quickly pivoting to an emphasis on data analytics? While the quality and cost benefits of improved operational IT systems (e.g. EMR and claims) are significant, they are insufficient to meet many of the new challenges associated with a healthcare industry that is consuming 17.1% of the US GDP – way ahead of the spend of the nearest developed country, Sweden, at 11.3%.1 Healthcare’s massive footprint and continuing growth necessitate high levels of investment to reduce costs with an emphasis on improving health outcomes. Much of that investment will be in big data analytics.

Key to Transforming Healthcare

For the purposes of this article, we will use “big data analytics” to mean analytics applied to large volumes of structured and unstructured data resulting from a variety of internal and external data sources. Big data analytics is capable of providing actionable insights to meaningfully, if not dramatically, improve economic and health outcomes through:

  • Improved clinical effectiveness in terms of quality of care, patient safety, reduction of medical errors, wellness, prevention and disease management
  • Improved operational effectiveness through optimized service area and network management, pay-for-performance, increased operating speed and agility
  • Improved financial performance through more appropriate utilization, optimized supply chain and human capital management resulting in increased revenue and margin
  • Improved member/patient satisfaction, acquisition and retention
  • Improved risk management, regulatory compliance and reduced fraud and abuse

The potential to realize these improvements depends on the quantity, quality and diversity of data available. Data volumes are exploding as a result of EMR adoption and proliferation of quality external, value-added data creating powerful new capabilities such as linking patient care plans more closely with public health data. Consider the likely impact of rapidly growing personal data coming from wearable devices, such as Fitbit or Apple Watch. Spending on wearables that create health related data will reach $52 million by 2019 according to ABI Research.2  Add in consumer-generated data from social media and the volume of data to support analytics will require big data deployments.

Will HCOs Value from Investments?

Time-to-value is the critical issue, and there are three major drags on time-to-value associated with traditional enterprise data warehouse and in-house analytics approaches that, so far, most business leaders, IT departments, and technology suppliers seem to be ignoring:

1. New knowledge from big data analytics has to be practically applied. Using predictive analytics in combination with the integration of EMR and non-EMR (administrative, patient, population, and environmental) data, the potential for quality and cost improvements through the creation of new knowledge is tremendous. But if this newly created knowledge is not translated into practice, it provides no benefits, and making it practical requires the right mix of experts to interpret and apply findings. Beyond the quants and the geeks in analytics and IT, successful teams must also include clinicians, policymakers, patients and more, to ensure that newly created knowledge is truly valuable and able to quickly be put into practice.

2. There is a healthcare informaticist gap. While there is ambiguity in the data, research indicates there are currently fewer than 15,000 medical professionals with formal training and certification in healthcare informatics in the US.  Demand for these experts over the next five years is expected to approach 70,000.

That is obviously a huge gap, and it is unrealistic to expect that supply will scale to meet that demand in any reasonable time horizon. The U.S. medical education system has only begun to respond with a few hundred credentialed healthcare informaticist program slots, and importing skills from overseas or offshoring may face regulatory hurdles. Bringing in data scientists from other industries – where they are also in high demand – and training them in the required clinical skills is also unlikely to reach the scale required.

It is hard to imagine that without a skilled workforce, business leaders and clinicians will ask the right questions, let alone get the correct answers and draw the right conclusions. Technology suppliers must adjust their products and services to help mitigate this shortfall of skilled workers.

3. Traditional technology delivery cycle. The process of bringing big data analytics projects online is much too slow to meet business needs in a reasonable time frame for many reasons, both in and out of the control of the CIO. The typical IT delivery cycle is glacial, and the implications of that prolonged time-to-value and inability to nimbly respond to market changes and opportunities can be devastating for HCOs.

Here’s an example:

It’s June 1 and Dara has the idea that adding hyper-local weather data to the predictive model for clinic staffing may reduce wait times and overtime costs during the upcoming flu season. 

Leadership is interested, and it only takes IT about a month to develop and get approval for a proof of concept (POC). Securing an interim agreement with the new weather data provider for the trial takes another month, and then a couple of additional weeks to ingest the data. At that point, it is determined more capacity is needed in the Hadoop cluster. No worries, IT is using the cloud, so it only takes a week to get things provisioned and up and running.

The analytics team is shorthanded and cannot prioritize the project for three weeks, and when they do jump in: darn, we need more capacity on the SAS environment; that adds another week.

Two weeks or so after the new data elements go live, the team produces promising initial results.The POC is a success.

By the end of August, the team is working furiously to pull the business case together and present a request for $5M to license the data and operationalize the solution. It’s the end of September before approval is granted and purchase orders are issued for more software and necessary outside data licenses. Capacity is provisioned in the cloud; software, scripts and models are built and tested. By the end of January, the new capability is online, providing insights that will influence clinic staffing in mid-February, which is about when flu season ends, delaying the ROI of the $5M+ investment until next flu season.

The cloud-enabled analytics described in the example above is the kind of technology most organizations think is going to make them the nimblest of the nimble, and it certainly shortened the time-to-value, but it is still way too slow.

Insights-as-a-Service can Address Challenges

Consider an alternative approach from above. What if Dara was able ask the question, “Can we add hyper-local weather data to improve our predictive model for clinic staffing during flu season?” of a vendor who could leverage its existing license for weather data and use its existing infrastructure, data scientists and healthcare informatics teams to answer that question rather than building a whole new systems capability?

The three months to complete the POC would shrink to just a few weeks or even days, and implementation could proceed immediately from there. What if Dara could ask a question each month, or each week? And rather than kicking off an elaborate procurement and development cycle, Dara could just pay per insight?

The Path Forward

Due to the informaticist gap, only the very largest HCOs will have the resources to attract and retain the appropriately skilled workforce to implement big data analytics capability internally for the foreseeable future.

The insights-as-a-service alternative described above, while promising, is still challenged, as true insights-as-a-service offerings are only just emerging in healthcare. Many suppliers claiming to offer insights-as-a-service are just applying a marketing label to a traditional hardware, software and consulting services model, which is ultimately subject to the same traditional technology adoption and delivery cycle timeframes.

Despite the relative immaturity of insights-as-a-service offerings generally available in the marketplace today, leading HCOs, like their peers in other data-driven industries such as telecommunications, are starting to be able to structure true insights-as-a-service partnerships with their technology suppliers today.

The organizations that pioneer these new relationships in healthcare look to enjoy a significant competitive advantage in their core markets. By focusing on time-to-value rather than building systems, CIOs and business leaders have an opportunity to go beyond the standard offerings of their suppliers and craft next-generation insights-as-a-service relationships, yielding competitive advantage through scalable knowledge creation put into practice.


  1. The World Bank Health Expenditure Report. The World Bank.
  2. Integrating Consumer Wearable Health Devices Will Drive Healthcare Big Data Adoption.” ABI Research.


About The Author

Each year more than 350,000 professionals advance their career with Elite Learning.