Mega-Volume Ingestion of Payer Data in 5 Weeks

The client: A large US insurance provider (payer) with over 350 operating clinics and care centers, and several subsidiary companies across the US servicing over 17 million members. The client’s network includes several lines of business, each with their own suite of IT systems, data sources, policies and EMR systems.

The issues: The client’s network includes sources and formats of data from hundreds of points-of-care systems, several legacy EMRs and multiple third-party vendors. The network’s IT infrastructure included pools of unstructured data sets, disjointed data exchange engines and incompatible security and policy features, which led to cumbersome and even inaccessible health information flow. The client experienced continued performance and downtime issues with their data services cloud vendor—which processed only ~50,000 transactions per second—which severely limited data exchange across the network. This caused restrictions on their scalability initiatives, which required manual interventions—allocating time, budget and resources for frequent infrastructural, upgrade and data management projects. Keeping in step with industry expansion trends, the client’s future objectives included new acquisitions and business consolidation. They recognized they could not scale and stay compliant with ONC and CMS mandates with their existing restrictive, manual, costly data exchange architecture.

The objective: The client approached Smile Digital Health to demonstrate the functional capabilities of our solution suite. Their evaluation objectives included:

- Mega volume, high speed data ingestion—at least 200,000 transactions per second today with the ability to scale beyond that in the future

- Standardization of data from their multiple point of care systems in a variety of formats—HL7 v2, HL7 v3, C-CDA, CSV and JSON—into the interoperable open FHIR^® format (i.e. no proprietary vendor lock-in models)

- Systematic, streamlined and automated information flow and processes on an event-based data architecture that can scale across the network and require minimal manual intervention

- IT infrastructure maintenance and updates with no down-time

- Shared security, auditability and encryption features

- Data exchange management in compliance with CMS 9115-F Patient Access API and Provider Directory API.

To avoid delays and interruptions to their business operations the client required a short implementation period with minimal downtime. 

Solution: Smile’s Health Data Platform (HDP) is a modular, highly configurable, robust data exchange and interoperability platform. It creates a data ingestion engine and repository that seamlessly integrates large volumes of unstandardized legacy data from hundreds of disconnected sources across state jurisdictions. Data is standardized into the FHIR format regardless of which EMR or external third-party systems it originated in.

Expertise: Based on the extensive experience from over 160 FHIR-based enterprise-level implementations, and actively innovating with HAPI FHIR and IGs (Implementation Guides), the team at Smile designed and implemented several architectural and data flow strategies that met all of the clients objectives.

The platform’s ingestion engine and the innovative solution architecture enabled the production time for Smile’s implementation of ~5 weeks—from the date of contract to the go-live day, including the evolutionary phases of architecture design.

Smile HDP’s exceptionally short production time—~5 weeks—was made possible by two main components:

Designing and Implementing a Streamlined and Secure Architecture

The architecture design included automations and real-time data processing to create, modify and query data across systems. This unified the disparate workflows and increased data access speed across the network. The overall throughput (system performance speed measured) was 255,087 interactions per second, with concurrent transactions—like search queries through databases across the network—returning in under 1,000 milli-seconds.

Smile’s solution includes built-in security by design—highly configurable encryption features like the KeyStore Protocol and Mutual Authentication, and certifications like HITRUST R2 v9.4, ISO 27001:2013, ISO 27018:2019, ISO 13485:2016, and SOC-2 Type II. This allowed every interaction across the network—access requests, queries, log-ins and more—to be sequentially recorded into an immutable log, available for audit.  

When this secure and streamlined architecture was installed, the client achieved a 99.99% annual availability (under 52.5 minutes of network downtime in a year).

Large-scale High-Speed Data Ingestion

The Smile HDP ingested and housed data for over 2 million patients, with over 2 billion resources in the FHIR format. The ingestion and conversion process to load these 2 billion resources into FHIR took just over 26 hours, compared to the industry average of ~5 days. These resources were standardized in FHIR, de-duplicated and made actionable. Smile’s cloud repository that housed these resources consumed ~2.5 TB of storage, which is less than half of its capacity.

With these main components implemented, the client could now scale without compromising performance and security. In addition, their in-house teams could continue technical maintenance, optimization upgrades and additions to the network without significant cost, effort and time.