How MegaScale enables near unlimited storage and scalability on the path of data transformation and interoperability.
We would love to see it: the fast and furious adoption of the HL7® FHIR® standard as developers build mission-critical health IT infrastructure to support growing population health needs. But rapid growth has created a new data "problem"—terabytes upon terabytes of data stored across multiple databases, including cloud, on-premise and hybrid.
Payers, providers and health information exchanges (HIEs) create, manage and access far more data than any single enterprise-level database can hold—for instance, most relational databases cannot reasonably handle more than 100TB of data in a single instance. Popular cloud-hosted databases can store even less. Increased security threats, privacy regulations are part of the current landscape. Those along with CMS and ONC rules that restrict data archival compound the data storage problem.
100TB may seem like a lot. Today, it represents the size of data held at a large hospital’s EHR or a large payer’s member claim history. As we move towards digitization and data transformation, even a small hospital or payer will surpass the 100TB limit in the very near future. Planning for an organization’s growth can be hard, so scale shouldn’t be limited by storage capacity.
A common solution right now is to create multiple database instances. But this only inflicts a new set of complicated and costly administrative headaches.
Multiple Instances: Good Intentions, Limited Use
From a governance and integration perspective, data can be stored in a single repository. However, payers, providers, payviders and most HIEs quickly exceed the maximum storage capacity of their chosen platform.
Archiving and purging the data isn't an option. The apparent answer is to create more storage space. But is setting up multiple instances the best way to go?
It's true that multiple instances can solve several problems. This strategy can:
-
Create more room for the data
-
Physically separate data for privacy or security reasons
-
Spread load across more than one instance or temporary database
-
Create separate environments for the data for various use cases
Despite good intentions, multiple instances have their limits. Some of which are:
-
Only a portion of necessary data is accessible
-
Data is incomplete, lacking detail, and often duplicated
-
Patches are harder to plan
-
Data manageability due to separate or split silos
At Smile, we created a better way to store, manage and scale health data.
A new feature addition to Smile’s Health Data Fabric (HDF) toolbox is MegaScale. MegaScale allows organizations unlimited amounts of data to be stored and accessed in a single FHIR server. Using distributed architecture, MegaScale enables data storage far beyond industry standards in a single instance allowing for near unlimited scalability.
It’s that easy!
Benefits of MegaScale:
Organizations that enable the MegaScale feature, gain the benefits of instance stacking without all the drawbacks:
-
Less administrative, operational and technical set-up work and lower maintenance— freeing up IT resources to work on other more core projects
-
Ensure privacy and protection—PII and PHI are physically separated as needed, shielded from data breaches
-
Data is complete and unduplicated—it is stored in a standardized and automated way with a normal level of detail
-
Log into a single system—data is fed into a central FHIR database
Getting Started
Your organization could benefit from a MegaScale-enabled server if:
-
The amount of data you need to store exceeds the natural limit of a single instance of the database technology being used
-
The data needs to be kept physically separate due to privacy or legal rules
-
You have multi-tenancy agreements with your customers that required strict data isolation
Smile will take care of the rest—processing which database to talk to, and which data-bank stores what type of information.
Megascale is available both to customers who are self-hosted, as well as those who employ Smile’s Health Data Fabric (HFD) solution. Additional support is also available through our service package called, Smile’s Managed Services.
What's Next
We have blasted through the industry expectations of scalability with MegaScale. What is next for us is to experiment and learn how it impacts the entire Health Data Fabric.
In creating MegaScale, Smile's HDF gives organizations of any size—from a global enterprise to a small local organization—more flexibility in data storage, access and structure to architect their system in whatever way they need.
But that’s not all. Our clients are in the sandbox with us, as we test out how MegaScale works with other more complex data architectures and models. The MegaScale feature enables storage and scalability without limits as they move towards modernization and digitization.
To learn how MegaScale impacts the entire Health Data Fabric with respect to capabilities and limitations while in "experimental mode," visit our documentation.
To keep a pulse on what’s new tools, features and abilities of Smile’s Health Data Fabric, stay tuned for our upcoming quarterly releases. The next one is scheduled live on May 16th 2023 at the Intelligent Health Symposium, held in Toronto, Canada.
Isn’t it time to scale to your organization’s current and future needs? Connect with one of Smile’s HDF experts today to break through your database ceiling.
