October 7, 2022
Smile CDR processes a groundbreaking 22,251 Resources Per Second on standard mid-size company infrastructure.
You’re likely reading this because you’re interested in evaluating FHIR® (Fast Healthcare Interoperability Resources) server performance. Excellent!
Benchmarks are a great objective measure, and at Smile Digital Health we’re proud of our world-class performance and happy to publish our benchmarks—including the test data and methodology for transparent reproducibility.
There are lots of great technology solutions out there with plenty of promise. They’re built with competence and tested thoroughly. But testing a solution in a small, controlled environment with hundreds or even thousands of subjects is one thing. Operating it at scale, with tens or hundreds of millions of real lives and their infinitely messy real world data, across various regulatory jurisdictions and nuanced use cases is completely different.
Performance testing at scale is essential because it most often provides the only accurate picture of performance and attendant issues at scale. Smile has a number of such implementations. For this benchmark, we’ve sought to simulate a real world example of one of these implementations.
It’s worth mentioning that beyond benchmarks, there’s also an equally important subjective component around features and customer experience. However, this piece explores only the objective measures.
The Scenario
In this real-world example, the organization is a mid-sized Health Information Exchange (HIE). This organization receives clinical and health data from upstream hospitals and providers, labs, imaging centers and pharmacies. Our example HIE initially ingests the patient information of one billion FHIR resources—comprising one million patients, each with an average of one thousand records. After the initial load, they need to accommodate a regular peak load of two thousand resources per second (RPS). RPS gives us the measure of how rapidly the platform can process fundamental data units.
The Software
In this benchmark we used a published build of Smile’s release 2022.05 “Tanuki”. Smile CDR is a full featured Healthcare Data Fabric with HL7® FHIR® at its core.
Smile CDR is powered by HAPI FHIR, the world's most popular open source implementation of the HL7 FHIR standard. Although it would require additional effort to configure HAPI FHIR in a functional cluster, the results presented here should also be achievable with HAPI FHIR.
The Environment and Setup
The architecture used for this test is shown in the diagram below. This HIE runs on multiple clouds. In this case, they choose to run performance tests on their newest Cloud Service Provider (CSP), Oracle Cloud Infrastructure (OCI), a second generation CSP.
Infrastructure Environment Specifications
| Autonomous Database OCPU count: 16 OCPU auto scaling: Enabled Storage: 5 TB Storage auto scaling: Enabled (allocated storage: 2.828 TB) Database Version: 19c |
Autonomous Database Network Access Type: Allow secure access from specific IPs and VCNs Access Control List: Enabled Mutual TLS (mTLS) Authentication: Not Required |
| Node Pools: 1 Kubernetes Version: v1.21.5 Image Name: Oracle-Linux-7.9-2022.01.24-0 Shape: VM.Standard.E4.Flex Total Worker Nodes: 10 Amount of Memory (GB): 32 OCPUs: 8 |
Performance Target Performance: Balanced (VPU/GB:10) Target IOPS: 2820 IOPS Target Throughput: 22.56 MB/s |
| Node details Shape: VM.Standard.E4.Flex OCPU count: 8 Network bandwidth (Gbps): 8 Memory (GB): 32 Local disk: Block storage only Node Instance’s Boot Volume size: 47GB |
Cluster’s Load Balancer Information Shape: Flexible Min Bandwidth: 50 Mbps Max Bandwidth: 1000 Mbps |
Test 1: Data Ingestion
In order to demonstrate mass data ingestion, a collection of roughly one million patient records was generated using Synthea, a total of 2.182 TiB of input data.
The FHIR $import (Bulk Import / Bulk Access Implementation Guide) operation was used to ingest the initial data.
This test produced the following results:
- Overall throughput: 22,251 RPS (Resources Per Second)
- Total time elapsed to ingest 2.182 TiB of data: 12.7 hours
- Time to ingest 1 TiB of data (minutes): 349 minutes
This graph illustrates that peak performance is sustained for over 12 hours and, conceivably, forever.
Test 2: Online Transactional Processing
Next, our HIE tested operational use of the data in the system. For this test, an increasing number of concurrent users were added. Each user performed an equal number of the following operations concurrently with read and write operations occurring at the same time.
The source for this test can be found in the class Test06_MixedBag in this github repository.
The operations included in this test were:
- A FHIR search for resources belonging to a given patient
- A FHIR read for a specific resource by ID
- A FHIR update on an existing individual resource
- A FHIR create for a new individual resource
The mean transaction time is shown in the following chart:
System performance shown in this graph is proportional to the number of concurrent users, even after adding 500 users.
In Conclusion…
In this real-world benchmarking example, the Smile FHIR server processed 22,251 resources per second over a sustained period. This is groundbreaking real world performance.
At Smile Digital Health, our FHIR data fabric is already successfully deployed by some of the largest provider and payer organizations, HIEs, software companies and governments around the world. We continuously invest in performance improvements throughout the product which is witnessed through these FHIR benchmarks.
Our benchmarks and features speak for themselves in their ability to perform at scale.
Endnotes:
1. We invite you to explore the functionality of Smile Digital Health here.
