LIFE SCIENCES
Clinical data integration creates a trusted, near real-time data environment
Connecting fragmented clinical data through governed integration, automated reconciliation, and human validation to deliver trusted, study-ready data in real time.
Client
Leading glucose management and diabetes care company
Goal
To consolidate clinical data across systems while improving traceability, governance, and the speed at which study-ready data becomes available
Tools and Technologies
Databricks, AWS Kinesis, Apache Spark
Business Challenge
Clinical study information was distributed across multiple applications, including regulatory document management systems and clinical trial management system platforms. With no common identifier linking records across these systems, consolidating and reconciling clinical data was a complex process.
The client needed to bring structured and unstructured data together at scale while maintaining the traceability and governance required for clinical research.
Manual reconciliation across systems was also time-consuming and prone to errors. As a result, study-ready data could take several days to become available, limiting the speed at which teams could access and act on critical information.
Solution
- Iris designed and implemented a Databricks-based data platform on AWS to integrate clinical data from multiple systems into a centralized and governed environment.
- A metadata-driven ingestion framework using a Bronze-Silver-Gold architecture was established to streamline the onboarding of new data sources while maintaining consistent data lineage.
- Iris also built a reconciliation and matching engine that generated ranked, explainable candidate matches across disparate systems. Human-in-the-loop workflows enabled users to review, validate, and approve matched records, providing an additional layer of control and accuracy.
- Enabled secure sharing of approved study, site, and regulatory data through Databricks and Delta Sharing.
Outcomes
The solution helped the client establish a more connected and responsive clinical data environment. Key outcomes included:
- Created a single trusted view of clinical data across multiple systems.
- Reduced data availability time from 3–5 days to near real-time, improving operational responsiveness.
- Reduced manual reconciliation efforts through automated, auditable matching and approval workflows.
- Approved study, site, and regulatory data was then made available to client environments through Delta Sharing, enabling secure, zero-copy data exchange.
