ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo
Using ATL/EMFTVM for import/export of medical data 
8-Oct-2014, Dennis Wagelaar
2
Using ATL/EMFTVM for import/export of medical data - #sda2014
Import/export 
a common programming scenario 
SuMEHR 
GPSMF 
PMF 
Corilus 
XML 
SuMEHR 
GPSMF 
PMF 
Corilus 
XML 
4
Corilus XML as pivot model 
SuMEHR 
GPSMF 
PMF 
Corilus 
XML 
5
Corilus XML as pivot model 
SuMEHR 
GPSMF 
PMF 
Corilus 
XML 
6
Import/export 
simplified 
Corilus 
XML 
Corilus 
XML 
7
Why ATL? 
(ATL Transformation Language) 
Domain-specific 
language for 
transformation 
More expressive than 
mapping frameworks 
embedded in Java, e.g. Dozer 
Less verbose for 
transformations than general-purpose 
languages 
Uses OCL standard for 
expressions 
Uses EMF for data 
representation 
Closely related to plain Java 
objects 
Enriched with additional 
concepts, e.g. containment 
and associated properties 
8
Why EMFTVM? 
(EMF Transformation Virtual Machine) 
Enhanced for ¡°online¡± 
use (performance) 
Reuse pre-loaded 
transformations for 
multiple executions 
JIT compiler translates to 
Java bytecode 
Adaptive matching 
algorithm adds 
performance 
Improved modularity 
Supports multiple rule 
inheritance across 
different modules 
Supports module import 
across different source 
languages 
9
Why EMFTVM? 
(EMF Transformation Virtual Machine) 
10 
https://wiki.eclipse.org/ATL/EMFTVM
EMF vs. POJOs 
(JPA) 
Uses Uses 
EMF Models Plain Old Java Objects 
11
EMF vs. POJOs 
This is what we have... 
(JPA) 
Uses 
Plain Old Java Objects 
12
From POJOs to EMF 
MoDisco and EMiFy 
Java Model Reverse Engineer Plain Old Java Objects 
13
From POJOs to EMF 
MoDisco and EMiFy 
Java Model 
Ecore Model 
EMiFy.atl 
14
From POJOs to EMF 
MoDisco and EMiFy 
Ecore Model 
Generate Model 
EMF Java Objects 
15
From POJOs to EMF 
MoDisco and EMiFy 
Java Model 
Ecore Model 
EMiFy.atl 
Reverse Engineer 
Generate Model 
EMF Java Objects 
16
Using ATL 
17
Using ATL 
18
Using ATL 
Implicit tracing 
19
Using ATL 
Implicit tracing 
20
Using ATL 
Collaboration 
21 
Java developer Java developer 
Common: 178 lines of ATL code 
Import: 7283 lines of ATL code 
Export: 4617 lines of ATL code 
.NET developer Java developer 
4906 lines of Java 
import/export code
Runtime performance 
Medium-large patient file 
(2277 record entries) 
22 
03-10-2014 | 13:57:19,223 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.rs.ImportResource | Import partial document 504 
03-10-2014 | 13:57:19,307 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Start full import for ... 
03-10-2014 | 13:57:20,006 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Starting transformation... 
03-10-2014 | 13:57:21,099 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.emftvm.impl.AbstractEMFTVMTransformer | Timing data: 
Loading finished at 0,000036 seconds (duration: 0,000036 seconds) 
Matching finished at 0,285989 seconds (duration: 0,285953 seconds) 
Applying finished at 1,091511 seconds (duration: 0,805522 seconds) 
Post-applying finished at 1,091712 seconds (duration: 0,000201 seconds) 
Recursive stage finished at 1,091751 seconds (duration: 0,000039 seconds) 
Execution finished at 1,091915 seconds (duration: 0,000203 seconds) 
03-10-2014 | 13:57:21,118 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 1 Patient instances 
03-10-2014 | 13:57:21,155 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 3 ExternalUser instances 
03-10-2014 | 13:57:21,156 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 1 HealthInsurance instances 
03-10-2014 | 13:57:21,161 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 138 Contact instances 
03-10-2014 | 13:57:21,282 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 33 Episode instances 
03-10-2014 | 13:57:21,316 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 33 HealthApproach instances 
03-10-2014 | 13:57:21,349 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 553 SubContact instances 
03-10-2014 | 13:57:21,925 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 2 ContactPerson instances 
03-10-2014 | 13:57:23,240 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 1 Vaccination instances 
03-10-2014 | 13:57:23,255 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 134 NormalMedicationEntry instances 
03-10-2014 | 13:57:23,649 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 4 MagistralMedicationEntry instances 
03-10-2014 | 13:57:23,673 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 138 FreeTextPosology instances 
03-10-2014 | 13:57:24,041 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 224 SOEPRule instances 
03-10-2014 | 13:57:24,473 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 78 Letter instances 
03-10-2014 | 13:57:25,416 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 30 LabResult instances 
03-10-2014 | 13:57:25,495 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 776 LabResultEntry instances 
03-10-2014 | 13:57:28,533 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 51 ParameterGroup instances 
03-10-2014 | 13:57:28,657 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 72 ParameterGroupEntry instances 
03-10-2014 | 13:57:28,992 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 3 SickLeave instances 
03-10-2014 | 13:57:29,023 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 2 Note instances 
03-10-2014 | 13:57:33,992 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.rs.ImportResource | Finished import partial document 504 in 14.768414891 
XML loading 
0,7 
sec 
Transformation 
1,1 
sec 
Hibernate storage 
12,9 
sec
Conclusion 
We tackled a complex and common programming scenario such as import/export 
by breaking it up in three ways: 
23 
Use specialised 
language for 
translating 
between 
domain model 
and pivot model 
Use pivot model 
for import/ 
export => only 
support a single 
import/export 
format 
Use regular Java to 
handle file I/O and 
database interaction 
XML
24 
Questions? 
http://www.healthconnect.be/ 
http://www.corilus.be/ 
http://www.careconnect.be/ 
Questions 
?

More Related Content

Using ATL/EMFTVM for import/export of medical data - #sda2014

  • 1. Using ATL/EMFTVM for import/export of medical data 8-Oct-2014, Dennis Wagelaar
  • 2. 2
  • 4. Import/export a common programming scenario SuMEHR GPSMF PMF Corilus XML SuMEHR GPSMF PMF Corilus XML 4
  • 5. Corilus XML as pivot model SuMEHR GPSMF PMF Corilus XML 5
  • 6. Corilus XML as pivot model SuMEHR GPSMF PMF Corilus XML 6
  • 8. Why ATL? (ATL Transformation Language) Domain-specific language for transformation More expressive than mapping frameworks embedded in Java, e.g. Dozer Less verbose for transformations than general-purpose languages Uses OCL standard for expressions Uses EMF for data representation Closely related to plain Java objects Enriched with additional concepts, e.g. containment and associated properties 8
  • 9. Why EMFTVM? (EMF Transformation Virtual Machine) Enhanced for ¡°online¡± use (performance) Reuse pre-loaded transformations for multiple executions JIT compiler translates to Java bytecode Adaptive matching algorithm adds performance Improved modularity Supports multiple rule inheritance across different modules Supports module import across different source languages 9
  • 10. Why EMFTVM? (EMF Transformation Virtual Machine) 10 https://wiki.eclipse.org/ATL/EMFTVM
  • 11. EMF vs. POJOs (JPA) Uses Uses EMF Models Plain Old Java Objects 11
  • 12. EMF vs. POJOs This is what we have... (JPA) Uses Plain Old Java Objects 12
  • 13. From POJOs to EMF MoDisco and EMiFy Java Model Reverse Engineer Plain Old Java Objects 13
  • 14. From POJOs to EMF MoDisco and EMiFy Java Model Ecore Model EMiFy.atl 14
  • 15. From POJOs to EMF MoDisco and EMiFy Ecore Model Generate Model EMF Java Objects 15
  • 16. From POJOs to EMF MoDisco and EMiFy Java Model Ecore Model EMiFy.atl Reverse Engineer Generate Model EMF Java Objects 16
  • 19. Using ATL Implicit tracing 19
  • 20. Using ATL Implicit tracing 20
  • 21. Using ATL Collaboration 21 Java developer Java developer Common: 178 lines of ATL code Import: 7283 lines of ATL code Export: 4617 lines of ATL code .NET developer Java developer 4906 lines of Java import/export code
  • 22. Runtime performance Medium-large patient file (2277 record entries) 22 03-10-2014 | 13:57:19,223 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.rs.ImportResource | Import partial document 504 03-10-2014 | 13:57:19,307 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Start full import for ... 03-10-2014 | 13:57:20,006 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Starting transformation... 03-10-2014 | 13:57:21,099 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.emftvm.impl.AbstractEMFTVMTransformer | Timing data: Loading finished at 0,000036 seconds (duration: 0,000036 seconds) Matching finished at 0,285989 seconds (duration: 0,285953 seconds) Applying finished at 1,091511 seconds (duration: 0,805522 seconds) Post-applying finished at 1,091712 seconds (duration: 0,000201 seconds) Recursive stage finished at 1,091751 seconds (duration: 0,000039 seconds) Execution finished at 1,091915 seconds (duration: 0,000203 seconds) 03-10-2014 | 13:57:21,118 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 1 Patient instances 03-10-2014 | 13:57:21,155 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 3 ExternalUser instances 03-10-2014 | 13:57:21,156 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 1 HealthInsurance instances 03-10-2014 | 13:57:21,161 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 138 Contact instances 03-10-2014 | 13:57:21,282 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 33 Episode instances 03-10-2014 | 13:57:21,316 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 33 HealthApproach instances 03-10-2014 | 13:57:21,349 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 553 SubContact instances 03-10-2014 | 13:57:21,925 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 2 ContactPerson instances 03-10-2014 | 13:57:23,240 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 1 Vaccination instances 03-10-2014 | 13:57:23,255 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 134 NormalMedicationEntry instances 03-10-2014 | 13:57:23,649 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 4 MagistralMedicationEntry instances 03-10-2014 | 13:57:23,673 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 138 FreeTextPosology instances 03-10-2014 | 13:57:24,041 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 224 SOEPRule instances 03-10-2014 | 13:57:24,473 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 78 Letter instances 03-10-2014 | 13:57:25,416 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 30 LabResult instances 03-10-2014 | 13:57:25,495 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 776 LabResultEntry instances 03-10-2014 | 13:57:28,533 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 51 ParameterGroup instances 03-10-2014 | 13:57:28,657 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 72 ParameterGroupEntry instances 03-10-2014 | 13:57:28,992 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 3 SickLeave instances 03-10-2014 | 13:57:29,023 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 2 Note instances 03-10-2014 | 13:57:33,992 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.rs.ImportResource | Finished import partial document 504 in 14.768414891 XML loading 0,7 sec Transformation 1,1 sec Hibernate storage 12,9 sec
  • 23. Conclusion We tackled a complex and common programming scenario such as import/export by breaking it up in three ways: 23 Use specialised language for translating between domain model and pivot model Use pivot model for import/ export => only support a single import/export format Use regular Java to handle file I/O and database interaction XML
  • 24. 24 Questions? http://www.healthconnect.be/ http://www.corilus.be/ http://www.careconnect.be/ Questions ?

Editor's Notes

  • #4: Most data-centric software must deal with some form of import/export of their internal data model to an external data format. In many cases, this external data format is some sort of standard format, or otherwise dictated by external sources, and does not map one-on-one to the internal data model.
  • #5: This was also the case for?CareConnect,?HealthConnect/Corilus¡® latest?Electronic Medical Recordsoftware.?CareConnect?must be able to import/export its data from/to?SUMEHR, PMF, and?GPSMF?documents. On top of this comes that?CareConnect¡¯s?internal data model consists of some 300 classes, which means there are a lot of mappings to define.
  • #11: EMFTVM performance is roughly 80% better than the default ATL EMF-specific VM. EMFTVM has a JIT-compiler that improves performance of complex code blocks. It also allows for reuse of a pre-loaded VM instance (when invoking from Java), which is useful when invoking the same transformation on different models many times over. Finally, it uses an?adaptive rule matching algorithm?that configures itself against the metamodels and transformation modules used on the first run of the VM. The EcoreUtil.Copier entry is the standard Java implementation for copying Ecore models, and forms the baseline ("it doesn't get faster than this"). On the following lines one can see the evolution in performance of the various ATL VMs.
  • #17: Note that the?MoDisco-EMiFy-EMF?round-trip scenario for generating the Ecore model and EMF reflective methods was performed for each change to our domain model, and takes about 10 minutes each time. Most of this time was taken up by running?MoDisco?and the EMF code generator. This takes up extra time from the developer who manages the domain model changes, besides the regular Java code changes and SQL migration code. The positive side of this picture is that the Ecore domain model can also be used reflectively to do dry-run transformations outside of the application codebase. This allows the ATL developers to test each change in isolation.
  • #18: ATL is a mapping language, which applies its mapping rules top-down for each model element it can find, and uses a two-pass compiler approach to ¡°weave¡± the elements generated by different rules back together. This frees the programmer from dealing with model traversal and rule execution order: this is taken care of by ATL. Instead, the programmer can focus on defining the mapping between specific input and output elements. To give you an idea, here¡¯s what the ATL rule for mapping lab results?looks like.
  • #19: Helpers can be used to encapsulate complex navigations. Helper attributes also improve performance, because their values are cached. They function as a sort of query index over the input data.
  • #20: The references LabResultEntry instances from the LabResult rule are transformed here. This rule takes care of title entries...
  • #21: ...and this rule takes care of the actual lab result values.
  • #22: The implicit tracing mechanism has allowed us to distribute the transformation writing over three to four developers. One of these programmers was trained in .NET instead of Java, which turned out not to be a barrier for writing ATL code. Because we also chose to separate our transformation code from our file I/O and database interaction code, another developer could work on the file I/O and database interaction in Java. Finally, the Corilus XML conversion service for SUMEHR, PMF, and GPSMF was taken care of by a separate development team at Corilus HQ, and was based on existing code. The critical path in the development pipeline was therefore made up by the ATL code, which is exactly the kind of workload that can easily be dispersed over multiple developers.