Benchmark “Medical Services” Solutions Analysis
This challenge is intended to compare the performance of different decision service implementations. It has only one (but large) decision table and a set of JSON input structures. We expect that DM practitioners will implement this simple decision service using their favorite BR&DM tools and will provide the actual execution time of their solutions.
Problem. A company that successfully used SQL to access their DB-based medical services decided to migrate to the Serverless architecture. The first Decision Microservice they want to implement is called “Determine Medical Service Coverage”. It should determine coverage attributes for various medical services using the decision table:
This table has 16,369 rules for various combinations of medical service characteristics used in the Condition columns. Click here to see and save this decision table in CSV format.
Expected Solution. You need to create a RESTful decision service that takes a medical service in JSON format and fills out its coverage attributes like in this example:
We expect that you will download 10 JSON test cases and execute your service against them producing an output similar to this one. Please report the actual average service execution time for these 10 cases (without network overhead).
We encourage the submitters to deploy their implementations on-cloud as a RESTful decision microservice and publish its endpoint URL and the exact JSON request. It would allow anybody to execute your solution from commonly used POSTMAN. Alternatively, you may explain how our readers could install and execute your decision model to check its actual performance.
Please submit your solutions to DecisionManagementCommunity@gmail.com. A summary of all solutions will be placed in the following table:
| Tool, Version |
Brief Description, (link to detailed description, submitted by) |
Deployed As (link to an endpoint URL if any) |
Average Execution Time (without networking) |
|---|---|---|---|
| Prolog | Implemented using SWI Prolog and deployed with Docker. All sources are available from GitLab. Can be installed and run locally – see Link Submitted by Matteo Redaelli |
Docker container | Under 1 millisecond |
| Camunda | Implemented using Camunda DMN Engine and deployed as an AWS Lambda function. You can execute it yourself from POSTMAN – see Link Submitted by Rob Parker | AWS Lambda —– With an inbound requests cache |
235 milliseconds —– 34 milliseconds |
| Java | Implemented as a simple Java engine specifically created for this problem – see Link Submitted by Pradeep Venkat |
A regular Java program | 2.9 milliseconds |
| Corticon | All rules were implemented using Corticon.js Studio that automatically compressed 16,369 rules down to only 1,010 rules and generated an AWS Lambda function – Link. Submitted by Seth Meldon, Sales Engineer, Progress Corticon | AWS Lambda (without any inbound requests cache) Download Postman Collection |
15 milliseconds |
| IBM ODM | decide4AI team (Azahara Talavera, Thomas Follon, Lorena Roa and Facundo López) published a solution using IBM ODM v.8.10.0 – see Link | Decision service exposed as HTDS REST | Provided 10 test cases were executed in 214 milliseconds |
| OpenRules |
Implemented with OpenRules Decision Manager 8.3.2 – see Link |
OpenRules API, Docker, AWS Lambda |
0.01 milliseconds, 0.12 milliseconds, 0.20 milliseconds |
| Java | A Custom HashMap-based decision table engine in Java – see Link Submitted by Dr. Bob More |
Docker AWS Lambda |
0.017 milliseconds 0.020 milliseconds |
| Drools | Drools 7.50 – see Link Submitted by Dr. Bob More |
Docker | 0.059 milliseconds |
| … | … | … | … |
* The submitters are solely responsible for the quality of the provided information about their implementations
dmcommunity.org 
You must be logged in to post a comment.