Challenge Feb-2021

Benchmark “Medical Services”                                                        Solutions   Analysis

This challenge is intended to compare the performance of different decision service implementations. It has only one (but large) decision table and a set of JSON input structures. We expect that DM practitioners  will implement this simple decision service using their favorite BR&DM tools and will provide the actual execution time of their solutions.

Problem. A company that successfully used SQL to access their DB-based medical services decided to migrate to the Serverless architecture. The first Decision Microservice they want to implement is called “Determine Medical Service Coverage”. It should determine coverage attributes for various medical services using the decision table:This table has 16,369 rules for various combinations of medical service characteristics used in the Condition columns. Click here to see and save this decision table in CSV format.

Expected Solution. You need to create a RESTful decision service that takes a medical service in JSON format and fills out its coverage attributes like in this example:

We expect that you will download 10 JSON test cases and execute your service against them producing an output similar to this one. Please report the actual average service execution time for these 10 cases (without network overhead).

We encourage the submitters to deploy their implementations on-cloud as a RESTful decision microservice and publish its endpoint URL and the exact JSON request. It would allow anybody to execute your solution from commonly used POSTMAN. Alternatively, you may explain how our readers could install and execute your decision model to check its actual performance.

Please submit your solutions to A summary of all solutions will be placed in the following table:


Brief Description,
(link to detailed description,
submitted by)
Deployed As
(link to an endpoint URL if any)
Time (without networking)
Prolog Implemented using SWI Prolog and deployed with Docker. All sources are available from GitLab. Can be installed and run locally – see Link
Submitted by Matteo Redaelli
Docker container Under 1 millisecond
Camunda Implemented using Camunda DMN Engine and deployed as an AWS Lambda function. You can execute it yourself from POSTMAN – see Link Submitted by Rob Parker AWS Lambda
With an inbound requests cache
235 milliseconds
34 milliseconds
Java Implemented as a simple Java engine specifically created for this problem – see Link
Submitted by Pradeep Venkat
A regular Java program 2.9 milliseconds
Corticon All rules were implemented using Corticon.js Studio that automatically compressed 16,369 rules down to only 1,010 rules and generated an AWS Lambda function – Link. Submitted by Seth Meldon, Sales Engineer, Progress Corticon AWS Lambda  (without any inbound requests cache)
Download Postman Collection
15 milliseconds
IBM ODM decide4AI team (Azahara Talavera, Thomas Follon, Lorena Roa and Facundo López) published a solution using IBM ODM v.8.10.0 – see Link Decision service exposed as HTDS REST Provided 10 test cases were executed in 214 milliseconds

Implemented with OpenRules Decision Manager 8.3.2  – see Link
Submitted by Jacob Feldman

OpenRules API, Docker, AWS Lambda

0.01 milliseconds, 0.12 milliseconds, 0.20 milliseconds

Java A Custom HashMap-based decision table engine in Java – see Link
Submitted by Dr. Bob More
AWS Lambda
0.017 milliseconds 0.020 milliseconds
Drools Drools 7.50 – see Link
Submitted by Dr. Bob More
Docker 0.059 milliseconds

* The submitters are solely responsible for the quality of the provided information about their implementations