Intelligent Process Mining using Machine Learning

This blog is brought to you by D.Sc. Teemu Lehto, Vice President, Process Mining at QPR Software. Teemu provides a detailed and technical explanation of what intelligent process mining is and how it utilizes machine learning. 

Machine learning helps organizations better utilize the process mining results for improving processes. Traditional process mining is capable of delivering great visibility to as-is processes, pointing out process problems, bottlenecks, and RPA opportunities. In addition to these core benefits, machine learning algorithms provide a great deal of new information - using the same process mining source data.

This article provides insight into Intelligent Process Mining business benefits and techniques for business and process analysts as well as data scientists. My follow-up article on Preventing Compliance Violations using Intelligent Process Mining shows how these Intelligent Process Mining techniques can be used to prevent compliance violations.

What is Intelligent Process Mining?

Intelligent Process Mining uses machine learning algorithms to add artificial intelligence into traditional process mining. Intelligent Process Mining capabilities can be divided into four categories: 

• Descriptive Process Mining – Discover a deep understanding of what happened in the past.
• Diagnostic Process Mining - Understand why something happened in the past.
• Predictive Process Mining - Predict what is most likely to happen in the future.
• Prescriptive Process Mining -  Recommend actions to prevent future problems.

Descriptive Process Mining - Deep understanding

Process mining is a descriptive method to discover patterns and gain a deep understanding of real-life business operations. The ability to draw a visual process flowchart is based on a statistical analysis of the event data. While process mining flowcharts highlight the most typical process flows, they also allow adding the detail level to capture all infrequent process variations and anomalies.

How does one make descriptive process mining more intelligent using machine learning?

This blog is about adding more intelligence to process mining using machine learning techniques. The following three analyses turn the "traditional" to "intelligent": 

• Process Mining Clustering – Use ML to naturally group similar cases into the same clusters 
• Process Mining Anomaly Detection - Use ML to detect anomaly cases (outliers)
• Process Mining Similarity - Use ML to find similar cases based on an example
  
  

Process Mining Clustering

It is essential to understand your business in order to improve it. Process mining gives you this understanding of the data that you have loaded into the system. The first process mining analyses typically show much more variations than expected, making it difficult to analyze detailed measures like lead times? However, the variations often result from the intended differences in the process - it has been agreed that in certain situations the process is supposed to be conducted in a different way than in others. Clustering helps to divide the cases into groups with similar characteristics so that those groups can be further analyzed separately. 

Benefits and use cases for process mining clustering:

• Understand the similarities and differences of cases based on Case Attributes
• Understand the similarities and differences of cases based on Process Behavior
• Divide & conquer -analysis
• Detect source data quality issues
• Detect unwanted cases
• Detect best-practice cases 
  
  
  

The picture below shows the clustering results of typical Purchase to Pay application data. After a few iterations with the source data, the analyst has requested the machine learning system to divide all cases into two clusters by using the case attribute "PO: Type" and all process mining events.

The results show that indeed the "PO: Type" is the most influential characteristic for building these two clusters such that all 53% of cases having the PO type as "PO - Standard" are assigned to Cluster 1 and the other 47% to the cluster with "PO - Shopping Cart" cases. As seen in the following rows, the typical "PO - Standard" cases experience events "Vendor confirmation", "PR Created" and "Invoice Block deactivated" compared top cluster 2 cases experiencing "Shopping Cart Approved", "Shopping Cart Created" and "Review Man." Based on these results it might be a good idea to analyze separately the cases in cluster 1 and cluster 2.  

Using the clustering analysis in QPR ProcessAnalyzer is simple - the default configuration immediately groups process mining cases into five clusters and shows the five most important characteristics for each cluster!

This helps the analyst find similar cases within a large process mining model and thus easily separates the apples from the oranges. Clustering analysis utilizes the full case history (ie. the event types visited) as well as business data (ie. the case attributes) in a unique combined way to identify similarities within cases. QPR Process Mining Clustering Analysis takes process mining to a new level, harnessing the power of machine learning in a super easy-to-use and fast format for all analysts and business people. My recommendation is to load your own data and see the results yourself - it only takes a few seconds in QPR ProcessAnalyzer Cloud!

Diagnostic Process Mining - Root Causes

Why did something happen in the past?

After detecting a process problem, you want to know why it happened. Diagnostic process mining helps you find the answer by looking at the full process mining data available in the model. The main functionalities in this area are: 

• Root Cause analysis – Find the root causes for any identified process problems
• Problem classification - Use machine learning algorithms to classify problems
• Trend analysis - Understand how the process has changed 

Root Cause analysis

Root Cause analysis shows the problematic areas with more than average nonconforming cases in red color and the best practice colors in blue color.

From the results below, we easily see that Region: Chicago is the biggest problem with 84% of cases non-conforming with design. Other problem areas include Account Manager Mary Wilson (70%) and Product Group Hats (60%). On the other hand, we also discover best practice areas when the conformance rate is very good - for example, in Region New York only 35% of cases are non-conforming. The Supplier Jeans International is relatively small with only 595 cases but the conformance rate of 99% is superior showing that it is indeed possible to execute the process as designed - maybe the people in charge if this supplier can tell us more about how they did it! :) 

Predictive Process Mining - Case-level predictions

The previous two steps of intelligent process mining have shown how to detect problems (descriptive process mining) and understand why did they happen (diagnostic process mining). In this third step, we want to predict future problems using the current as-is information in our process mining model extracted from the ERP systems.

Predictive process mining is a method to predict what will happen next in any given ongoing case. It is possible to make these predictions since the process morning model already includes full details of each completed case. By using this information the machine learning system can predict - in a very similar way as a human expert would predict - the outcome of each case. The more data there is, the better the accuracy of the prediction will be. Different predictive process mining scenarios include: 

• Predict KPI outcome - Use ML to predict the outcome of any KPI for all ongoing cases
• Predict the next event - Use ML to predict the next event for all ongoing cases
• Predict final outcome - Use ML to predict all future events and KPIs for all ongoing cases
  
  

Case-level Predictions

Case-level Predictions allow:

• Predict the outcome of an open case like sales order, purchase order, service ticket.
• Predict the success of on-time delivery or meeting the service level agreement.
• React in time to cases that seem to fail their promise and fix the issue before your customer gets disappointed.

Case-level predictions are based on the integrated super-fast in-memory machine learning library. Results are shown using existing QPR ProcessAnalyzer dashboarding and analysis functionality.

How to use the prediction with the existing model?

The picture above shows a dashboard with predictions for On-Time Delivery (OTD). 15.2% of the currently ongoing cases are predicted to be delivered late, according to the machine learning model that is trained using the already completed customer orders. Further analysis is available using the trend and benchmarking charts. 

Prescriptive Process Mining - Intelligent Orchestrator

How to recommend actions to prevent future problems?

The previous three steps of intelligent process mining have shown how to 1. detect problems (descriptive process mining), 2. understand why did they happen (diagnostic process mining) and 3. predict what is going to happen in each open case. As soon as we have the prediction, we can start the work to affect the actual outcome of each case: prevent the problems and utilize opportunities.  

Prescriptive process mining introduces an intelligent orchestrator to help you to succeed in your business operations. You may listen to the intelligent orchestrator's advice, follow the advice if it sounds good, ignore the advice if you know more background information than the system, and train the machine learning-based orchestrator to become an even better companion for running your operations.

Typical jobs for an Intelligent Orchestrator include:

• Send email notifications - notify business people about situations where an existing business rule is violated
• Start RPA bots
• Start new business workflows
• Update data in ERP systems

The screenshot above from QPR ProcessAnalyzer shows an example of the Intelligent Orchestrator in action. Each ongoing case is shown in one row with the customer name, order number, and other details. The column OTD Prediction shows the machine learning model-based prediction for on-time delivery. The last event shows the latest event that has taken occurred for the case. The last column - Next Orchestrated Action - show the suggested next action with a hyperlink to perform the action itself. Some actions, such as "Email Payment Reminder", launch an email system to send an email to customers. "Manage order" is a link to the ERP system for manually making changes to the customer order. the "Start RPA - Remove Delivery Block" starts an RPA bot to remove the delivery block from SAP. 

Intelligent Orchestrator is a key component in the QPR Predict and Act system where AI and business rules are used to make the predictions and discover business rule violations, and orchestrator then prevents and mitigates the emerging problems in time.

QPR integrates with the ERP systems, AI frameworks, and RPA/Workflow systems to orchestrate the whole process.  

What do you think? Want to learn more about Intelligent Process mining?

I am personally super happy to talk more about intelligent process mining so feel free to book a 30-minutes meeting using this link.