Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 4 additions & 10 deletions content/patterns/mlops-fraud-detection/_index.adoc
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
---
title: MLOps Fraud Detection
date: 2023-11-12
date: 2025-11-20
tier: sandbox
summary: This pattern demonstrates how Red Hat OpenShift Data Science and MLFlow can be used together to build an end-to-end MLOps platform. It demonstrates this using a credit card fraud detection use case.
summary: This pattern demonstrates how Red Hat OpenShift AI can be used to build an end-to-end MLOps platform. It demonstrates this using a credit card fraud detection use case.
rh_products:
- Red Hat OpenShift Container Platform
- Red Hat OpenShift Data Science
- Red Hat OpenShift Data Foundation
- Red Hat OpenShift AI
industries:
- financial services
aliases: /mlops-fraud-detection/
Expand All @@ -17,11 +16,6 @@ links:
arch: https://www.redhat.com/architect/portfolio/architecturedetail?ppid=6
bugs: https://github.com/validatedpatterns/mlops-fraud-detection/issues
feedback: https://docs.google.com/forms/d/e/1FAIpQLScI76b6tD1WyPu2-d_9CCVDr3Fu5jYERthqLKJDUGwqBg7Vcg/viewform
ci: mfd
contributor:
name: Arslan Khan
contact: mailto:[email protected]
git: https://github.com/arslankhanali
---
:toc:
:imagesdir: /images
Expand All @@ -34,4 +28,4 @@ include::modules/mfd-architecture.adoc[leveloffset=+1]
[id="next-steps_mfd-index"]
== Next steps

* link:mfd-getting-started[Deploy the management hub] using Helm.
* link:mfd-getting-started[Install this pattern.]
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
title: Explore this pattern
weight: 20
aliases: /mlops-fraud-detection/mfd-explore-this-pattern/
---
:toc:
:imagesdir: /images
:_content-type: ASSEMBLY
include::modules/comm-attributes.adoc[]

== RHOAI Components

Most of the components installed as part of this pattern are available in the RHOAI console. You can navigate to this page by selecting the link to Red Hat OpenShift AI in the 9-dots menu of the OpenShift console.

.The RHOAI Link
image::/images/mlops-fraud-detection/mfd-rhoai-link.png[]

=== The Kubeflow Pipeline

A Kubeflow pipeline to build and train the fraud detection model is automatically created and run as part of the pattern's installation. You can see more details of this pipeline inside of RHOAI by selecting the Pipelines tab.

.The Pipelines Tab
image::/images/mlops-fraud-detection/mfd-pipelines-tab.png[]

In this tab we can see the fraud-detection pipeline deployed as part of this pattern. To see the specific run that trained the initial model, we can select the Runs tab below the Pipelines tab and select the job-run item.

.The Runs Tab
image::/images/mlops-fraud-detection/mfd-runs-tab.png[]

On this page, we can see a diagram of the pipeline including the 3 major steps: getting the training data, training the model, and uploading the model to Minio. It's possible to check out the logs of any of the stages, like the training stage, where we can see the accuracy changing over each epoch of model training.

.The job-run Pipeline Details
image::/images/mlops-fraud-detection/mfd-job-run-detail.png[]

NOTE: You can find the source code for this pipeline run in the pattern repo at link:https://github.com/validatedpatterns/mlops-fraud-detection/blob/main/src/kubeflow-pipelines/small-model/train_upload_model.yaml[src/kubeflow-pipelines/small-model].

=== Kserve Model Serving

You can see the model deployment inside of RHOAI in the Model Deployments tab.

.The Model Deployment
image::/images/mlops-fraud-detection/mfd-model-deployments.png[]

== The Inferencing App

The pattern installs a simple Gradio frontend to communicate with your fraud detection model. To reach it, click the link in the 9-dots menu in your OpenShift cluster.

.The Inferencing App Link
image::/images/mlops-fraud-detection/mfd-inf-app-link.png[]

You can manually set transaction details in the form on the left but the app also ships with two examples, a fraudulent transaction on top and a non-fraudulent transaction on the bottom.

.Using the Fraud Example
image::/images/mlops-fraud-detection/mfd-inferencing-app.png[]

IMPORTANT: Due to the non-deterministic nature of training the small model, it may not always identify these transactions accurately.

NOTE: The source code for the inferencing app is in the pattern repo at link:https://github.com/validatedpatterns/mlops-fraud-detection/blob/main/src/inferencing-app/app.py[src/inferencing-app].
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ include::modules/mfd-deploying-mfd-pattern.adoc[leveloffset=1]

= Next Steps

* link:../mfd-running-the-demo[Run the demo].
* link:../mfd-explore-this-pattern[Explore this pattern].
16 changes: 0 additions & 16 deletions content/patterns/mlops-fraud-detection/mfd-running-the-demo.adoc

This file was deleted.

22 changes: 11 additions & 11 deletions modules/mfd-about-mlops-fraud-detection.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@
:imagesdir: ../../images

[id="about-mlops-fraud-detection-pattern"]
= About the MLOps Fraud Detection
= About the MLOps Fraud Detection Pattern

NOTE: This pattern has been reworked for a modern RHOAI experience. To see the original pattern, check out the link:https://github.com/validatedpatterns/mlops-fraud-detection/tree/legacy[legacy branch].

MLOps Credit Card Fraud Detection use case::
* Build and train models in RHODS to detect credit card fraud
* Track and store those models with MLFlow
* Serve a model stored in MLFlow using RHODS Model Serving (or MLFlow serving)
* Deploy a model application in OpenShift that runs sends data to the served model and displays the prediction
* Build, train and serve models in RHOAI to detect credit card fraud
* Use Kubeflow pipelines in RHOAI for declarative model building workflows
* Store models in S3-compatible storage with Minio
* Serve ML models using Kserve on RHOAI

+
Background::
Expand All @@ -21,8 +23,9 @@ The model is built on a Credit Card Fraud Detection model, which predicts if a c

== Technology Highlights:
* Event-Driven Architecture
* Data Science on OpenShift
* Model registry using MLFlow
* Data Science on Red Hat OpenShift AI
* Declarative MLOps pipeline with Kubeflow
* ML model serving with Kserve

== Solution Discussion

Expand All @@ -33,7 +36,4 @@ This architecture pattern demonstrates four strengths:
* *Cost Efficiency*: By automating the detection process, AI reduces the need for extensive manual review of transactions, which can be time-consuming and costly.
* *Flexibility and Agility*: An cloud native architecture that supports the use of microservices, containers, and serverless computing, allowing for more flexible and agile development and deployment of AI models. This means faster iteration and deployment of new fraud detection algorithms.

== Demo Video

.Overview of the solution for credit card fraud detection
video::9Yx_XUOMMYI[youtube]
INFO: This pattern is based on the OpenShift AI tutorial for link:https://docs.redhat.com/en/documentation/red_hat_openshift_ai_cloud_service/1/html-single/openshift_ai_tutorial_-_fraud_detection_example/index[fraud detection].
24 changes: 5 additions & 19 deletions modules/mfd-architecture.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,11 @@

Description of each component:

* *Data Set*: The data set contains the data used for training and evaluating the model we will build in this demo.
* *RHODS Notebook*: We will build and train the model using a Jupyter Notebook running in RHODS.
* *MLFlow Experiment tracking*: We use MLFlow to track the parameters and metrics (such as accuracy, loss, etc) of a model training run. These runs can be grouped under different "experiments", making it easy to keep track of the runs.
* *MLFlow Model registry*: As we track the experiment we also store the trained model through MLFlow so we can easily version it and assign a stage to it (for example Staging, Production, Archive).
* *S3 (ODF)*: This is where the models are stored and what the MLFlow model registry interfaces with. We use ODF (OpenShift Data Foundation) according to the MLFlow guide, but it can be replaced with another solution.
* *RHODS Model Serving*: We recommend using RHODS Model Serving for serving the model. It's based on ModelMesh and allows us to easily send requests to an endpoint for getting predictions.
* *Application interface*: This is the interface used to run predictions with the model. In our case, we will build a visual interface (interactive app) using Gradio and let it load the model from the MLFlow model registry.
* *Data Set*: The data set contains the data used for training and evaluating the model we will build in this demo. The dataset is sourced from link:https://github.com/rh-aiservices-bu/fraud-detection/tree/main/data[github.com/rh-aiservices-bu/fraud-detection]
* *Kubeflow Pipeline*: We build, train and upload the model from a Kubeflow pipeline. The source for this pipeline is included in the pattern repository at link:https://github.com/validatedpatterns/mlops-fraud-detection/blob/main/src/kubeflow-pipelines/small-model/train_upload_model.yaml[src/kubeflow-pipelines/small-model/train_upload_model.yaml]. This pipeline is automatically run once upon pattern installation to train the initial model.
* *S3 (Minio)*: This is where the models are stored and what the Kubeflow pipeline interfaces with. We use Minio for convenience and parity with the source tutorial but any S3-compatible storage solution would work as well.
* *Kserve Model Serving*: We use the Kserve model serving capabilities built into RHOAI to serve our models with an OpenVINO model server.
* *Application interface*: This is the interface used to run predictions with the model. In our case, we will build a visual interface (interactive app) using Gradio and let it load the model from Minio.

//figure 1 originally
.Overview of the solution reference architecture
Expand All @@ -39,15 +37,3 @@ Red Hat® OpenShift® AI is an AI-focused portfolio that provides tools to train

https://www.redhat.com/en/technologies/cloud-computing/openshift/try-it[Red Hat OpenShift GitOps]::
A declarative application continuous delivery tool for Kubernetes based on the ArgoCD project. Application definitions, configurations, and environments are declarative and version controlled in Git. It can automatically push the desired application state into a cluster, quickly find out if the application state is in sync with the desired state, and manage applications in multi-cluster environments.

https://www.redhat.com/en/technologies/jboss-middleware/amq[Red Hat AMQ Streams]::
Red Hat AMQ streams is a massively scalable, distributed, and high-performance data streaming platform based on the Apache Kafka project. It offers a distributed backbone that allows microservices and other applications to share data with high throughput and low latency. Red Hat AMQ Streams is available in the Red Hat AMQ product.

Hashicorp Vault (community)::
Provides a secure centralized store for dynamic infrastructure and applications across clusters, including over low-trust networks between clouds and data centers.

MLFlow Model Registry (community)::
A centralized model store, set of APIs, and UI, to collaboratively manage the full lifecycle of an MLflow Model. It provides model lineage (which MLflow experiment and run produced the model), model versioning, model aliasing, model tagging, and annotations.

Other::
This solution also uses a variety of _observability tools_ including the Prometheus monitoring and Grafana dashboard that are integrated with OpenShift as well as components of the Observatorium meta-project which includes Thanos and the Loki API.
152 changes: 5 additions & 147 deletions modules/mfd-deploying-mfd-pattern.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,26 +6,17 @@

== Prerequisites

. An OpenShift cluster (Go to https://console.redhat.com/openshift/create[the OpenShift console]). Cluster must have a dynamic StorageClass to provision PersistentVolumes.
. An OpenShift cluster (Go to https://console.redhat.com/openshift/create[the OpenShift console]). Cluster must have a dynamic StorageClass to provision PersistentVolumes.
// See also link:../../mlops-fraud-detection/cluster-sizing[sizing your cluster].
. A GitHub account (and a token for it with repositories permissions, to read from and write to your forks)

For installation tooling dependencies, see link:https://validatedpatterns.io/learn/quickstart/[Patterns quick start].

The use of this pattern depends on having a Red Hat OpenShift cluster. In this version of the validated pattern
there is no dedicated Hub / Edge cluster for the *MLOps Fraud Detection* pattern. This single node pattern can be extend as a managed cluster(s) to a central hub.
// See link:../../mlops-fraud-detection/ideas-for-customization[ideas for customization.]

If you do not have a running Red Hat OpenShift cluster you can start one on a
public or private cloud by using link:https://console.redhat.com/openshift/create[Red Hat's cloud service].

[id="utilities"]
= Utilities

A number of utilities have been built by the validated patterns team to lower the barrier to entry for using the community or Red Hat Validated Patterns. To use these utilities you will need to export some environment variables for your cloud provider:

[id="preparation"]
= Preparation
[id="installation"]
= Installing this pattern

. Fork the link:https://github.com/validatedpatterns/mlops-fraud-detection[mlops-fraud-detection] repo on GitHub. It is necessary to fork because your fork will be updated as part of the GitOps and DevOps processes.
. Clone the forked copy of this repository.
Expand All @@ -35,68 +26,6 @@ A number of utilities have been built by the validated patterns team to lower th
git clone [email protected]:<your-username>/mlops-fraud-detection.git
----

. Create a local copy of the Helm secrets values file that can safely include credentials
+
*DO NOT COMMIT THIS FILE*
+
You do not want to push credentials to GitHub.
+
[source,terminal]
----
cp values-secret-mlops-fraud-detection.yaml.template ~/values-secret.yaml
vi ~/values-secret.yaml
----

*values-secret.yaml example*

[source,yaml]
----
secrets:
//Nothing at time of writing.
----

When you edit the file you can make changes to the various DB and Grafana passwords if you wish.

. Customize the `values-global.yaml` for your deployment
+
[source,terminal]
----
git checkout -b my-branch
vi values-global.yaml
----

*Replace instances of PROVIDE_ with your specific configuration*

[source,yaml]
----
global:
pattern: mlops-fraud-detection
hubClusterDomain: "AUTO" # this is for test only This value is automatically fetched when Invoking against a cluster

options:
useCSV: false
syncPolicy: Automatic
installPlanApproval: Automatic

main:
clusterGroupName: hub
----

[source,terminal]
----
git add values-global.yaml
git commit values-global.yaml
git push origin my-branch
----

. You can deploy the pattern using the link:/infrastructure/using-validated-pattern-operator/[validated pattern operator]. If you do use the operator then skip to Validating the Environment below.
. Preview the changes that will be made to the Helm charts.
+
[source,terminal]
----
./pattern.sh make show
----

. Login to your cluster using oc login or exporting the KUBECONFIG
+
[source,terminal]
Expand All @@ -111,82 +40,11 @@ oc login
export KUBECONFIG=~/my-ocp-env/auth/kubeconfig
----

[id="check-the-values-files-before-deployment-getting-started"]
== Check the values files before deployment

You can run a check before deployment to make sure that you have the required variables to deploy the
MLOps Fraud Detection Validated Pattern.

You can run `make predeploy` to check your values. This will allow you to review your values and changed them in
the case there are typos or old values. The values files that should be reviewed prior to deploying the
MLOps Fraud Detection Validated Pattern are:

|===
| Values File | Description

| values-secret.yaml / values-secret-mlops-fraud-detection.yaml
| This is the values file that will include the rhpam and fhir-psql-db sections with all database et al secrets

| values-global.yaml
| File that is used to contain all the global values used by Helm
|===

= Deploy

. Apply the changes to your cluster
. Install this pattern on your cluster
+
[source,terminal]
----
./pattern.sh make install
----
+
If the install fails and you go back over the instructions and see what was missed and change it, then run `make update` to continue the installation.

. This takes some time. Especially for the OpenShift Data Foundation operator components to install and synchronize. The `make install` provides some progress updates during the install. It can take up to twenty minutes. Compare your `make install` run progress with the following video showing a successful install.

. Check that the operators have been installed in the UI.
.. To verify, in the OpenShift Container Platform web console, navigate to *Operators → Installed Operators* page.
.. Check that the Operator is installed in the `openshift-operators` namespace and its status is `Succeeded`.

[id="using-openshift-gitops-to-check-on-application-progress-getting-started"]
== Using OpenShift GitOps to check on Application progress

You can also check on the progress using OpenShift GitOps to check on the various applications deployed.

. Obtain the ArgoCD URLs and passwords.
+
The URLs and login credentials for ArgoCD change depending on the pattern
name and the site names they control. Follow the instructions below to find
them, however you choose to deploy the pattern.
+
Display the fully qualified domain names, and matching login credentials, for
all ArgoCD instances:
+
[source,terminal]
----
ARGO_CMD=`oc get secrets -A -o jsonpath='{range .items[*]}{"oc get -n "}{.metadata.namespace}{" routes; oc -n "}{.metadata.namespace}{" extract secrets/"}{.metadata.name}{" --to=-\\n"}{end}' | grep gitops-cluster`
CMD=`echo $ARGO_CMD | sed 's|- oc|-;oc|g'`
eval $CMD
----
+
The result should look something like:
+
[,text]
----
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
hub-gitops-server hub-gitops-server-mlops-fraud-detection-hub.apps.mfd-cluster.aws.validatedpatterns.com hub-gitops-server https passthrough/Redirect None
# admin.password
xsyYU6eSWtwniEk1X3jL0c2TGfQgVpDH
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
cluster cluster-openshift-gitops.apps.mfd-cluster.aws.validatedpatterns.com cluster 8080 reencrypt/Allow None
kam kam-openshift-gitops.apps.mfd-cluster.aws.validatedpatterns.com kam 8443 passthrough/None None
openshift-gitops-server openshift-gitops-server-openshift-gitops.apps.mfd-cluster.aws.validatedpatterns.com openshift-gitops-server https passthrough/Redirect None
# admin.password
FdGgWHsBYkeqOczE3PuRpU1jLn7C2fD6
----
+
The most important ArgoCD instance to examine at this point is `mlops-fraud-detection-hub`. This is where all the applications for the pattern can be tracked.

. Check all applications are synchronised. There are thirteen different ArgoCD "applications" deployed as part of this pattern.


The installation will conclude when all the ArgoCD applications installed as part of this pattern are healthy and synced.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.