diff --git a/Recipes/ClearScape_Functions/KMeans_KMeansPredict.ipynb b/Recipes/ClearScape_Functions/KMeans_KMeansPredict.ipynb new file mode 100644 index 00000000..14c8641a --- /dev/null +++ b/Recipes/ClearScape_Functions/KMeans_KMeansPredict.ipynb @@ -0,0 +1,419 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "bc549e6c-0cc4-4188-94a3-a9bdd3ae3dfa", + "metadata": {}, + "source": [ + "
\n", + "

\n", + " KMeans and KMeansPredict Functions in Vantage\n", + "
\n", + " \"Teradata\"\n", + "

\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "7ae7611a-0795-4168-b716-01fee6880cbd", + "metadata": {}, + "source": [ + "

Introduction

\n", + "

The K-means() function groups a set of observations into k clusters in which each observation belongs to the cluster with the nearest mean (cluster centers or cluster centroid). KMeansPredict uses the k-means algorithm to predict the target class of unseen or new data. In this notebook we will see how we can use the KMeans and KMeansPredict function available in Vantage.

" + ] + }, + { + "cell_type": "markdown", + "id": "6b3a00b4-6661-4c91-9b2d-cb7b0b403140", + "metadata": {}, + "source": [ + "
\n", + "1. Initiate a connection to Vantage" + ] + }, + { + "cell_type": "markdown", + "id": "2346857f-e0d3-488a-8a3f-ac6dff752c2b", + "metadata": {}, + "source": [ + "

In the section, we import the required libraries and set environment variables and environment paths (if required)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c5af5af3-29d5-4f6a-8334-9df6924e7787", + "metadata": {}, + "outputs": [], + "source": [ + "from teradataml import *\n", + "\n", + "# Modify the following to match the specific client environment settings\n", + "display.max_rows = 5" + ] + }, + { + "cell_type": "markdown", + "id": "ad3dd7b4-831c-4fb3-ab71-719c8c99a71c", + "metadata": {}, + "source": [ + "

\n", + "

1.1 Connect to Vantage

\n", + "

You will be prompted to provide the password. Enter your password, press the Enter key, and then use the down arrow to go to the next cell.

" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2742444c-4349-4b0f-b4e5-b068a8785cd9", + "metadata": {}, + "outputs": [], + "source": [ + "%run -i ../../UseCases/startup.ipynb\n", + "eng = create_context(host = 'host.docker.internal', username='demo_user', password = password)\n", + "print(eng)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "01c4a128-d106-46ea-8dee-34acc5abd29f", + "metadata": {}, + "outputs": [], + "source": [ + "%%capture\n", + "execute_sql('''SET query_band='DEMO=PP_KMeans_KMeansPredict_Python.ipynb;' UPDATE FOR SESSION; ''')" + ] + }, + { + "cell_type": "markdown", + "id": "efe2fd2d-63ff-4278-9157-8b9110d682e8", + "metadata": {}, + "source": [ + "

Begin running steps with Shift + Enter keys.

" + ] + }, + { + "cell_type": "markdown", + "id": "f003f332-7489-4bdd-a740-4af2a0a22280", + "metadata": {}, + "source": [ + "
\n", + "\n", + "

1.2 Getting Data for This Demo

\n", + "\n", + "

Here, we will get the data which is available in the teradataml library and use the same to show the usage of the function.

" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "45c86176-734c-4b1c-ace0-d0c88657b4f8", + "metadata": {}, + "outputs": [], + "source": [ + "load_example_data(\"kmeans\", \"computers_train1\")" + ] + }, + { + "cell_type": "markdown", + "id": "2401d6d3-4fcd-46fc-8a94-7cafcd1258b0", + "metadata": {}, + "source": [ + "

Next is an optional step – if you want to see the status of databases/tables created and space used.

" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "87429200-db02-450d-9472-4d1e2030124d", + "metadata": {}, + "outputs": [], + "source": [ + "%run -i ../../UseCases/run_procedure.py \"call space_report();\" # Takes 10 seconds" + ] + }, + { + "cell_type": "markdown", + "id": "2a3762ac-ba27-4fa3-adba-d577262a4290", + "metadata": {}, + "source": [ + "
\n", + "2. Data Exploration\n", + "

Create a \"Virtual DataFrame\" that points to the data set in Vantage. Check the shape of the dataframe as check the datatype of all the columns of the dataframe.

" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3d936fab-7ca7-4e94-ba64-95c1da08b74f", + "metadata": {}, + "outputs": [], + "source": [ + "# Create teradataml DataFrame objects.\n", + "computers_train = DataFrame.from_table(\"computers_train1\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3c5a0992-f651-49bc-9080-828fd9c0c982", + "metadata": {}, + "outputs": [], + "source": [ + "computers_train" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "96070c57-9a6b-451d-aa05-4ea3825a4bff", + "metadata": {}, + "outputs": [], + "source": [ + "computers_train.shape" + ] + }, + { + "cell_type": "markdown", + "id": "0a70cf26-0b67-4acd-a4a1-28881016ca39", + "metadata": {}, + "source": [ + "
\n", + "

2.1 KMeans Function

" + ] + }, + { + "cell_type": "markdown", + "id": "ffef7ff5-0fad-4e47-a415-a1d58e0b37fd", + "metadata": {}, + "source": [ + "

We want to divide our data into two clusters, we will use KMeans function for this.
\n", + "Detailed help can be found by passing function name to built-in help function.

" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "703f9e25-6785-4f91-a22b-3bc491b046a2", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "help(KMeans)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7de768e5-826f-499e-95f3-4d231d4c1c7b", + "metadata": {}, + "outputs": [], + "source": [ + "KMeans_out = KMeans(id_column=\"id\",\n", + " target_columns=['price', 'speed'],\n", + " data=computers_train,\n", + " num_clusters=2)\n", + "# Print the result DataFrame\n", + "KMeans_out.result" + ] + }, + { + "cell_type": "markdown", + "id": "8bddd414-903c-4c64-8f67-2323d332908b", + "metadata": {}, + "source": [ + "

We can also specify initial centroid information instead of number of clusters.

" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f042acff-500e-46f4-9f5c-437476aa3cc7", + "metadata": {}, + "outputs": [], + "source": [ + "kmeans_initial_centroids_table = computers_train.loc[[19, 97]]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e07260ab-61a3-45a0-9536-c4df4971d7ed", + "metadata": {}, + "outputs": [], + "source": [ + "kmeans_initial_centroids_table" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8675e314-4b55-4265-8d78-9ac8e3c9661e", + "metadata": {}, + "outputs": [], + "source": [ + "KMeans_out_1 = KMeans(id_column=\"id\",\n", + " target_columns=['price', 'speed'],\n", + " data=computers_train,\n", + " centroids_data=kmeans_initial_centroids_table)\n", + " \n", + " # Print the result DataFrames.\n", + "KMeans_out_1.result" + ] + }, + { + "cell_type": "markdown", + "id": "49fbbae3-aecb-4237-85ed-9586cbdc1a4f", + "metadata": {}, + "source": [ + "
\n", + "

2.2 KMeansPredict Function

" + ] + }, + { + "cell_type": "markdown", + "id": "ab6a9a63-7f1e-4964-b3f5-bfedd7a4c28d", + "metadata": {}, + "source": [ + "

We can assign the input data points to the cluster centroid using the model generated by the KMeans() function in KMeansPredict() function.
Detailed help can be found by passing function name to built-in help function.

\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5c092ed4-c489-43a7-bce0-ffaf2a974539", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "help(KMeansPredict)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fedfa443-e84e-4eb0-8517-cfba00371401", + "metadata": {}, + "outputs": [], + "source": [ + "KMeansPredict_out = KMeansPredict(data=computers_train,\n", + " object=KMeans_out.result\n", + " )\n", + " \n", + "# Print the result DataFrames.\n", + "KMeansPredict_out.result" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1a0adb0e-a721-43e3-842d-19dfa47d3ab7", + "metadata": {}, + "outputs": [], + "source": [ + "KMeansPredict_out_1 = KMeansPredict(data=computers_train,\n", + " object=KMeans_out_1,\n", + " accumulate=[\"ram\",\"price\",\"speed\"],\n", + " output_distance=True\n", + " )\n", + " \n", + "# Print the result DataFrames.\n", + "KMeansPredict_out_1.result" + ] + }, + { + "cell_type": "markdown", + "id": "151d5db4-29a9-49d9-8a61-d53f9627a294", + "metadata": {}, + "source": [ + "
\n", + "3. Cleanup" + ] + }, + { + "cell_type": "markdown", + "id": "a562f058-fb24-4966-a25d-f2960e6ddfb8", + "metadata": {}, + "source": [ + "
\n", + "

Databases and Tables

\n", + "

The following code will clean up tables and databases created above.

" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e6b3935b-47c2-4a96-bec2-68106d172116", + "metadata": {}, + "outputs": [], + "source": [ + "db_drop_table(\"computers_train1\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "157fe3d4-4e0e-4d92-b343-9f758f3bf690", + "metadata": {}, + "outputs": [], + "source": [ + "remove_context()" + ] + }, + { + "cell_type": "markdown", + "id": "4317a6cf-1479-4aa8-b30a-ee0a3b5231a8", + "metadata": {}, + "source": [ + "
\n", + "\n", + "

Links:

\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "b2dcca28-5de5-44d7-88cb-45a12153b3f8", + "metadata": {}, + "source": [ + "" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.10" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}