Update Jupyter notebook examples with dict configs

Author: gustavocidornelas

examples/llms/general-llm/product-names.ipynb

@@ -175,31 +175,17 @@
    "source": [
     "The `openlayer` Python client comes with LLM runners, which are wrappers around common LLMs -- such as OpenAI's. The idea is that these LLM runners adhere to a common interface and can be called to make predictions on pandas dataframes. \n",
     "\n",
-    "To use `openlayer`'s LLM runners, we must follow the steps:\n",
-    "\n",
-    "**1. Create a new directory**\n",
-    "\n",
-    "This directory will house all the configs and files related to the LLM of our choice. Let's call ours `llm_package`:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "ed8f150d",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!mkdir llm_package"
+    "To use `openlayer`'s LLM runners, we must follow the steps:"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "f639ce93",
    "metadata": {},
    "source": [
-    "**2. Write a YAML config file**\n",
+    "**1. Prepare the config**\n",
     "\n",
-    "Now, we can write a YAML config file called `model_config.yaml` to our newly created directory:"
+    "We need to prepare a config for the LLM:"
    ]
   },
   {
@@ -245,8 +231,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import yaml\n",
-    "\n",
     "# Note the camelCase for the keys\n",
     "model_config = {\n",
     "    \"prompt\": prompt,\n",
@@ -256,25 +240,17 @@
     "    \"modelParameters\": {\n",
     "        \"temperature\": 0\n",
     "    },\n",
-    "    \"modelType\": \"api\",\n",
-    "    \"name\": \"Product name suggestor\",\n",
-    "    \"architectureType\": \"llm\",\n",
-    "}\n",
-    "\n",
-    "with open(\"llm_package/model_config.yaml\", \"w\") as model_config_file:\n",
-    "    yaml.dump(model_config, model_config_file, default_flow_style=False)"
+    "}"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "9543123e",
    "metadata": {},
    "source": [
-    "You can check out the details for every field of the `model_config.yaml` file in our documentation. \n",
-    "\n",
     "To highlight a few important fields:\n",
     "- `prompt`: this is the prompt that will get sent to the LLM. Notice that our variables are refered to in the prompt template with double handlebars `{{ }}`. When we make the request, the prompt will get injected with the input variables data from the pandas dataframe. Also, we follow OpenAI's convention with messages with `role` and `content` regardless of the LLM provider you choose.\n",
-    "- `inputVariableNames`: this is a list with the names of the input variables. Each input variable should be a column in the pandas dataframe that we will use. Furthermore, these are the input variables referenced in the `promptTemplate` with the handlebars.\n",
+    "- `inputVariableNames`: this is a list with the names of the input variables. Each input variable should be a column in the pandas dataframe that we will use. Furthermore, these are the input variables referenced in the `prompt` with the handlebars.\n",
     "- `modelProvider`: one of the supported model providers, such as `OpenAI`.\n",
     "- `model`: name of the model from the `modelProvider`. In our case `gpt-3.5-turbo`.\n",
     "- `modelParameters`: a dictionary with the model parameters for that specific `model`. For `gpt-3.5-turbo`, for example, we could specify the `temperature`, the `tokenLimit`, etc."
@@ -285,7 +261,7 @@
    "id": "0d36b925",
    "metadata": {},
    "source": [
-    "**3. Get the model runner**\n",
+    "**2. Get the model runner**\n",
     "\n",
     "Now we can import `models` from `openlayer` and call the `get_model_runner` function, which will return a `ModelRunner` object. This is where we'll pass the OpenAI API key. For a different LLM `modelProvider` you might need to pass a different argument -- refer to our documentation for details."
    ]
@@ -301,8 +277,8 @@
     "\n",
     "llm_runner = models.get_model_runner(\n",
     "    task_type=tasks.TaskType.LLM,\n",
-    "    model_package=\"llm_package\",\n",
-    "    openai_api_key=\"YOUR_OPENAI_API_KEY_HERE\"\n",
+    "    openai_api_key=\"YOUR_OPENAI_API_KEY_HERE\",\n",
+    "    **model_config\n",
     ")"
    ]
   },
@@ -321,7 +297,7 @@
    "id": "ca5d75e5",
    "metadata": {},
    "source": [
-    "**4. Run the LLM to get the predictions**\n",
+    "**3. Run the LLM to get the predictions**\n",
     "\n",
     "Every model runner comes with a `run` method. This method expects a pandas dataframe with the input variables as input and returns a pandas dataframe with a single column: the predictions.\n",
     "\n",
@@ -464,11 +440,11 @@
    "source": [
     "### <a id=\"dataset\">Uploading datasets</a>\n",
     "\n",
-    "Before adding the datasets to a project, we need to do prepare a `dataset_config.yaml` file. \n",
+    "Before adding the datasets to a project, we need to do Prepare a `dataset_config`.  \n",
     "\n",
-    "This is a file that contains all the information needed by the Openlayer platform to utilize the dataset. It should include the column names, the input variable names, etc. For details on the fields of the `dataset_config.yaml` file, see the [API reference](https://reference.openlayer.com/reference/api/openlayer.OpenlayerClient.add_dataset.html#openlayer.OpenlayerClient.add_dataset).\n",
+    "This is a Python dictionary that contains all the information needed by the Openlayer platform to utilize the dataset. It should include the column names, the input variable names, etc. For details on the `dataset_config` items, see the [API reference](https://reference.openlayer.com/reference/api/openlayer.OpenlayerClient.add_dataset.html#openlayer.OpenlayerClient.add_dataset).\n",
     "\n",
-    "Let's prepare the `dataset_config.yaml` files for our validation set:"
+    "Let's prepare the `dataset_config` for our validation set:"
    ]
   },
   {
@@ -478,8 +454,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Some variables that will go into the `dataset_config.yaml` file\n",
-    "column_names = list(dataset.columns)\n",
+    "# Some variables that will go into the `dataset_config`\n",
     "input_variable_names = [\"description\", \"seed_words\"]\n",
     "output_column_name = \"model_output\""
    ]
@@ -491,17 +466,11 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import yaml \n",
-    "\n",
     "validation_dataset_config = {\n",
-    "    \"columnNames\": column_names,\n",
     "    \"inputVariableNames\": input_variable_names,\n",
     "    \"label\": \"validation\",\n",
     "    \"outputColumnName\": output_column_name,\n",
-    "}\n",
-    "\n",
-    "with open(\"validation_dataset_config.yaml\", \"w\") as dataset_config_file:\n",
-    "    yaml.dump(validation_dataset_config, dataset_config_file, default_flow_style=False)"
+    "}"
    ]
   },
   {
@@ -514,7 +483,7 @@
     "# Validation set\n",
     "project.add_dataframe(\n",
     "    dataset_df=dataset,\n",
-    "    dataset_config_file_path=\"validation_dataset_config.yaml\",\n",
+    "    dataset_config=validation_dataset_config,\n",
     ")"
    ]
   },
@@ -563,7 +532,7 @@
     "\n",
     "Note that to use a direct-to-API model on the platform, you'll need to **provide your model provider's API key (such as the OpenAI API key) using the platform's UI**, under the project settings.\n",
     "\n",
-    "Since we used an LLM runner in this notebook, we already wrote a model config YAML file. We will write it again just for completeness:"
+    "Since we used an LLM runner in this notebook, we already wrote a model config for the LLM. We'll write it again for completeness:"
    ]
   },
   {
@@ -573,8 +542,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import yaml\n",
-    "\n",
     "# Note the camelCase for the keys\n",
     "model_config = {\n",
     "    \"prompt\": prompt,\n",
@@ -585,12 +552,7 @@
     "        \"temperature\": 0\n",
     "    },\n",
     "    \"modelType\": \"api\",\n",
-    "    \"name\": \"Product name suggestor\",\n",
-    "    \"architectureType\": \"llm\",\n",
-    "}\n",
-    "\n",
-    "with open(\"llm_package/model_config.yaml\", \"w\") as model_config_file:\n",
-    "    yaml.dump(model_config, model_config_file, default_flow_style=False)"
+    "}"
    ]
   },
   {
@@ -602,7 +564,7 @@
    "source": [
     "# Adding the model\n",
     "project.add_model(\n",
-    "    model_config_file_path=\"llm_package/model_config.yaml\",\n",
+    "    model_config=model_config,\n",
     ")"
    ]
   },

examples/llms/langchain/question-answering.ipynb

@@ -402,11 +402,11 @@
    "source": [
     "### <a id=\"dataset\">Uploading datasets</a>\n",
     "\n",
-    "Before adding the datasets to a project, we need to do prepare a `dataset_config.yaml` file. \n",
+    "Before adding the datasets to a project, we need to do Prepare a `dataset_config`.  \n",
     "\n",
-    "This is a file that contains all the information needed by the Openlayer platform to utilize the dataset. It should include the column names, the input variable names, etc. For details on the fields of the `dataset_config.yaml` file, see the [API reference](https://reference.openlayer.com/reference/api/openlayer.OpenlayerClient.add_dataset.html#openlayer.OpenlayerClient.add_dataset).\n",
+    "This is a Python dictionary that contains all the information needed by the Openlayer platform to utilize the dataset. It should include the column names, the input variable names, etc. For details on the `dataset_config` items, see the [API reference](https://reference.openlayer.com/reference/api/openlayer.OpenlayerClient.add_dataset.html#openlayer.OpenlayerClient.add_dataset).\n",
     "\n",
-    "Let's prepare the `dataset_config.yaml` files for our validation set:"
+    "Let's prepare the `dataset_config` for our validation set:"
    ]
   },
   {
@@ -416,8 +416,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Some variables that will go into the `dataset_config.yaml` file\n",
-    "column_names = list(dataset.columns)\n",
+    "# Some variables that will go into the `dataset_config`\n",
     "input_variable_names = [\"question\"]\n",
     "output_column_name = \"answer\""
    ]
@@ -429,17 +428,11 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import yaml \n",
-    "\n",
     "validation_dataset_config = {\n",
-    "    \"columnNames\": column_names,\n",
     "    \"inputVariableNames\": input_variable_names,\n",
     "    \"label\": \"validation\",\n",
     "    \"outputColumnName\": output_column_name,\n",
-    "}\n",
-    "\n",
-    "with open(\"validation_dataset_config.yaml\", \"w\") as dataset_config_file:\n",
-    "    yaml.dump(validation_dataset_config, dataset_config_file, default_flow_style=False)"
+    "}"
    ]
   },
   {
@@ -452,7 +445,7 @@
     "# Validation set\n",
     "project.add_dataframe(\n",
     "    dataset_df=dataset,\n",
-    "    dataset_config_file_path=\"validation_dataset_config.yaml\",\n",
+    "    dataset_config=validation_dataset_config,\n",
     ")"
    ]
   },
@@ -487,9 +480,9 @@
     "\n",
     "Shell models are the most straightforward way to get started. They are comprised of metadata and all the analysis is done via their predictions (which are [uploaded with the datasets](#dataset), in the `outputColumnName`).\n",
     "\n",
-    "To upload a shell model, we only need to define its name, the architecture type, and add some metadata that will be rendered in the platform to help us identify it. This information should be saved to a `model_config.yaml` file.\n",
+    "To upload a shell model, we only need to prepare its `model_config` Python dictionary.\n",
     "\n",
-    "Let's create a `model_config.yaml` file for our model:"
+    "Let's create a `model_config` for our model:"
    ]
   },
   {
@@ -499,14 +492,10 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import yaml\n",
-    "\n",
     "# Note the camelCase for the keys\n",
     "model_config = {\n",
     "    \"inputVariableNames\": [\"question\"],\n",
     "    \"modelType\": \"shell\",\n",
-    "    \"name\": \"LangChain with OpenAI\",\n",
-    "    \"architectureType\": \"llm\",\n",
     "    \"prompt\": [ # Optionally log the prompt, following the same format as OpenAI\n",
     "        {\"role\": \"system\", \"content\": template}, \n",
     "        {\"role\": \"user\", \"content\": human_template}\n",
@@ -516,10 +505,7 @@
     "        \"vector_db_used\": False,\n",
     "        \"temperature\": 0\n",
     "    }\n",
-    "}\n",
-    "\n",
-    "with open(\"model_config.yaml\", \"w\") as model_config_file:\n",
-    "    yaml.dump(model_config, model_config_file, default_flow_style=False)"
+    "}"
    ]
   },
   {
@@ -531,7 +517,7 @@
    "source": [
     "# Adding the model\n",
     "project.add_model(\n",
-    "    model_config_file_path=\"model_config.yaml\",\n",
+    "    model_config=model_config\n",
     ")"
    ]
   },