-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathFundamentals of Building AI Agents using RAG and LangChain.html
422 lines (420 loc) · 142 KB
/
Fundamentals of Building AI Agents using RAG and LangChain.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
<html><head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<style>
.linenums {
list-style-type: none;
}
.formatted-line-numbers {
display: none;
}
.action-code-block {
display: none;
}
table {
border-collapse: collapse;
width: 100%;
}
table, th, td {
border: 1px solid black;
padding: 8px;
text-align: left;
}
</style>
</head><body><h1><span class="header-link octicon octicon-link"></span>Cheat Sheet: Fundamentals of Building AI Agents using RAG and LangChain</h1><table>
<colgroup>
<col style="width:9%;">
<col style="width:57%;">
<col style="width:57%;">
</colgroup>
<thead>
<tr class="header">
<th>Package/Method</th>
<th>Description</th>
<th>Code example</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>Generate text</td>
<td>This code snippet generates text sequences based on the input and doesn't compute the gradient to generate output.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li></ol><ol class="linenums"><li class="L0"><span class="com"># Generate text</span></li><li class="L1"><span class="pln">output_ids </span><span class="pun">=</span><span class="pln"> model</span><span class="pun">.</span><span class="pln">generate</span><span class="pun">(</span></li><li class="L2"><span class="pln"> inputs</span><span class="pun">.</span><span class="pln">input_ids</span><span class="pun">,</span><span class="pln"> </span></li><li class="L3"><span class="pln"> attention_mask</span><span class="pun">=</span><span class="pln">inputs</span><span class="pun">.</span><span class="pln">attention_mask</span><span class="pun">,</span></li><li class="L4"><span class="pln"> pad_token_id</span><span class="pun">=</span><span class="pln">tokenizer</span><span class="pun">.</span><span class="pln">eos_token_id</span><span class="pun">,</span></li><li class="L5"><span class="pln"> max_length</span><span class="pun">=</span><span class="lit">50</span><span class="pun">,</span><span class="pln"> </span></li><li class="L6"><span class="pln"> num_return_sequences</span><span class="pun">=</span><span class="lit">1</span></li><li class="L7"><span class="pun">)</span></li><li class="L8"><span class="pln">output_ids</span></li><li class="L9"><span class="kwd">or</span></li><li class="L0"><span class="kwd">with</span><span class="pln"> torch</span><span class="pun">.</span><span class="pln">no_grad</span><span class="pun">():</span></li><li class="L1"><span class="pln"> outputs </span><span class="pun">=</span><span class="pln"> model</span><span class="pun">(**</span><span class="pln">inputs</span><span class="pun">)</span><span class="pln"> </span></li><li class="L2"><span class="pln">outputs</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-0">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>formatting_prompts_func_no_response function</td>
<td>The prompt function generates formatted text prompts from a data set
by using the instructions from the data set. It creates strings that
include only the instruction and a placeholder for the response.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li><li>18</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> formatting_prompts_func</span><span class="pun">(</span><span class="pln">mydataset</span><span class="pun">):</span></li><li class="L1"><span class="pln"> output_texts </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[]</span></li><li class="L2"><span class="pln"> </span><span class="kwd">for</span><span class="pln"> i </span><span class="kwd">in</span><span class="pln"> range</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">mydataset</span><span class="pun">[</span><span class="str">'instruction'</span><span class="pun">])):</span></li><li class="L3"><span class="pln"> text </span><span class="pun">=</span><span class="pln"> </span><span class="pun">(</span></li><li class="L4"><span class="pln"> f</span><span class="str">"### Instruction:\n{mydataset['instruction'][i]}"</span></li><li class="L5"><span class="pln"> f</span><span class="str">"\n\n### Response:\n{mydataset['output'][i]}"</span></li><li class="L6"><span class="pln"> </span><span class="pun">)</span></li><li class="L7"><span class="pln"> output_texts</span><span class="pun">.</span><span class="pln">append</span><span class="pun">(</span><span class="pln">text</span><span class="pun">)</span></li><li class="L8"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> output_texts</span></li><li class="L9"><span class="kwd">def</span><span class="pln"> formatting_prompts_func_no_response</span><span class="pun">(</span><span class="pln">mydataset</span><span class="pun">):</span></li><li class="L0"><span class="pln"> output_texts </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[]</span></li><li class="L1"><span class="pln"> </span><span class="kwd">for</span><span class="pln"> i </span><span class="kwd">in</span><span class="pln"> range</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">mydataset</span><span class="pun">[</span><span class="str">'instruction'</span><span class="pun">])):</span></li><li class="L2"><span class="pln"> text </span><span class="pun">=</span><span class="pln"> </span><span class="pun">(</span></li><li class="L3"><span class="pln"> f</span><span class="str">"### Instruction:\n{mydataset['instruction'][i]}"</span></li><li class="L4"><span class="pln"> f</span><span class="str">"\n\n### Response:\n"</span></li><li class="L5"><span class="pln"> </span><span class="pun">)</span></li><li class="L6"><span class="pln"> output_texts</span><span class="pun">.</span><span class="pln">append</span><span class="pun">(</span><span class="pln">text</span><span class="pun">)</span></li><li class="L7"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> output_texts</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-1">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>torch.no_grad()</td>
<td>This code snippet helps generate text sequences from the pipeline
function. It ensures that the gradient computations are disabled and
optimizes the performance and memory usage.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li></ol><ol class="linenums"><li class="L0"><span class="kwd">with</span><span class="pln"> torch</span><span class="pun">.</span><span class="pln">no_grad</span><span class="pun">():</span></li><li class="L1"><span class="pln"> </span><span class="com"># Due to resource limitation, only apply the function on 3 records using "instructions_torch[:10]"</span></li><li class="L2"><span class="pln"> pipeline_iterator</span><span class="pun">=</span><span class="pln"> gen_pipeline</span><span class="pun">(</span><span class="pln">instructions_torch</span><span class="pun">[:</span><span class="lit">3</span><span class="pun">],</span></li><li class="L3"><span class="pln"> max_length</span><span class="pun">=</span><span class="lit">50</span><span class="pun">,</span><span class="pln"> </span><span class="com"># this is set to 50 due to resource constraint, using a GPU, you can increase it to the length of your choice</span></li><li class="L4"><span class="pln"> num_beams</span><span class="pun">=</span><span class="lit">5</span><span class="pun">,</span></li><li class="L5"><span class="pln"> early_stopping</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">,)</span></li><li class="L6"><span class="pln">generated_outputs_lora </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[]</span></li><li class="L7"><span class="kwd">for</span><span class="pln"> text </span><span class="kwd">in</span><span class="pln"> pipeline_iterator</span><span class="pun">:</span></li><li class="L8"><span class="pln"> generated_outputs_lora</span><span class="pun">.</span><span class="pln">append</span><span class="pun">(</span><span class="pln">text</span><span class="pun">[</span><span class="lit">0</span><span class="pun">][</span><span class="str">"generated_text"</span><span class="pun">])</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-2">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>mixtral-8x7b-instruct-v01 watsonx.ai inference model object</td>
<td>Adjusts the parameters to push the limits of creativity and response length.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li></ol><ol class="linenums"><li class="L0"><span class="pln">model_id </span><span class="pun">=</span><span class="pln"> </span><span class="str">'mistralai/mixtral-8x7b-instruct-v01'</span></li><li class="L1"><span class="pln">parameters </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L2"><span class="pln"> </span><span class="typ">GenParams</span><span class="pun">.</span><span class="pln">MAX_NEW_TOKENS</span><span class="pun">:</span><span class="pln"> </span><span class="lit">256</span><span class="pun">,</span><span class="pln"> </span><span class="com"># this controls the maximum number of tokens in the generated output</span></li><li class="L3"><span class="pln"> </span><span class="typ">GenParams</span><span class="pun">.</span><span class="pln">TEMPERATURE</span><span class="pun">:</span><span class="pln"> </span><span class="lit">0.5</span><span class="pun">,</span><span class="pln"> </span><span class="com"># this randomness or creativity of the model's responses</span></li><li class="L4"><span class="pun">}</span></li><li class="L5"><span class="pln">credentials </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L6"><span class="pln"> </span><span class="str">"url"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"https://us-south.ml.cloud.ibm.com"</span></li><li class="L7"><span class="pun">}</span></li><li class="L8"><span class="pln">project_id </span><span class="pun">=</span><span class="pln"> </span><span class="str">"skills-network"</span></li><li class="L9"><span class="pln">model </span><span class="pun">=</span><span class="pln"> </span><span class="typ">ModelInference</span><span class="pun">(</span></li><li class="L0"><span class="pln"> model_id</span><span class="pun">=</span><span class="pln">model_id</span><span class="pun">,</span></li><li class="L1"><span class="pln"> </span><span class="kwd">params</span><span class="pun">=</span><span class="pln">parameters</span><span class="pun">,</span></li><li class="L2"><span class="pln"> credentials</span><span class="pun">=</span><span class="pln">credentials</span><span class="pun">,</span></li><li class="L3"><span class="pln"> project_id</span><span class="pun">=</span><span class="pln">project_id</span></li><li class="L4"><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-3">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>String prompt templates</td>
<td>Used to format a single string and are generally used for simpler inputs.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain_core</span><span class="pun">.</span><span class="pln">prompts </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">PromptTemplate</span></li><li class="L1"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">.</span><span class="pln">from_template</span><span class="pun">(</span><span class="str">"Tell me one {adjective} joke about {topic}"</span><span class="pun">)</span></li><li class="L2"><span class="pln">input_ </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span><span class="str">"adjective"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"funny"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"topic"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"cats"</span><span class="pun">}</span><span class="pln"> </span><span class="com"># create a dictionary to store the corresponding input to placeholders in prompt template</span></li><li class="L3"><span class="pln">prompt</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">input_</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-4">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Chat prompt templates</td>
<td>Used to format a list of messages. These "templates" consist of a list of templates themselves.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain_core</span><span class="pun">.</span><span class="pln">prompts </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">ChatPromptTemplate</span></li><li class="L1"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">ChatPromptTemplate</span><span class="pun">.</span><span class="pln">from_messages</span><span class="pun">([</span></li><li class="L2"><span class="pln"> </span><span class="pun">(</span><span class="str">"system"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"You are a helpful assistant"</span><span class="pun">),</span></li><li class="L3"><span class="pln"> </span><span class="pun">(</span><span class="str">"user"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"Tell me a joke about {topic}"</span><span class="pun">)</span></li><li class="L4"><span class="pun">])</span></li><li class="L5"><span class="pln">input_ </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span><span class="str">"topic"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"cats"</span><span class="pun">}</span></li><li class="L6"><span class="pln">prompt</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">input_</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-5">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>Messages place holder</td>
<td>This prompt template is responsible for adding a list of messages in
a particular place. But if you want the user to pass in a list of
messages that you would slot into a particular spot, the given code
snippet is helpful.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain_core</span><span class="pun">.</span><span class="pln">prompts </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">MessagesPlaceholder</span></li><li class="L1"><span class="kwd">from</span><span class="pln"> langchain_core</span><span class="pun">.</span><span class="pln">messages </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">HumanMessage</span></li><li class="L2"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">ChatPromptTemplate</span><span class="pun">.</span><span class="pln">from_messages</span><span class="pun">([</span></li><li class="L3"><span class="pln"> </span><span class="pun">(</span><span class="str">"system"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"You are a helpful assistant"</span><span class="pun">),</span></li><li class="L4"><span class="pln"> </span><span class="typ">MessagesPlaceholder</span><span class="pun">(</span><span class="str">"msgs"</span><span class="pun">)</span></li><li class="L5"><span class="pun">])</span></li><li class="L6"><span class="pln">input_ </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span><span class="str">"msgs"</span><span class="pun">:</span><span class="pln"> </span><span class="pun">[</span><span class="typ">HumanMessage</span><span class="pun">(</span><span class="pln">content</span><span class="pun">=</span><span class="str">"What is the day after Tuesday?"</span><span class="pun">)]}</span></li><li class="L7"><span class="pln">prompt</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">input_</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-6">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Example selector</td>
<td>If you have many examples, you may need to select which ones to
include in the prompt. The Example Selector is the class responsible for
doing so.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li><li>18</li><li>19</li><li>20</li><li>21</li><li>22</li><li>23</li><li>24</li><li>25</li><li>26</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain_core</span><span class="pun">.</span><span class="pln">example_selectors </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">LengthBasedExampleSelector</span></li><li class="L1"><span class="kwd">from</span><span class="pln"> langchain_core</span><span class="pun">.</span><span class="pln">prompts </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">FewShotPromptTemplate</span><span class="pun">,</span><span class="pln"> </span><span class="typ">PromptTemplate</span></li><li class="L2"><span class="com"># Examples of a pretend task of creating antonyms.</span></li><li class="L3"><span class="pln">examples </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span></li><li class="L4"><span class="pln"> </span><span class="pun">{</span><span class="str">"input"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"happy"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"output"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"sad"</span><span class="pun">},</span></li><li class="L5"><span class="pln"> </span><span class="pun">{</span><span class="str">"input"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"tall"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"output"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"short"</span><span class="pun">},</span></li><li class="L6"><span class="pln"> </span><span class="pun">{</span><span class="str">"input"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"energetic"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"output"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"lethargic"</span><span class="pun">},</span></li><li class="L7"><span class="pln"> </span><span class="pun">{</span><span class="str">"input"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"sunny"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"output"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"gloomy"</span><span class="pun">},</span></li><li class="L8"><span class="pln"> </span><span class="pun">{</span><span class="str">"input"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"windy"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"output"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"calm"</span><span class="pun">},</span></li><li class="L9"><span class="pun">]</span></li><li class="L0"><span class="pln">example_prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">(</span></li><li class="L1"><span class="pln"> input_variables</span><span class="pun">=[</span><span class="str">"input"</span><span class="pun">,</span><span class="pln"> </span><span class="str">"output"</span><span class="pun">],</span></li><li class="L2"><span class="pln"> </span><span class="kwd">template</span><span class="pun">=</span><span class="str">"Input: {input}\nOutput: {output}"</span><span class="pun">,</span></li><li class="L3"><span class="pun">)</span></li><li class="L4"><span class="pln">example_selector </span><span class="pun">=</span><span class="pln"> </span><span class="typ">LengthBasedExampleSelector</span><span class="pun">(</span></li><li class="L5"><span class="pln"> examples</span><span class="pun">=</span><span class="pln">examples</span><span class="pun">,</span></li><li class="L6"><span class="pln"> example_prompt</span><span class="pun">=</span><span class="pln">example_prompt</span><span class="pun">,</span></li><li class="L7"><span class="pln"> max_length</span><span class="pun">=</span><span class="lit">25</span><span class="pun">,</span><span class="pln"> </span><span class="com"># The maximum length that the formatted examples should be.</span></li><li class="L8"><span class="pun">)</span></li><li class="L9"><span class="pln">dynamic_prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">FewShotPromptTemplate</span><span class="pun">(</span></li><li class="L0"><span class="pln"> example_selector</span><span class="pun">=</span><span class="pln">example_selector</span><span class="pun">,</span></li><li class="L1"><span class="pln"> example_prompt</span><span class="pun">=</span><span class="pln">example_prompt</span><span class="pun">,</span></li><li class="L2"><span class="pln"> prefix</span><span class="pun">=</span><span class="str">"Give the antonym of every input"</span><span class="pun">,</span></li><li class="L3"><span class="pln"> suffix</span><span class="pun">=</span><span class="str">"Input: {adjective}\nOutput:"</span><span class="pun">,</span></li><li class="L4"><span class="pln"> input_variables</span><span class="pun">=[</span><span class="str">"adjective"</span><span class="pun">],</span></li><li class="L5"><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-7">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>JSON parser</td>
<td>This output parser allows users to specify an arbitrary JSON schema and query LLMs for outputs that conform to that schema.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li><li>18</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain_core</span><span class="pun">.</span><span class="pln">output_parsers </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">JsonOutputParser</span></li><li class="L1"><span class="kwd">from</span><span class="pln"> langchain_core</span><span class="pun">.</span><span class="pln">pydantic_v1 </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">BaseModel</span><span class="pun">,</span><span class="pln"> </span><span class="typ">Field</span></li><li class="L2"><span class="com"># Define your desired data structure.</span></li><li class="L3"><span class="kwd">class</span><span class="pln"> </span><span class="typ">Joke</span><span class="pun">(</span><span class="typ">BaseModel</span><span class="pun">):</span></li><li class="L4"><span class="pln"> setup</span><span class="pun">:</span><span class="pln"> str </span><span class="pun">=</span><span class="pln"> </span><span class="typ">Field</span><span class="pun">(</span><span class="pln">description</span><span class="pun">=</span><span class="str">"question to set up a joke"</span><span class="pun">)</span></li><li class="L5"><span class="pln"> punchline</span><span class="pun">:</span><span class="pln"> str </span><span class="pun">=</span><span class="pln"> </span><span class="typ">Field</span><span class="pun">(</span><span class="pln">description</span><span class="pun">=</span><span class="str">"answer to resolve the joke"</span><span class="pun">)</span></li><li class="L6"><span class="com"># And a query intented to prompt a language model to populate the data structure.</span></li><li class="L7"><span class="pln">joke_query </span><span class="pun">=</span><span class="pln"> </span><span class="str">"Tell me a joke."</span></li><li class="L8"><span class="com"># Set up a parser + inject instructions into the prompt template.</span></li><li class="L9"><span class="pln">output_parser </span><span class="pun">=</span><span class="pln"> </span><span class="typ">JsonOutputParser</span><span class="pun">(</span><span class="pln">pydantic_object</span><span class="pun">=</span><span class="typ">Joke</span><span class="pun">)</span></li><li class="L0"><span class="pln">format_instructions </span><span class="pun">=</span><span class="pln"> output_parser</span><span class="pun">.</span><span class="pln">get_format_instructions</span><span class="pun">()</span></li><li class="L1"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">(</span></li><li class="L2"><span class="pln"> </span><span class="kwd">template</span><span class="pun">=</span><span class="str">"Answer the user query.\n{format_instructions}\n{query}\n"</span><span class="pun">,</span></li><li class="L3"><span class="pln"> input_variables</span><span class="pun">=[</span><span class="str">"query"</span><span class="pun">],</span></li><li class="L4"><span class="pln"> partial_variables</span><span class="pun">={</span><span class="str">"format_instructions"</span><span class="pun">:</span><span class="pln"> format_instructions</span><span class="pun">},</span></li><li class="L5"><span class="pun">)</span></li><li class="L6"><span class="pln">chain </span><span class="pun">=</span><span class="pln"> prompt </span><span class="pun">|</span><span class="pln"> mixtral_llm </span><span class="pun">|</span><span class="pln"> output_parser</span></li><li class="L7"><span class="pln">chain</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">({</span><span class="str">"query"</span><span class="pun">:</span><span class="pln"> joke_query</span><span class="pun">})</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-8">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Comma separated list parser</td>
<td>This output parser can be used when you want to return a list of comma-separated items.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain</span><span class="pun">.</span><span class="pln">output_parsers </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">CommaSeparatedListOutputParser</span></li><li class="L1"><span class="pln">output_parser </span><span class="pun">=</span><span class="pln"> </span><span class="typ">CommaSeparatedListOutputParser</span><span class="pun">()</span></li><li class="L2"><span class="pln">format_instructions </span><span class="pun">=</span><span class="pln"> output_parser</span><span class="pun">.</span><span class="pln">get_format_instructions</span><span class="pun">()</span></li><li class="L3"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">(</span></li><li class="L4"><span class="pln"> </span><span class="kwd">template</span><span class="pun">=</span><span class="str">"Answer the user query. {format_instructions}\nList five {subject}."</span><span class="pun">,</span></li><li class="L5"><span class="pln"> input_variables</span><span class="pun">=[</span><span class="str">"subject"</span><span class="pun">],</span></li><li class="L6"><span class="pln"> partial_variables</span><span class="pun">={</span><span class="str">"format_instructions"</span><span class="pun">:</span><span class="pln"> format_instructions</span><span class="pun">},</span></li><li class="L7"><span class="pun">)</span></li><li class="L8"><span class="pln">chain </span><span class="pun">=</span><span class="pln"> prompt </span><span class="pun">|</span><span class="pln"> mixtral_llm </span><span class="pun">|</span><span class="pln"> output_parser</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-9">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>Document object</td>
<td>Contains information about some data in LangChain. It has two attributes:
<b>page_content: str:</b> This attribute holds the content of the document.<br>
<b>metadata: dict:</b> This attribute contains arbitrary metadata
associated with the document. It can be used to track various details
such as the document id, file name, and so on.
</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain_core</span><span class="pun">.</span><span class="pln">documents </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">Document</span></li><li class="L1"><span class="typ">Document</span><span class="pun">(</span><span class="pln">page_content</span><span class="pun">=</span><span class="str">"""Python is an interpreted high-level general-purpose programming language. </span></li><li class="L2"><span class="str">
Python's design philosophy emphasizes code
readability with its notable use of significant indentation."""</span><span class="pun">,</span></li><li class="L3"><span class="pln"> metadata</span><span class="pun">={</span></li><li class="L4"><span class="pln"> </span><span class="str">'my_document_id'</span><span class="pln"> </span><span class="pun">:</span><span class="pln"> </span><span class="lit">234234</span><span class="pun">,</span></li><li class="L5"><span class="pln"> </span><span class="str">'my_document_source'</span><span class="pln"> </span><span class="pun">:</span><span class="pln"> </span><span class="str">"About Python"</span><span class="pun">,</span></li><li class="L6"><span class="pln"> </span><span class="str">'my_document_create_time'</span><span class="pln"> </span><span class="pun">:</span><span class="pln"> </span><span class="lit">1680013019</span></li><li class="L7"><span class="pln"> </span><span class="pun">})</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-10">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>text_splitter</td>
<td>At a high level, text splitters work as follows:<br>
• Split the text into small, semantically meaningful chunks (often sentences).<br>
• Start combining these small chunks into a larger chunk until you reach a certain size (as measured by some function).<br>
• Once you reach that size, make that chunk its own piece of text and
start creating a new chunk with some overlap (to keep context between
chunks).
</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li></ol><ol class="linenums"><li class="L0"><span class="pln">text_splitter </span><span class="pun">=</span><span class="pln"> </span><span class="typ">CharacterTextSplitter</span><span class="pun">(</span><span class="pln">chunk_size</span><span class="pun">=</span><span class="lit">200</span><span class="pun">,</span><span class="pln"> chunk_overlap</span><span class="pun">=</span><span class="lit">20</span><span class="pun">,</span><span class="pln"> separator</span><span class="pun">=</span><span class="str">"\n"</span><span class="pun">)</span><span class="pln"> </span><span class="com"># define chunk_size which is length of characters, and also separator.</span></li><li class="L1"><span class="pln">chunks </span><span class="pun">=</span><span class="pln"> text_splitter</span><span class="pun">.</span><span class="pln">split_documents</span><span class="pun">(</span><span class="pln">document</span><span class="pun">)</span></li><li class="L2"><span class="kwd">print</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">chunks</span><span class="pun">))</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-11">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>Embedding models</td>
<td>Embedding models are specifically designed to interface with text
embeddings.
Embeddings generate a vector representation for a given piece of text.
This is advantageous as it allows you to conceptualize text within a
vector space. Consequently, you can perform operations such as semantic
search, where you identify pieces of text that are most similar within
the vector space.
</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> ibm_watsonx_ai</span><span class="pun">.</span><span class="pln">metanames </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">EmbedTextParamsMetaNames</span></li><li class="L1"><span class="pln">embed_params </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L2"><span class="pln"> </span><span class="typ">EmbedTextParamsMetaNames</span><span class="pun">.</span><span class="pln">TRUNCATE_INPUT_TOKENS</span><span class="pun">:</span><span class="pln"> </span><span class="lit">3</span><span class="pun">,</span></li><li class="L3"><span class="pln"> </span><span class="typ">EmbedTextParamsMetaNames</span><span class="pun">.</span><span class="pln">RETURN_OPTIONS</span><span class="pun">:</span><span class="pln"> </span><span class="pun">{</span><span class="str">"input_text"</span><span class="pun">:</span><span class="pln"> </span><span class="kwd">True</span><span class="pun">},</span></li><li class="L4"><span class="pun">}</span></li><li class="L5"><span class="kwd">from</span><span class="pln"> langchain_ibm </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">WatsonxEmbeddings</span></li><li class="L6"><span class="pln">watsonx_embedding </span><span class="pun">=</span><span class="pln"> </span><span class="typ">WatsonxEmbeddings</span><span class="pun">(</span></li><li class="L7"><span class="pln"> model_id</span><span class="pun">=</span><span class="str">"ibm/slate-125m-english-rtrvr"</span><span class="pun">,</span></li><li class="L8"><span class="pln"> url</span><span class="pun">=</span><span class="str">"https://us-south.ml.cloud.ibm.com"</span><span class="pun">,</span></li><li class="L9"><span class="pln"> project_id</span><span class="pun">=</span><span class="str">"skills-network"</span><span class="pun">,</span></li><li class="L0"><span class="pln"> </span><span class="kwd">params</span><span class="pun">=</span><span class="pln">embed_params</span><span class="pun">,</span></li><li class="L1"><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-12">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Vector store-backed retriever</td>
<td>A retriever that uses a vector store to retrieve documents. It is a
lightweight wrapper around the vector store class to make it conform to
the retriever interface. It uses the search methods implemented by a
vector store, like similarity search and MMR (maximum marginal
relevance), to query the texts in the vector store.
Since we've constructed a vector store docsearch, it's very easy to
construct a retriever.
</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li></ol><ol class="linenums"><li class="L0"><span class="pln">retriever </span><span class="pun">=</span><span class="pln"> docsearch</span><span class="pun">.</span><span class="pln">as_retriever</span><span class="pun">()</span></li><li class="L1"><span class="pln">docs </span><span class="pun">=</span><span class="pln"> retriever</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="str">"Langchain"</span><span class="pun">)</span><span class="pln">A vector store retriever </span><span class="kwd">is</span><span class="pln"> a retriever that uses a vector store to retrieve documents</span><span class="pun">.</span><span class="pln"> </span><span class="typ">It</span><span class="pln"> </span><span class="kwd">is</span><span class="pln"> a lightweight wrapper around the vector store </span><span class="kwd">class</span><span class="pln"> to make it conform to the retriever </span><span class="kwd">interface</span><span class="pun">.</span><span class="pln"> </span><span class="typ">It</span><span class="pln"> uses the search methods implemented </span><span class="kwd">by</span><span class="pln"> a vector store</span><span class="pun">,</span><span class="pln"> like similarity search </span><span class="kwd">and</span><span class="pln"> MMR </span><span class="pun">(</span><span class="typ">Maximum</span><span class="pln"> marginal relevance</span><span class="pun">),</span><span class="pln"> to query the texts </span><span class="kwd">in</span><span class="pln"> the vector store</span><span class="pun">.</span></li><li class="L2"><span class="typ">Since</span><span class="pln"> we</span><span class="str">'ve constructed a vector store docsearch, it'</span><span class="pln">s very easy to construct a retriever</span><span class="pun">.</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-13">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>ChatMessageHistory class</td>
<td>One of the core utility classes underpinning most (if not all)
memory modules is the ChatMessageHistory class. This super lightweight
wrapper provides convenient methods for saving HumanMessages,
AIMessages, and then fetching them all.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain</span><span class="pun">.</span><span class="pln">memory </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">ChatMessageHistory</span></li><li class="L1"><span class="pln">chat </span><span class="pun">=</span><span class="pln"> mixtral_llm</span></li><li class="L2"><span class="pln">history </span><span class="pun">=</span><span class="pln"> </span><span class="typ">ChatMessageHistory</span><span class="pun">()</span></li><li class="L3"><span class="pln">history</span><span class="pun">.</span><span class="pln">add_ai_message</span><span class="pun">(</span><span class="str">"hi!"</span><span class="pun">)</span></li><li class="L4"><span class="pln">history</span><span class="pun">.</span><span class="pln">add_user_message</span><span class="pun">(</span><span class="str">"what is the capital of France?"</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-14">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>langchain.chains</td>
<td>This code snippet uses a LangChain, library for building language
model applications, creating a chain to generate popular dish
recommendations based on the specified locations. It also configures
model inference settings for further processing.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain</span><span class="pun">.</span><span class="pln">chains </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">LLMChain</span></li><li class="L1"><span class="kwd">template</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""Your job is to come up with a classic dish from the area that the users suggests.</span></li><li class="L2"><span class="str"> {location}</span></li><li class="L3"><span class="str"> YOUR RESPONSE:</span></li><li class="L4"><span class="str">"""</span></li><li class="L5"><span class="pln">prompt_template </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">(</span><span class="kwd">template</span><span class="pun">=</span><span class="kwd">template</span><span class="pun">,</span><span class="pln"> input_variables</span><span class="pun">=[</span><span class="str">'location'</span><span class="pun">])</span></li><li class="L6"><span class="com"># chain 1</span></li><li class="L7"><span class="pln">location_chain </span><span class="pun">=</span><span class="pln"> </span><span class="typ">LLMChain</span><span class="pun">(</span><span class="pln">llm</span><span class="pun">=</span><span class="pln">mixtral_llm</span><span class="pun">,</span><span class="pln"> prompt</span><span class="pun">=</span><span class="pln">prompt_template</span><span class="pun">,</span><span class="pln"> output_key</span><span class="pun">=</span><span class="str">'meal'</span><span class="pun">)</span></li><li class="L8"><span class="pln">location_chain</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">input</span><span class="pun">={</span><span class="str">'location'</span><span class="pun">:</span><span class="str">'China'</span><span class="pun">})</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-15">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>Simple sequential chain</td>
<td>Sequential chains allow the output of one LLM to be used as the
input for another. This approach is beneficial for dividing tasks and
maintaining the focus of your LLM.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li><li>18</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain</span><span class="pun">.</span><span class="pln">chains </span><span class="kwd">import</span><span class="pln"> </span><span class="typ">SequentialChain</span></li><li class="L1"><span class="kwd">template</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""Given a meal {meal}, give a short and simple recipe on how to make that dish at home.</span></li><li class="L2"><span class="str"> YOUR RESPONSE:</span></li><li class="L3"><span class="str">"""</span></li><li class="L4"><span class="pln">prompt_template </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">(</span><span class="kwd">template</span><span class="pun">=</span><span class="kwd">template</span><span class="pun">,</span><span class="pln"> input_variables</span><span class="pun">=[</span><span class="str">'meal'</span><span class="pun">])</span></li><li class="L5"><span class="com"># chain 2</span></li><li class="L6"><span class="pln">dish_chain </span><span class="pun">=</span><span class="pln"> </span><span class="typ">LLMChain</span><span class="pun">(</span><span class="pln">llm</span><span class="pun">=</span><span class="pln">mixtral_llm</span><span class="pun">,</span><span class="pln"> prompt</span><span class="pun">=</span><span class="pln">prompt_template</span><span class="pun">,</span><span class="pln"> output_key</span><span class="pun">=</span><span class="str">'recipe'</span><span class="pun">)</span></li><li class="L7"><span class="kwd">template</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""Given the recipe {recipe}, estimate how much time I need to cook it.</span></li><li class="L8"><span class="str"> YOUR RESPONSE:</span></li><li class="L9"><span class="str">"""</span></li><li class="L0"><span class="pln">prompt_template </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">(</span><span class="kwd">template</span><span class="pun">=</span><span class="kwd">template</span><span class="pun">,</span><span class="pln"> input_variables</span><span class="pun">=[</span><span class="str">'recipe'</span><span class="pun">])</span></li><li class="L1"><span class="com"># chain 3</span></li><li class="L2"><span class="pln">recipe_chain </span><span class="pun">=</span><span class="pln"> </span><span class="typ">LLMChain</span><span class="pun">(</span><span class="pln">llm</span><span class="pun">=</span><span class="pln">mixtral_llm</span><span class="pun">,</span><span class="pln"> prompt</span><span class="pun">=</span><span class="pln">prompt_template</span><span class="pun">,</span><span class="pln"> output_key</span><span class="pun">=</span><span class="str">'time'</span><span class="pun">)</span></li><li class="L3"><span class="com"># overall chain</span></li><li class="L4"><span class="pln">overall_chain </span><span class="pun">=</span><span class="pln"> </span><span class="typ">SequentialChain</span><span class="pun">(</span><span class="pln">chains</span><span class="pun">=[</span><span class="pln">location_chain</span><span class="pun">,</span><span class="pln"> dish_chain</span><span class="pun">,</span><span class="pln"> recipe_chain</span><span class="pun">],</span></li><li class="L5"><span class="pln"> input_variables</span><span class="pun">=[</span><span class="str">'location'</span><span class="pun">],</span></li><li class="L6"><span class="pln"> output_variables</span><span class="pun">=[</span><span class="str">'meal'</span><span class="pun">,</span><span class="pln"> </span><span class="str">'recipe'</span><span class="pun">,</span><span class="pln"> </span><span class="str">'time'</span><span class="pun">],</span></li><li class="L7"><span class="pln"> verbose</span><span class="pun">=</span><span class="pln"> </span><span class="kwd">True</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-16">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>load_summarize_chain</td>
<td>This code snippet uses LangChain library for loading and using a
summarization chain with a specific language model and chain type. This
chain type will be applied to web data to print a resulting summary.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> langchain</span><span class="pun">.</span><span class="pln">chains</span><span class="pun">.</span><span class="pln">summarize </span><span class="kwd">import</span><span class="pln"> load_summarize_chain</span></li><li class="L1"><span class="pln">chain </span><span class="pun">=</span><span class="pln"> load_summarize_chain</span><span class="pun">(</span><span class="pln">llm</span><span class="pun">=</span><span class="pln">mixtral_llm</span><span class="pun">,</span><span class="pln"> chain_type</span><span class="pun">=</span><span class="str">"stuff"</span><span class="pun">,</span><span class="pln"> verbose</span><span class="pun">=</span><span class="kwd">False</span><span class="pun">)</span></li><li class="L2"><span class="pln">response </span><span class="pun">=</span><span class="pln"> chain</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">web_data</span><span class="pun">)</span></li><li class="L3"><span class="kwd">print</span><span class="pun">(</span><span class="pln">response</span><span class="pun">[</span><span class="str">'output_text'</span><span class="pun">])</span><span class="pln">n</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-17">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>TextClassifier</td>
<td>Represents a simple text classifier that uses an embedding layer, a
hidden linear layer with a ReLU avtivation, and an output linear layer.
The constructor takes the following arguments:
num_class: The number of classes to classify.
freeze: Whether to freeze the embedding layer.
</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li><li>18</li><li>19</li></ol><ol class="linenums"><li class="L0"><span class="kwd">from</span><span class="pln"> torch </span><span class="kwd">import</span><span class="pln"> nn</span></li><li class="L1"><span class="kwd">class</span><span class="pln"> </span><span class="typ">TextClassifier</span><span class="pun">(</span><span class="pln">nn</span><span class="pun">.</span><span class="typ">Module</span><span class="pun">):</span></li><li class="L2"><span class="pln"> </span><span class="kwd">def</span><span class="pln"> __init__</span><span class="pun">(</span><span class="kwd">self</span><span class="pun">,</span><span class="pln"> num_classes</span><span class="pun">,</span><span class="pln">freeze</span><span class="pun">=</span><span class="kwd">False</span><span class="pun">):</span></li><li class="L3"><span class="pln"> </span><span class="kwd">super</span><span class="pun">(</span><span class="typ">TextClassifier</span><span class="pun">,</span><span class="pln"> </span><span class="kwd">self</span><span class="pun">).</span><span class="pln">__init__</span><span class="pun">()</span></li><li class="L4"><span class="pln"> </span><span class="kwd">self</span><span class="pun">.</span><span class="pln">embedding </span><span class="pun">=</span><span class="pln"> nn</span><span class="pun">.</span><span class="typ">Embedding</span><span class="pun">.</span><span class="pln">from_pretrained</span><span class="pun">(</span><span class="pln">glove_embedding</span><span class="pun">.</span><span class="pln">vectors</span><span class="pun">.</span><span class="pln">to</span><span class="pun">(</span><span class="pln">device</span><span class="pun">),</span><span class="pln">freeze</span><span class="pun">=</span><span class="pln">freeze</span><span class="pun">)</span></li><li class="L5"><span class="pln"> </span><span class="com"># An example of adding additional layers: A linear layer and a ReLU activation</span></li><li class="L6"><span class="pln"> </span><span class="kwd">self</span><span class="pun">.</span><span class="pln">fc1 </span><span class="pun">=</span><span class="pln"> nn</span><span class="pun">.</span><span class="typ">Linear</span><span class="pun">(</span><span class="pln">in_features</span><span class="pun">=</span><span class="lit">100</span><span class="pun">,</span><span class="pln"> out_features</span><span class="pun">=</span><span class="lit">128</span><span class="pun">)</span></li><li class="L7"><span class="pln"> </span><span class="kwd">self</span><span class="pun">.</span><span class="pln">relu </span><span class="pun">=</span><span class="pln"> nn</span><span class="pun">.</span><span class="typ">ReLU</span><span class="pun">()</span></li><li class="L8"><span class="pln"> </span><span class="com"># The output layer that gives the final probabilities for the classes</span></li><li class="L9"><span class="pln"> </span><span class="kwd">self</span><span class="pun">.</span><span class="pln">fc2 </span><span class="pun">=</span><span class="pln"> nn</span><span class="pun">.</span><span class="typ">Linear</span><span class="pun">(</span><span class="pln">in_features</span><span class="pun">=</span><span class="lit">128</span><span class="pun">,</span><span class="pln"> out_features</span><span class="pun">=</span><span class="pln">num_classes</span><span class="pun">)</span></li><li class="L0"><span class="pln"> </span><span class="kwd">def</span><span class="pln"> forward</span><span class="pun">(</span><span class="kwd">self</span><span class="pun">,</span><span class="pln"> x</span><span class="pun">):</span></li><li class="L1"><span class="pln"> </span><span class="com"># Pass the input through the embedding layer</span></li><li class="L2"><span class="pln"> x </span><span class="pun">=</span><span class="pln"> </span><span class="kwd">self</span><span class="pun">.</span><span class="pln">embedding</span><span class="pun">(</span><span class="pln">x</span><span class="pun">)</span></li><li class="L3"><span class="pln"> </span><span class="com"># Here you can use a simple mean pooling</span></li><li class="L4"><span class="pln"> x </span><span class="pun">=</span><span class="pln"> torch</span><span class="pun">.</span><span class="pln">mean</span><span class="pun">(</span><span class="pln">x</span><span class="pun">,</span><span class="pln"> dim</span><span class="pun">=</span><span class="lit">1</span><span class="pun">)</span></li><li class="L5"><span class="pln"> </span><span class="com"># Pass the pooled embeddings through the additional layers</span></li><li class="L6"><span class="pln"> x </span><span class="pun">=</span><span class="pln"> </span><span class="kwd">self</span><span class="pun">.</span><span class="pln">fc1</span><span class="pun">(</span><span class="pln">x</span><span class="pun">)</span></li><li class="L7"><span class="pln"> x </span><span class="pun">=</span><span class="pln"> </span><span class="kwd">self</span><span class="pun">.</span><span class="pln">relu</span><span class="pun">(</span><span class="pln">x</span><span class="pun">)</span></li><li class="L8"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> </span><span class="kwd">self</span><span class="pun">.</span><span class="pln">fc2</span><span class="pun">(</span><span class="pln">x</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-18">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Train the model</td>
<td>This code snippet outlines the function to train a machine learning
model using PyTorch. This function trains the model over a specified
number of epochs, tracks them, and evaluates the performance on the data
set.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li><li>18</li><li>19</li><li>20</li><li>21</li><li>22</li><li>23</li><li>24</li><li>25</li><li>26</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> train_model</span><span class="pun">(</span><span class="pln">model</span><span class="pun">,</span><span class="pln"> optimizer</span><span class="pun">,</span><span class="pln"> criterion</span><span class="pun">,</span><span class="pln"> train_dataloader</span><span class="pun">,</span><span class="pln"> valid_dataloader</span><span class="pun">,</span><span class="pln"> epochs</span><span class="pun">=</span><span class="lit">100</span><span class="pun">,</span><span class="pln"> model_name</span><span class="pun">=</span><span class="str">"my_modeldrop"</span><span class="pun">):</span></li><li class="L1"><span class="pln"> cum_loss_list </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[]</span></li><li class="L2"><span class="pln"> acc_epoch </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[]</span></li><li class="L3"><span class="pln"> best_acc </span><span class="pun">=</span><span class="pln"> </span><span class="lit">0</span></li><li class="L4"><span class="pln"> file_name </span><span class="pun">=</span><span class="pln"> model_name</span></li><li class="L5"><span class="pln"> </span><span class="kwd">for</span><span class="pln"> epoch </span><span class="kwd">in</span><span class="pln"> tqdm</span><span class="pun">(</span><span class="pln">range</span><span class="pun">(</span><span class="lit">1</span><span class="pun">,</span><span class="pln"> epochs </span><span class="pun">+</span><span class="pln"> </span><span class="lit">1</span><span class="pun">)):</span></li><li class="L6"><span class="pln"> model</span><span class="pun">.</span><span class="pln">train</span><span class="pun">()</span></li><li class="L7"><span class="pln"> cum_loss </span><span class="pun">=</span><span class="pln"> </span><span class="lit">0</span></li><li class="L8"><span class="pln"> </span><span class="kwd">for</span><span class="pln"> _</span><span class="pun">,</span><span class="pln"> </span><span class="pun">(</span><span class="pln">label</span><span class="pun">,</span><span class="pln"> text</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">in</span><span class="pln"> enumerate</span><span class="pun">(</span><span class="pln">train_dataloader</span><span class="pun">):</span><span class="pln"> </span></li><li class="L9"><span class="pln"> optimizer</span><span class="pun">.</span><span class="pln">zero_grad</span><span class="pun">()</span></li><li class="L0"><span class="pln"> predicted_label </span><span class="pun">=</span><span class="pln"> model</span><span class="pun">(</span><span class="pln">text</span><span class="pun">)</span></li><li class="L1"><span class="pln"> loss </span><span class="pun">=</span><span class="pln"> criterion</span><span class="pun">(</span><span class="pln">predicted_label</span><span class="pun">,</span><span class="pln"> label</span><span class="pun">)</span></li><li class="L2"><span class="pln"> loss</span><span class="pun">.</span><span class="pln">backward</span><span class="pun">()</span></li><li class="L3"><span class="pln"> torch</span><span class="pun">.</span><span class="pln">nn</span><span class="pun">.</span><span class="pln">utils</span><span class="pun">.</span><span class="pln">clip_grad_norm_</span><span class="pun">(</span><span class="pln">model</span><span class="pun">.</span><span class="pln">parameters</span><span class="pun">(),</span><span class="pln"> </span><span class="lit">0.1</span><span class="pun">)</span></li><li class="L4"><span class="pln"> optimizer</span><span class="pun">.</span><span class="pln">step</span><span class="pun">()</span></li><li class="L5"><span class="pln"> cum_loss </span><span class="pun">+=</span><span class="pln"> loss</span><span class="pun">.</span><span class="pln">item</span><span class="pun">()</span></li><li class="L6"><span class="pln"> </span><span class="com">#print("Loss:", cum_loss)</span></li><li class="L7"><span class="pln"> cum_loss_list</span><span class="pun">.</span><span class="pln">append</span><span class="pun">(</span><span class="pln">cum_loss</span><span class="pun">)</span></li><li class="L8"><span class="pln"> acc_val </span><span class="pun">=</span><span class="pln"> evaluate</span><span class="pun">(</span><span class="pln">valid_dataloader</span><span class="pun">,</span><span class="pln"> model</span><span class="pun">,</span><span class="pln"> device</span><span class="pun">)</span></li><li class="L9"><span class="pln"> acc_epoch</span><span class="pun">.</span><span class="pln">append</span><span class="pun">(</span><span class="pln">acc_val</span><span class="pun">)</span></li><li class="L0"><span class="pln"> </span><span class="kwd">if</span><span class="pln"> acc_val </span><span class="pun">></span><span class="pln"> best_acc</span><span class="pun">:</span></li><li class="L1"><span class="pln"> best_acc </span><span class="pun">=</span><span class="pln"> acc_val</span></li><li class="L2"><span class="pln"> </span><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"New best accuracy: {acc_val:.4f}"</span><span class="pun">)</span></li><li class="L3"><span class="pln"> </span><span class="com">#torch.save(model.state_dict(), f"{model_name}.pth")</span></li><li class="L4"><span class="pln"> </span><span class="com">#save_list_to_file(cum_loss_list, f"{model_name}_loss.pkl")</span></li><li class="L5"><span class="pln"> </span><span class="com">#save_list_to_file(acc_epoch, f"{model_name}_acc.pkl")</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-19">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>llm_model</td>
<td>This code snippet defines function 'llm_model' for generating text
using the language model from the mistral.ai platform, specifically the
'mitral-8x7b-instruct-v01' model. The function helps in customizing
generating parameters and interacts with IBM Watson's machine learning
services.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li><li>18</li><li>19</li><li>20</li><li>21</li><li>22</li><li>23</li><li>24</li><li>25</li><li>26</li><li>27</li><li>28</li><li>29</li><li>30</li><li>31</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> llm_model</span><span class="pun">(</span><span class="pln">prompt_txt</span><span class="pun">,</span><span class="pln"> </span><span class="kwd">params</span><span class="pun">=</span><span class="kwd">None</span><span class="pun">):</span></li><li class="L1"><span class="pln"> model_id </span><span class="pun">=</span><span class="pln"> </span><span class="str">'mistralai/mixtral-8x7b-instruct-v01'</span></li><li class="L2"><span class="pln"> default_params </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L3"><span class="pln"> </span><span class="str">"max_new_tokens"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">256</span><span class="pun">,</span></li><li class="L4"><span class="pln"> </span><span class="str">"min_new_tokens"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">0</span><span class="pun">,</span></li><li class="L5"><span class="pln"> </span><span class="str">"temperature"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">0.5</span><span class="pun">,</span></li><li class="L6"><span class="pln"> </span><span class="str">"top_p"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">0.2</span><span class="pun">,</span></li><li class="L7"><span class="pln"> </span><span class="str">"top_k"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">1</span></li><li class="L8"><span class="pln"> </span><span class="pun">}</span></li><li class="L9"><span class="pln"> </span><span class="kwd">if</span><span class="pln"> </span><span class="kwd">params</span><span class="pun">:</span></li><li class="L0"><span class="pln"> default_params</span><span class="pun">.</span><span class="pln">update</span><span class="pun">(</span><span class="kwd">params</span><span class="pun">)</span></li><li class="L1"><span class="pln"> parameters </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L2"><span class="pln"> </span><span class="typ">GenParams</span><span class="pun">.</span><span class="pln">MAX_NEW_TOKENS</span><span class="pun">:</span><span class="pln"> default_params</span><span class="pun">[</span><span class="str">"max_new_tokens"</span><span class="pun">],</span><span class="pln"> </span><span class="com"># this controls the maximum number of tokens in the generated output</span></li><li class="L3"><span class="pln"> </span><span class="typ">GenParams</span><span class="pun">.</span><span class="pln">MIN_NEW_TOKENS</span><span class="pun">:</span><span class="pln"> default_params</span><span class="pun">[</span><span class="str">"min_new_tokens"</span><span class="pun">],</span><span class="pln"> </span><span class="com"># this controls the minimum number of tokens in the generated output</span></li><li class="L4"><span class="pln"> </span><span class="typ">GenParams</span><span class="pun">.</span><span class="pln">TEMPERATURE</span><span class="pun">:</span><span class="pln"> default_params</span><span class="pun">[</span><span class="str">"temperature"</span><span class="pun">],</span><span class="pln"> </span><span class="com"># this randomness or creativity of the model's responses</span></li><li class="L5"><span class="pln"> </span><span class="typ">GenParams</span><span class="pun">.</span><span class="pln">TOP_P</span><span class="pun">:</span><span class="pln"> default_params</span><span class="pun">[</span><span class="str">"top_p"</span><span class="pun">],</span></li><li class="L6"><span class="pln"> </span><span class="typ">GenParams</span><span class="pun">.</span><span class="pln">TOP_K</span><span class="pun">:</span><span class="pln"> default_params</span><span class="pun">[</span><span class="str">"top_k"</span><span class="pun">]</span></li><li class="L7"><span class="pln"> </span><span class="pun">}</span></li><li class="L8"><span class="pln"> credentials </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L9"><span class="pln"> </span><span class="str">"url"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"https://us-south.ml.cloud.ibm.com"</span></li><li class="L0"><span class="pln"> </span><span class="pun">}</span></li><li class="L1"><span class="pln"> project_id </span><span class="pun">=</span><span class="pln"> </span><span class="str">"skills-network"</span></li><li class="L2"><span class="pln"> model </span><span class="pun">=</span><span class="pln"> </span><span class="typ">Model</span><span class="pun">(</span></li><li class="L3"><span class="pln"> model_id</span><span class="pun">=</span><span class="pln">model_id</span><span class="pun">,</span></li><li class="L4"><span class="pln"> </span><span class="kwd">params</span><span class="pun">=</span><span class="pln">parameters</span><span class="pun">,</span></li><li class="L5"><span class="pln"> credentials</span><span class="pun">=</span><span class="pln">credentials</span><span class="pun">,</span></li><li class="L6"><span class="pln"> project_id</span><span class="pun">=</span><span class="pln">project_id</span></li><li class="L7"><span class="pln"> </span><span class="pun">)</span></li><li class="L8"><span class="pln"> mixtral_llm </span><span class="pun">=</span><span class="pln"> </span><span class="typ">WatsonxLLM</span><span class="pun">(</span><span class="pln">model</span><span class="pun">=</span><span class="pln">model</span><span class="pun">)</span></li><li class="L9"><span class="pln"> response </span><span class="pun">=</span><span class="pln"> mixtral_llm</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">prompt_txt</span><span class="pun">)</span></li><li class="L0"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> response</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-20">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Zero-shot prompt</td>
<td>Zero-shot learning is crucial for testing a model's ability to apply
its pre-trained knowledge to new, unseen tasks without additional
training. This capability is valuable for gauging the model's
generalization skills.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li></ol><ol class="linenums"><li class="L0"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""Classify the following statement as true or false: </span></li><li class="L1"><span class="str"> 'The Eiffel Tower is located in Berlin.'</span></li><li class="L2"><span class="str"> Answer:</span></li><li class="L3"><span class="str">"""</span></li><li class="L4"><span class="pln">response </span><span class="pun">=</span><span class="pln"> llm_model</span><span class="pun">(</span><span class="pln">prompt</span><span class="pun">,</span><span class="pln"> </span><span class="kwd">params</span><span class="pun">)</span></li><li class="L5"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"prompt: {prompt}\n"</span><span class="pun">)</span></li><li class="L6"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"response : {response}\n"</span><span class="pun">)</span><span class="pln"> </span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-21">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>One-shot prompt</td>
<td>One-shot learning example where the model is given a single example
to help guide its translation from English to French.
The prompt provides a sample translation pairing, "How is the weather
today?" translated to "Comment est le temps aujourd'hui?" This example
serves as a guide for the model to understand the task context and
desired format. The model is then tasked with translating a new
sentence, "Where is the nearest supermarket?" without further guidance.
</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li></ol><ol class="linenums"><li class="L0"><span class="kwd">params</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L1"><span class="pln"> </span><span class="str">"max_new_tokens"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">20</span><span class="pun">,</span></li><li class="L2"><span class="pln"> </span><span class="str">"temperature"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">0.1</span><span class="pun">,</span></li><li class="L3"><span class="pun">}</span></li><li class="L4"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""Here is an example of translating a sentence from English to French:</span></li><li class="L5"><span class="str"> English: “How is the weather today?”</span></li><li class="L6"><span class="str"> French: “Comment est le temps aujourd'hui?”</span></li><li class="L7"><span class="str"> Now, translate the following sentence from English to French:</span></li><li class="L8"><span class="str"> English: “Where is the nearest supermarket?”</span></li><li class="L9"><span class="str">"""</span></li><li class="L0"><span class="pln">response </span><span class="pun">=</span><span class="pln"> llm_model</span><span class="pun">(</span><span class="pln">prompt</span><span class="pun">,</span><span class="pln"> </span><span class="kwd">params</span><span class="pun">)</span></li><li class="L1"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"prompt: {prompt}\n"</span><span class="pun">)</span></li><li class="L2"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"response : {response}\n"</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-22">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Few-shot prompt</td>
<td>This code snippet classifies emotions using a few-shot learning
approach. The prompt includes various examples where statements are
associated with their respective emotions.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li></ol><ol class="linenums"><li class="L0"><span class="com">#parameters `max_new_tokens` to 10, which constrains the model to generate brief responses</span></li><li class="L1"><span class="kwd">params</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L2"><span class="pln"> </span><span class="str">"max_new_tokens"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">10</span><span class="pun">,</span></li><li class="L3"><span class="pun">}</span></li><li class="L4"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""Here are few examples of classifying emotions in statements:</span></li><li class="L5"><span class="str"> Statement: 'I just won my first marathon!'</span></li><li class="L6"><span class="str"> Emotion: Joy</span></li><li class="L7"><span class="str"> Statement: 'I can't believe I lost my keys again.'</span></li><li class="L8"><span class="str"> Emotion: Frustration</span></li><li class="L9"><span class="str"> Statement: 'My best friend is moving to another country.'</span></li><li class="L0"><span class="str"> Emotion: Sadness</span></li><li class="L1"><span class="str"> Now, classify the emotion in the following statement:</span></li><li class="L2"><span class="str"> Statement: 'That movie was so scary I had to cover my eyes.’</span></li><li class="L3"><span class="str">"""</span></li><li class="L4"><span class="pln">response </span><span class="pun">=</span><span class="pln"> llm_model</span><span class="pun">(</span><span class="pln">prompt</span><span class="pun">,</span><span class="pln"> </span><span class="kwd">params</span><span class="pun">)</span></li><li class="L5"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"prompt: {prompt}\n"</span><span class="pun">)</span></li><li class="L6"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"response : {response}\n"</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-23">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>Chain-of-thought (CoT) prompting</td>
<td>The Chain-of-Thought (CoT) prompting technique, designed to guide
the model through a sequence of reasoning steps to solve a problem.
<p>The CoT technique involves structuring the prompt by instructing the
model to "Break down each step of your calculation." This encourages the
model to include explicit reasoning steps, mimicking human-like
problem-solving processes.</p>
</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li></ol><ol class="linenums"><li class="L0"><span class="kwd">params</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L1"><span class="pln"> </span><span class="str">"max_new_tokens"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">512</span><span class="pun">,</span></li><li class="L2"><span class="pln"> </span><span class="str">"temperature"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">0.5</span><span class="pun">,</span></li><li class="L3"><span class="pun">}</span></li><li class="L4"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""Consider the problem: 'A store had 22 apples. They sold 15 apples today and got a new delivery of 8 apples. </span></li><li class="L5"><span class="str"> How many apples are there now?’</span></li><li class="L6"><span class="str"> Break down each step of your calculation</span></li><li class="L7"><span class="str">"""</span></li><li class="L8"><span class="pln">response </span><span class="pun">=</span><span class="pln"> llm_model</span><span class="pun">(</span><span class="pln">prompt</span><span class="pun">,</span><span class="pln"> </span><span class="kwd">params</span><span class="pun">)</span></li><li class="L9"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"prompt: {prompt}\n"</span><span class="pun">)</span></li><li class="L0"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"response : {response}\n"</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-24">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Self-consistency</td>
<td>This code snippet determines the consistent result for age-related
problems and generates multiple responses. The 'params' dictionary
specifies the maximum number of tokens to generate responses.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li></ol><ol class="linenums"><li class="L0"><span class="kwd">params</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L1"><span class="pln"> </span><span class="str">"max_new_tokens"</span><span class="pun">:</span><span class="pln"> </span><span class="lit">512</span><span class="pun">,</span></li><li class="L2"><span class="pun">}</span></li><li class="L3"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""When I was 6, my sister was half of my age. Now I am 70, what age is my sister?</span></li><li class="L4"><span class="str"> Provide three independent calculations and explanations, then determine the most consistent result.</span></li><li class="L5"><span class="str">"""</span></li><li class="L6"><span class="pln">response </span><span class="pun">=</span><span class="pln"> llm_model</span><span class="pun">(</span><span class="pln">prompt</span><span class="pun">,</span><span class="pln"> </span><span class="kwd">params</span><span class="pun">)</span></li><li class="L7"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"prompt: {prompt}\n"</span><span class="pun">)</span></li><li class="L8"><span class="kwd">print</span><span class="pun">(</span><span class="pln">f</span><span class="str">"response : {response}\n"</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-25">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>Prompt template</td>
<td>A key concept in LangChain, it helps to translate user input and
parameters into instructions for a language model. This can be used to
guide a model's response, helping it understand the context and generate
relevant and coherent language-based output.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li></ol><ol class="linenums"><li class="L0"><span class="pln">model_id </span><span class="pun">=</span><span class="pln"> </span><span class="str">'mistralai/mixtral-8x7b-instruct-v01'</span></li><li class="L1"><span class="pln">parameters </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L2"><span class="pln"> </span><span class="typ">GenParams</span><span class="pun">.</span><span class="pln">MAX_NEW_TOKENS</span><span class="pun">:</span><span class="pln"> </span><span class="lit">256</span><span class="pun">,</span><span class="pln"> </span><span class="com"># this controls the maximum number of tokens in the generated output</span></li><li class="L3"><span class="pln"> </span><span class="typ">GenParams</span><span class="pun">.</span><span class="pln">TEMPERATURE</span><span class="pun">:</span><span class="pln"> </span><span class="lit">0.5</span><span class="pun">,</span><span class="pln"> </span><span class="com"># this randomness or creativity of the model's responses</span></li><li class="L4"><span class="pun">}</span></li><li class="L5"><span class="pln">credentials </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></li><li class="L6"><span class="pln"> </span><span class="str">"url"</span><span class="pun">:</span><span class="pln"> </span><span class="str">"https://us-south.ml.cloud.ibm.com"</span></li><li class="L7"><span class="pun">}</span></li><li class="L8"><span class="pln">project_id </span><span class="pun">=</span><span class="pln"> </span><span class="str">"skills-network"</span></li><li class="L9"><span class="pln">model </span><span class="pun">=</span><span class="pln"> </span><span class="typ">Model</span><span class="pun">(</span></li><li class="L0"><span class="pln"> model_id</span><span class="pun">=</span><span class="pln">model_id</span><span class="pun">,</span></li><li class="L1"><span class="pln"> </span><span class="kwd">params</span><span class="pun">=</span><span class="pln">parameters</span><span class="pun">,</span></li><li class="L2"><span class="pln"> credentials</span><span class="pun">=</span><span class="pln">credentials</span><span class="pun">,</span></li><li class="L3"><span class="pln"> project_id</span><span class="pun">=</span><span class="pln">project_id</span></li><li class="L4"><span class="pun">)</span></li><li class="L5"><span class="pln">mixtral_llm </span><span class="pun">=</span><span class="pln"> </span><span class="typ">WatsonxLLM</span><span class="pun">(</span><span class="pln">model</span><span class="pun">=</span><span class="pln">model</span><span class="pun">)</span></li><li class="L6"><span class="pln">mixtral_llm</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-26">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Text summarization</td>
<td>Text summarization agent designed to help summarize the content you provide to the LLM.
You can store the content to be summarized in a variable, allowing for repeated use of the prompt.
</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li></ol><ol class="linenums"><li class="L0"><span class="pln">content </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""</span></li><li class="L1"><span class="str">
The rapid advancement of technology in the 21st century has
transformed various industries, including healthcare, education, and
transportation. </span></li><li class="L2"><span class="str">
Innovations such as artificial intelligence, machine learning, and the
Internet of Things have revolutionized how we approach everyday tasks
and complex problems. </span></li><li class="L3"><span class="str">
For instance, AI-powered diagnostic tools are improving the accuracy
and speed of medical diagnoses, while smart transportation systems are
making cities more efficient and reducing traffic congestion. </span></li><li class="L4"><span class="str">
Moreover, online learning platforms are making education more
accessible to people around the world, breaking down geographical and
financial barriers. </span></li><li class="L5"><span class="str">
These technological developments are not only enhancing productivity
but also contributing to a more interconnected and informed society.</span></li><li class="L6"><span class="str">"""</span></li><li class="L7"><span class="kwd">template</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""Summarize the {content} in one sentence.</span></li><li class="L8"><span class="str">"""</span></li><li class="L9"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">.</span><span class="pln">from_template</span><span class="pun">(</span><span class="kwd">template</span><span class="pun">)</span></li><li class="L0"><span class="pln">llm_chain </span><span class="pun">=</span><span class="pln"> </span><span class="typ">LLMChain</span><span class="pun">(</span><span class="pln">prompt</span><span class="pun">=</span><span class="pln">prompt</span><span class="pun">,</span><span class="pln"> llm</span><span class="pun">=</span><span class="pln">mixtral_llm</span><span class="pun">)</span></li><li class="L1"><span class="pln">response </span><span class="pun">=</span><span class="pln"> llm_chain</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">input </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span><span class="str">"content"</span><span class="pun">:</span><span class="pln"> content</span><span class="pun">})</span></li><li class="L2"><span class="kwd">print</span><span class="pun">(</span><span class="pln">response</span><span class="pun">[</span><span class="str">"text"</span><span class="pun">])</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-27">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>Question answering</td>
<td>An agent that enables the LLM to learn from the provided content and
answer questions based on what it has learned. Occasionally, if the LLM
does not have sufficient information, it might generate a speculative
answer. To manage this, you'll specifically instruct it to respond with
"Unsure about the answer" if it is uncertain about the correct response.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li></ol><ol class="linenums"><li class="L0"><span class="pln">content </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""</span></li><li class="L1"><span class="str">
The solar system consists of the Sun, eight planets, their
moons, dwarf planets, and smaller objects like asteroids and comets. </span></li><li class="L2"><span class="str"> The inner planets—Mercury, Venus, Earth, and Mars—are rocky and solid. </span></li><li class="L3"><span class="str"> The outer planets—Jupiter, Saturn, Uranus, and Neptune—are much larger and gaseous.</span></li><li class="L4"><span class="str">"""</span></li><li class="L5"><span class="pln">question </span><span class="pun">=</span><span class="pln"> </span><span class="str">"Which planets in the solar system are rocky and solid?"</span></li><li class="L6"><span class="kwd">template</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""</span></li><li class="L7"><span class="str"> Answer the {question} based on the {content}.</span></li><li class="L8"><span class="str"> Respond "</span><span class="typ">Unsure</span><span class="pln"> about answer</span><span class="str">" if not sure about the answer.</span></li><li class="L9"><span class="str"> Answer:</span></li><li class="L0"><span class="str">"""</span></li><li class="L1"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">.</span><span class="pln">from_template</span><span class="pun">(</span><span class="kwd">template</span><span class="pun">)</span></li><li class="L2"><span class="pln">output_key </span><span class="pun">=</span><span class="pln"> </span><span class="str">"answer"</span></li><li class="L3"><span class="pln">llm_chain </span><span class="pun">=</span><span class="pln"> </span><span class="typ">LLMChain</span><span class="pun">(</span><span class="pln">prompt</span><span class="pun">=</span><span class="pln">prompt</span><span class="pun">,</span><span class="pln"> llm</span><span class="pun">=</span><span class="pln">mixtral_llm</span><span class="pun">,</span><span class="pln"> output_key</span><span class="pun">=</span><span class="pln">output_key</span><span class="pun">)</span></li><li class="L4"><span class="pln">response </span><span class="pun">=</span><span class="pln"> llm_chain</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">input </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span><span class="str">"question"</span><span class="pun">:</span><span class="pln">question </span><span class="pun">,</span><span class="str">"content"</span><span class="pun">:</span><span class="pln"> content</span><span class="pun">})</span></li><li class="L5"><span class="kwd">print</span><span class="pun">(</span><span class="pln">response</span><span class="pun">[</span><span class="str">"answer"</span><span class="pun">])</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-28">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Code generation</td>
<td>An agent that is designed to generate SQL queries based on given
descriptions. It interprets the requirements from your input and
translates them into executable SQL code.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li></ol><ol class="linenums"><li class="L0"><span class="pln">description </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""</span></li><li class="L1"><span class="str">
Retrieve the names and email addresses of all customers from the
'customers' table who have made a purchase in the last 30 days. </span></li><li class="L2"><span class="str"> The table 'purchases' contains a column 'purchase_date'</span></li><li class="L3"><span class="str">"""</span></li><li class="L4"><span class="kwd">template</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""</span></li><li class="L5"><span class="str"> Generate an SQL query based on the {description}</span></li><li class="L6"><span class="str"> SQL Query:</span></li><li class="L7"><span class="str">"""</span></li><li class="L8"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">.</span><span class="pln">from_template</span><span class="pun">(</span><span class="kwd">template</span><span class="pun">)</span></li><li class="L9"><span class="pln">output_key </span><span class="pun">=</span><span class="pln"> </span><span class="str">"query"</span></li><li class="L0"><span class="pln">llm_chain </span><span class="pun">=</span><span class="pln"> </span><span class="typ">LLMChain</span><span class="pun">(</span><span class="pln">prompt</span><span class="pun">=</span><span class="pln">prompt</span><span class="pun">,</span><span class="pln"> llm</span><span class="pun">=</span><span class="pln">mixtral_llm</span><span class="pun">,</span><span class="pln"> output_key</span><span class="pun">=</span><span class="pln">output_key</span><span class="pun">)</span></li><li class="L1"><span class="pln">response </span><span class="pun">=</span><span class="pln"> llm_chain</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">input </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span><span class="str">"description"</span><span class="pun">:</span><span class="pln">description</span><span class="pun">})</span></li><li class="L2"><span class="kwd">print</span><span class="pun">(</span><span class="pln">response</span><span class="pun">[</span><span class="str">"query"</span><span class="pun">])</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-29">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>Role playing</td>
<td>Configures the LLM to assume specific roles as defined by us,
enabling it to follow predetermined rules and behave like a
task-oriented chatbot.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li></ol><ol class="linenums"><li class="L0"><span class="pln">role </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""</span></li><li class="L1"><span class="str"> game master</span></li><li class="L2"><span class="str">"""</span></li><li class="L3"><span class="pln">tone </span><span class="pun">=</span><span class="pln"> </span><span class="str">"engaging and immersive"</span></li><li class="L4"><span class="kwd">template</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="str">"""</span></li><li class="L5"><span class="str"> You are an expert {role}. I have this question {question}. I would like our conversation to be {tone}.</span></li><li class="L6"><span class="str"> Answer:</span></li><li class="L7"><span class="str">"""</span></li><li class="L8"><span class="pln">prompt </span><span class="pun">=</span><span class="pln"> </span><span class="typ">PromptTemplate</span><span class="pun">.</span><span class="pln">from_template</span><span class="pun">(</span><span class="kwd">template</span><span class="pun">)</span></li><li class="L9"><span class="pln">output_key </span><span class="pun">=</span><span class="pln"> </span><span class="str">"answer"</span></li><li class="L0"><span class="pln">llm_chain </span><span class="pun">=</span><span class="pln"> </span><span class="typ">LLMChain</span><span class="pun">(</span><span class="pln">prompt</span><span class="pun">=</span><span class="pln">prompt</span><span class="pun">,</span><span class="pln"> llm</span><span class="pun">=</span><span class="pln">mixtral_llm</span><span class="pun">,</span><span class="pln"> output_key</span><span class="pun">=</span><span class="pln">output_key</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-30">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>class_names</td>
<td>This code snippet maps numerical labels to their corresponding
textual descriptions to classify tasks. This code helps in machine
learning to interpret the output model, where the model's predictions
are numerical and should be presented in a more human-readable format.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li></ol><ol class="linenums"><li class="L0"><span class="pln">class_names </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span><span class="lit">0</span><span class="pun">:</span><span class="pln"> </span><span class="str">"negative"</span><span class="pun">,</span><span class="pln"> </span><span class="lit">1</span><span class="pun">:</span><span class="pln"> </span><span class="str">"positive"</span><span class="pun">}</span></li><li class="L1"><span class="pln">class_names</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-31">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>read_and_split_text</td>
<td>Involves opening the file, reading its contents, and splitting the
text into individual paragraphs. Each paragraph represents a section of
the company policies. You can also filter out any empty paragraphs to
clean your data set.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> read_and_split_text</span><span class="pun">(</span><span class="pln">filename</span><span class="pun">):</span></li><li class="L1"><span class="pln"> </span><span class="kwd">with</span><span class="pln"> open</span><span class="pun">(</span><span class="pln">filename</span><span class="pun">,</span><span class="pln"> </span><span class="str">'r'</span><span class="pun">,</span><span class="pln"> encoding</span><span class="pun">=</span><span class="str">'utf-8'</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">as</span><span class="pln"> file</span><span class="pun">:</span></li><li class="L2"><span class="pln"> text </span><span class="pun">=</span><span class="pln"> file</span><span class="pun">.</span><span class="pln">read</span><span class="pun">()</span></li><li class="L3"><span class="pln"> </span><span class="com"># Split the text into paragraphs (simple split by newline characters)</span></li><li class="L4"><span class="pln"> paragraphs </span><span class="pun">=</span><span class="pln"> text</span><span class="pun">.</span><span class="pln">split</span><span class="pun">(</span><span class="str">'\n'</span><span class="pun">)</span></li><li class="L5"><span class="pln"> </span><span class="com"># Filter out any empty paragraphs or undesired entries</span></li><li class="L6"><span class="pln"> paragraphs </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">para</span><span class="pun">.</span><span class="pln">strip</span><span class="pun">()</span><span class="pln"> </span><span class="kwd">for</span><span class="pln"> para </span><span class="kwd">in</span><span class="pln"> paragraphs </span><span class="kwd">if</span><span class="pln"> len</span><span class="pun">(</span><span class="pln">para</span><span class="pun">.</span><span class="pln">strip</span><span class="pun">())</span><span class="pln"> </span><span class="pun">></span><span class="pln"> </span><span class="lit">0</span><span class="pun">]</span></li><li class="L7"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> paragraphs</span></li><li class="L8"><span class="com"># Read the text file and split it into paragraphs</span></li><li class="L9"><span class="pln">paragraphs </span><span class="pun">=</span><span class="pln"> read_and_split_text</span><span class="pun">(</span><span class="str">'companyPolicies.txt'</span><span class="pun">)</span></li><li class="L0"><span class="pln">paragraphs</span><span class="pun">[</span><span class="lit">0</span><span class="pun">:</span><span class="lit">10</span><span class="pun">]</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-32">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>encode_contexts</td>
<td>This code snippet encodes a list of texts into embeddings using
content_tokenizer and context_encoder. This code helps iterate through
each text in the input list, tokenizes and encodes it, and then appends
the pooler_output to the embeddings list. The resulting embeddings get
stored in the context_embeddings variables and generate embeddings from
text data for various natural language processing (NLP) applications.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> encode_contexts</span><span class="pun">(</span><span class="pln">text_list</span><span class="pun">):</span></li><li class="L1"><span class="pln"> </span><span class="com"># Encode a list of texts into embeddings</span></li><li class="L2"><span class="pln"> embeddings </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[]</span></li><li class="L3"><span class="pln"> </span><span class="kwd">for</span><span class="pln"> text </span><span class="kwd">in</span><span class="pln"> text_list</span><span class="pun">:</span></li><li class="L4"><span class="pln"> inputs </span><span class="pun">=</span><span class="pln"> context_tokenizer</span><span class="pun">(</span><span class="pln">text</span><span class="pun">,</span><span class="pln"> return_tensors</span><span class="pun">=</span><span class="str">'pt'</span><span class="pun">,</span><span class="pln"> padding</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">,</span><span class="pln"> truncation</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">,</span><span class="pln"> max_length</span><span class="pun">=</span><span class="lit">256</span><span class="pun">)</span></li><li class="L5"><span class="pln"> outputs </span><span class="pun">=</span><span class="pln"> context_encoder</span><span class="pun">(**</span><span class="pln">inputs</span><span class="pun">)</span></li><li class="L6"><span class="pln"> embeddings</span><span class="pun">.</span><span class="pln">append</span><span class="pun">(</span><span class="pln">outputs</span><span class="pun">.</span><span class="pln">pooler_output</span><span class="pun">)</span></li><li class="L7"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> torch</span><span class="pun">.</span><span class="pln">cat</span><span class="pun">(</span><span class="pln">embeddings</span><span class="pun">).</span><span class="pln">detach</span><span class="pun">().</span><span class="pln">numpy</span><span class="pun">()</span></li><li class="L8"><span class="com"># you would now encode these paragraphs to create embeddings.</span></li><li class="L9"><span class="pln">context_embeddings </span><span class="pun">=</span><span class="pln"> encode_contexts</span><span class="pun">(</span><span class="pln">paragraphs</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-33">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>import faiss</td>
<td>FAISS (Facebook AI Similarity Search) is an efficient library
developed by Facebook for similarity search and clustering of dense
vectors. FAISS is designed for fast similarity search, which is
particularly valuable when dealing with large data sets. It is highly
suitable for tasks in natural language processing where retrieval speed
is critical.
It effectively handles large volumes of data, maintaining performance
even as data set sizes increase.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li></ol><ol class="linenums"><li class="L0"><span class="kwd">import</span><span class="pln"> faiss</span></li><li class="L1"><span class="com"># Convert list of numpy arrays into a single numpy array</span></li><li class="L2"><span class="pln">embedding_dim </span><span class="pun">=</span><span class="pln"> </span><span class="lit">768</span><span class="pln"> </span><span class="com"># This should match the dimension of your embeddings</span></li><li class="L3"><span class="pln">context_embeddings_np </span><span class="pun">=</span><span class="pln"> np</span><span class="pun">.</span><span class="pln">array</span><span class="pun">(</span><span class="pln">context_embeddings</span><span class="pun">).</span><span class="pln">astype</span><span class="pun">(</span><span class="str">'float32'</span><span class="pun">)</span></li><li class="L4"><span class="com"># Create a FAISS index for the embeddings</span></li><li class="L5"><span class="pln">index </span><span class="pun">=</span><span class="pln"> faiss</span><span class="pun">.</span><span class="typ">IndexFlatL2</span><span class="pun">(</span><span class="pln">embedding_dim</span><span class="pun">)</span></li><li class="L6"><span class="pln">index</span><span class="pun">.</span><span class="pln">add</span><span class="pun">(</span><span class="pln">context_embeddings_np</span><span class="pun">)</span><span class="pln"> </span><span class="com"># Add the context embeddings to the index</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-34">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>search_relevant_contexts</td>
<td>This code snippet is useful in searching relevant contexts for a
given question. It tokenizes the question using the question_tokenizer,
encodes the question using question_encoder, and searches an index for
retrieving the relevant context based on question embedding.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> search_relevant_contexts</span><span class="pun">(</span><span class="pln">question</span><span class="pun">,</span><span class="pln"> question_tokenizer</span><span class="pun">,</span><span class="pln"> question_encoder</span><span class="pun">,</span><span class="pln"> index</span><span class="pun">,</span><span class="pln"> k</span><span class="pun">=</span><span class="lit">5</span><span class="pun">):</span></li><li class="L1"><span class="pln"> </span><span class="str">"""</span></li><li class="L2"><span class="str"> Searches for the most relevant contexts to a given question.</span></li><li class="L3"><span class="str"> Returns:</span></li><li class="L4"><span class="str"> tuple: Distances and indices of the top k relevant contexts.</span></li><li class="L5"><span class="str"> """</span></li><li class="L6"><span class="pln"> </span><span class="com"># Tokenize the question</span></li><li class="L7"><span class="pln"> question_inputs </span><span class="pun">=</span><span class="pln"> question_tokenizer</span><span class="pun">(</span><span class="pln">question</span><span class="pun">,</span><span class="pln"> return_tensors</span><span class="pun">=</span><span class="str">'pt'</span><span class="pun">)</span></li><li class="L8"><span class="pln"> </span><span class="com"># Encode the question to get the embedding</span></li><li class="L9"><span class="pln"> question_embedding </span><span class="pun">=</span><span class="pln"> question_encoder</span><span class="pun">(**</span><span class="pln">question_inputs</span><span class="pun">).</span><span class="pln">pooler_output</span><span class="pun">.</span><span class="pln">detach</span><span class="pun">().</span><span class="pln">numpy</span><span class="pun">()</span></li><li class="L0"><span class="pln"> </span><span class="com"># Search the index to retrieve top k relevant contexts</span></li><li class="L1"><span class="pln"> D</span><span class="pun">,</span><span class="pln"> I </span><span class="pun">=</span><span class="pln"> index</span><span class="pun">.</span><span class="pln">search</span><span class="pun">(</span><span class="pln">question_embedding</span><span class="pun">,</span><span class="pln"> k</span><span class="pun">)</span></li><li class="L2"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> D</span><span class="pun">,</span><span class="pln"> I</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-35">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>generate_answer_without_context</td>
<td>This code snippet generates responses using the entered prompt
without requiring additional context. It tokenizes the input questions
using the tokenizer, generates the output text using the model, and
decodes the generated text to obtain the answer.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> generate_answer_without_context</span><span class="pun">(</span><span class="pln">question</span><span class="pun">):</span></li><li class="L1"><span class="pln"> </span><span class="com"># Tokenize the input question</span></li><li class="L2"><span class="pln"> inputs </span><span class="pun">=</span><span class="pln"> tokenizer</span><span class="pun">(</span><span class="pln">question</span><span class="pun">,</span><span class="pln"> return_tensors</span><span class="pun">=</span><span class="str">'pt'</span><span class="pun">,</span><span class="pln"> max_length</span><span class="pun">=</span><span class="lit">1024</span><span class="pun">,</span><span class="pln"> truncation</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">)</span></li><li class="L3"><span class="pln"> </span><span class="com"># Generate output directly from the question without additional context</span></li><li class="L4"><span class="pln"> summary_ids </span><span class="pun">=</span><span class="pln"> model</span><span class="pun">.</span><span class="pln">generate</span><span class="pun">(</span><span class="pln">inputs</span><span class="pun">[</span><span class="str">'input_ids'</span><span class="pun">],</span><span class="pln"> max_length</span><span class="pun">=</span><span class="lit">150</span><span class="pun">,</span><span class="pln"> min_length</span><span class="pun">=</span><span class="lit">40</span><span class="pun">,</span><span class="pln"> length_penalty</span><span class="pun">=</span><span class="lit">2.0</span><span class="pun">,</span><span class="pln"> num_beams</span><span class="pun">=</span><span class="lit">4</span><span class="pun">,</span><span class="pln"> early_stopping</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">)</span></li><li class="L5"><span class="pln"> </span><span class="com"># Decode and return the generated text</span></li><li class="L6"><span class="pln"> answer </span><span class="pun">=</span><span class="pln"> tokenizer</span><span class="pun">.</span><span class="pln">decode</span><span class="pun">(</span><span class="pln">summary_ids</span><span class="pun">[</span><span class="lit">0</span><span class="pun">],</span><span class="pln"> skip_special_tokens</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">)</span></li><li class="L7"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> answer</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-36">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>Generating answers with DPR contexts</td>
<td>Answers are generated when the model utilizes contexts retrieved via
DPR, which are expected to enhance the answer's relevance and depth:</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> generate_answer</span><span class="pun">(</span><span class="pln">contexts</span><span class="pun">):</span></li><li class="L1"><span class="pln"> </span><span class="com"># Concatenate the retrieved contexts to form the input to BART</span></li><li class="L2"><span class="pln"> input_text </span><span class="pun">=</span><span class="pln"> </span><span class="str">' '</span><span class="pun">.</span><span class="pln">join</span><span class="pun">(</span><span class="pln">contexts</span><span class="pun">)</span></li><li class="L3"><span class="pln"> inputs </span><span class="pun">=</span><span class="pln"> tokenizer</span><span class="pun">(</span><span class="pln">input_text</span><span class="pun">,</span><span class="pln"> return_tensors</span><span class="pun">=</span><span class="str">'pt'</span><span class="pun">,</span><span class="pln"> max_length</span><span class="pun">=</span><span class="lit">1024</span><span class="pun">,</span><span class="pln"> truncation</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">)</span></li><li class="L4"><span class="pln"> </span><span class="com"># Generate output using BART</span></li><li class="L5"><span class="pln"> summary_ids </span><span class="pun">=</span><span class="pln"> model</span><span class="pun">.</span><span class="pln">generate</span><span class="pun">(</span><span class="pln">inputs</span><span class="pun">[</span><span class="str">'input_ids'</span><span class="pun">],</span><span class="pln"> max_length</span><span class="pun">=</span><span class="lit">150</span><span class="pun">,</span><span class="pln"> min_length</span><span class="pun">=</span><span class="lit">40</span><span class="pun">,</span><span class="pln"> length_penalty</span><span class="pun">=</span><span class="lit">2.0</span><span class="pun">,</span><span class="pln"> num_beams</span><span class="pun">=</span><span class="lit">4</span><span class="pun">,</span><span class="pln"> early_stopping</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">)</span></li><li class="L6"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> tokenizer</span><span class="pun">.</span><span class="pln">decode</span><span class="pun">(</span><span class="pln">summary_ids</span><span class="pun">[</span><span class="lit">0</span><span class="pun">],</span><span class="pln"> skip_special_tokens</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-37">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>aggregate_embeddings function</td>
<td>The function aggregate_embeddings takes token indices and their
corresponding attention masks, and uses a BERT model to convert these
tokens into word embeddings. It then filters out the embeddings for
zero-padded tokens and computes the mean embedding for each sequence.
This helps in reducing the dimensionality of the data while retaining
the most important information from the embeddings.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li><li>13</li><li>14</li><li>15</li><li>16</li><li>17</li><li>18</li><li>19</li><li>20</li><li>21</li><li>22</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> aggregate_embeddings</span><span class="pun">(</span><span class="pln">input_ids</span><span class="pun">,</span><span class="pln"> attention_masks</span><span class="pun">,</span><span class="pln"> bert_model</span><span class="pun">=</span><span class="pln">bert_model</span><span class="pun">):</span></li><li class="L1"><span class="pln"> </span><span class="str">"""</span></li><li class="L2"><span class="str"> Converts token indices and masks to word embeddings, filters out zero-padded embeddings,</span></li><li class="L3"><span class="str"> and aggregates them by computing the mean embedding for each input sequence.</span></li><li class="L4"><span class="str"> """</span></li><li class="L5"><span class="pln"> mean_embeddings </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[]</span></li><li class="L6"><span class="pln"> </span><span class="com"># Process each sequence in the batch</span></li><li class="L7"><span class="pln"> </span><span class="kwd">print</span><span class="pun">(</span><span class="str">'number of inputs'</span><span class="pun">,</span><span class="pln">len</span><span class="pun">(</span><span class="pln">input_ids</span><span class="pun">))</span></li><li class="L8"><span class="pln"> </span><span class="kwd">for</span><span class="pln"> input_id</span><span class="pun">,</span><span class="pln"> mask </span><span class="kwd">in</span><span class="pln"> tqdm</span><span class="pun">(</span><span class="pln">zip</span><span class="pun">(</span><span class="pln">input_ids</span><span class="pun">,</span><span class="pln"> attention_masks</span><span class="pun">)):</span></li><li class="L9"><span class="pln"> input_ids_tensor </span><span class="pun">=</span><span class="pln"> torch</span><span class="pun">.</span><span class="pln">tensor</span><span class="pun">([</span><span class="pln">input_id</span><span class="pun">]).</span><span class="pln">to</span><span class="pun">(</span><span class="pln">DEVICE</span><span class="pun">)</span></li><li class="L0"><span class="pln"> mask_tensor </span><span class="pun">=</span><span class="pln"> torch</span><span class="pun">.</span><span class="pln">tensor</span><span class="pun">([</span><span class="pln">mask</span><span class="pun">]).</span><span class="pln">to</span><span class="pun">(</span><span class="pln">DEVICE</span><span class="pun">)</span></li><li class="L1"><span class="pln"> </span><span class="kwd">with</span><span class="pln"> torch</span><span class="pun">.</span><span class="pln">no_grad</span><span class="pun">():</span></li><li class="L2"><span class="pln"> </span><span class="com"># Obtain the word embeddings from the BERT model</span></li><li class="L3"><span class="pln"> word_embeddings </span><span class="pun">=</span><span class="pln"> bert_model</span><span class="pun">(</span><span class="pln">input_ids_tensor</span><span class="pun">,</span><span class="pln"> attention_mask</span><span class="pun">=</span><span class="pln">mask_tensor</span><span class="pun">)[</span><span class="lit">0</span><span class="pun">].</span><span class="pln">squeeze</span><span class="pun">(</span><span class="lit">0</span><span class="pun">)</span></li><li class="L4"><span class="pln"> </span><span class="com"># Filter out the embeddings at positions where the mask is zero </span></li><li class="L5"><span class="pln"> valid_embeddings_mask</span><span class="pun">=</span><span class="pln">mask_tensor</span><span class="pun">[</span><span class="lit">0</span><span class="pun">]</span><span class="pln"> </span><span class="pun">!=</span><span class="pln"> </span><span class="lit">0</span><span class="pln"> </span></li><li class="L6"><span class="pln"> valid_embeddings </span><span class="pun">=</span><span class="pln"> word_embeddings</span><span class="pun">[</span><span class="pln">valid_embeddings_mask</span><span class="pun">,:]</span></li><li class="L7"><span class="pln"> </span><span class="com"># Compute the mean of the filtered embeddings</span></li><li class="L8"><span class="pln"> mean_embedding </span><span class="pun">=</span><span class="pln"> valid_embeddings</span><span class="pun">.</span><span class="pln">mean</span><span class="pun">(</span><span class="pln">dim</span><span class="pun">=</span><span class="lit">0</span><span class="pun">)</span><span class="pln"> mean_embeddings</span><span class="pun">.</span><span class="pln">append</span><span class="pun">(</span><span class="pln">mean_embedding</span><span class="pun">.</span><span class="pln">unsqueeze</span><span class="pun">(</span><span class="lit">0</span><span class="pun">))</span></li><li class="L9"><span class="pln"> </span><span class="com"># Concatenate the mean embeddings from all sequences in the batch</span></li><li class="L0"><span class="pln"> aggregated_mean_embeddings </span><span class="pun">=</span><span class="pln"> torch</span><span class="pun">.</span><span class="pln">cat</span><span class="pun">(</span><span class="pln">mean_embeddings</span><span class="pun">)</span></li><li class="L1"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> aggregated_mean_embeddings</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-38">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>text_to_emb</td>
<td>Designed to convert a list of text strings into their corresponding embeddings using a pre-defined tokenizer.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> text_to_emb</span><span class="pun">(</span><span class="pln">list_of_text</span><span class="pun">,</span><span class="pln">max_input</span><span class="pun">=</span><span class="lit">512</span><span class="pun">):</span></li><li class="L1"><span class="pln"> data_token_index </span><span class="pun">=</span><span class="pln"> tokenizer</span><span class="pun">.</span><span class="pln">batch_encode_plus</span><span class="pun">(</span><span class="pln">list_of_text</span><span class="pun">,</span><span class="pln"> add_special_tokens</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">,</span><span class="pln">padding</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">,</span><span class="pln">truncation</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">,</span><span class="pln">max_length</span><span class="pun">=</span><span class="pln">max_input</span><span class="pun">)</span><span class="pln"> question_embeddings</span><span class="pun">=</span><span class="pln">aggregate_embeddings</span><span class="pun">(</span><span class="pln">data_token_index</span><span class="pun">[</span><span class="str">'input_ids'</span><span class="pun">],</span><span class="pln"> data_token_index</span><span class="pun">[</span><span class="str">'attention_mask'</span><span class="pun">])</span></li><li class="L2"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> question_embeddings</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-39">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>process_song</td>
<td>Convert both the predefined appropriateness questions and the song
lyrics into "RAG embeddings" and measure the similarity between them to
determine the appropriateness.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li></ol><ol class="linenums"><li class="L0"><span class="kwd">import</span><span class="pln"> re</span></li><li class="L1"><span class="kwd">def</span><span class="pln"> process_song</span><span class="pun">(</span><span class="pln">song</span><span class="pun">):</span></li><li class="L2"><span class="pln"> </span><span class="com"># Remove line breaks from the song</span></li><li class="L3"><span class="pln"> song_new </span><span class="pun">=</span><span class="pln"> re</span><span class="pun">.</span><span class="kwd">sub</span><span class="pun">(</span><span class="pln">r</span><span class="str">'[\n]'</span><span class="pun">,</span><span class="pln"> </span><span class="str">' '</span><span class="pun">,</span><span class="pln"> song</span><span class="pun">)</span></li><li class="L4"><span class="pln"> </span><span class="com"># Remove single quotes from the song</span></li><li class="L5"><span class="pln"> song_new </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">song_new</span><span class="pun">.</span><span class="pln">replace</span><span class="pun">(</span><span class="str">"\'"</span><span class="pun">,</span><span class="pln"> </span><span class="str">""</span><span class="pun">)]</span></li><li class="L6"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> song_new</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-40">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>RAG_QA</td>
<td>This code snippet performs question-answering using question
embeddings and provides embeddings. It helps reshape the results for
processing, sorting the indices in descending order, and printing the
top 'n-responses' based on the highest dot product values. </td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li><li>11</li><li>12</li></ol><ol class="linenums"><li class="L0"><span class="kwd">def</span><span class="pln"> RAG_QA</span><span class="pun">(</span><span class="pln">embeddings_questions</span><span class="pun">,</span><span class="pln"> embeddings</span><span class="pun">,</span><span class="pln"> n_responses</span><span class="pun">=</span><span class="lit">3</span><span class="pun">):</span></li><li class="L1"><span class="pln"> </span><span class="com">#
Calculate the dot product between the question embeddings and the
provided embeddings (transpose of the second matrix for proper
alignment).</span></li><li class="L2"><span class="pln"> dot_product </span><span class="pun">=</span><span class="pln"> embeddings_questions </span><span class="pun">@</span><span class="pln"> embeddings</span><span class="pun">.</span><span class="pln">T</span></li><li class="L3"><span class="pln"> </span><span class="com"># Reshape the dot product results to a 1D tensor for easier processing.</span></li><li class="L4"><span class="pln"> dot_product </span><span class="pun">=</span><span class="pln"> dot_product</span><span class="pun">.</span><span class="pln">reshape</span><span class="pun">(-</span><span class="lit">1</span><span class="pun">)</span></li><li class="L5"><span class="pln"> </span><span class="com">#
Sort the indices of the dot product results in descending order
(setting descending to False should be True for typical similarity
tasks).</span></li><li class="L6"><span class="pln"> sorted_indices </span><span class="pun">=</span><span class="pln"> torch</span><span class="pun">.</span><span class="pln">argsort</span><span class="pun">(</span><span class="pln">dot_product</span><span class="pun">,</span><span class="pln"> </span><span class="kwd">descending</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">)</span></li><li class="L7"><span class="pln"> </span><span class="com"># Convert sorted indices to a list for easier iteration.</span></li><li class="L8"><span class="pln"> sorted_indices </span><span class="pun">=</span><span class="pln"> sorted_indices</span><span class="pun">.</span><span class="pln">tolist</span><span class="pun">()</span></li><li class="L9"><span class="pln"> </span><span class="com"># Print the top 'n_responses' responses from the sorted list, which correspond to the highest dot product values.</span></li><li class="L0"><span class="pln"> </span><span class="kwd">for</span><span class="pln"> index </span><span class="kwd">in</span><span class="pln"> sorted_indices</span><span class="pun">[:</span><span class="pln">n_responses</span><span class="pun">]:</span></li><li class="L1"><span class="pln"> </span><span class="kwd">print</span><span class="pun">(</span><span class="pln">yes_responses</span><span class="pun">[</span><span class="pln">index</span><span class="pun">])</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-41">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>model_name_or_path</td>
<td>This code snippet defines the model name to 'gpt2' and initializes
the token and model using the GPT-2 model. In this code, add special
tokens for padding by keeping the maximum sequence length to 1024.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li></ol><ol class="linenums"><li class="L0"><span class="com"># Define the model name or path</span></li><li class="L1"><span class="pln">model_name_or_path </span><span class="pun">=</span><span class="pln"> </span><span class="str">"gpt2"</span></li><li class="L2"><span class="com"># Initialize tokenizer and model</span></li><li class="L3"><span class="pln">tokenizer </span><span class="pun">=</span><span class="pln"> GPT2Tokenizer</span><span class="pun">.</span><span class="pln">from_pretrained</span><span class="pun">(</span><span class="pln">model_name_or_path</span><span class="pun">,</span><span class="pln"> use_fast</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">)</span></li><li class="L4"><span class="pln">model </span><span class="pun">=</span><span class="pln"> GPT2ForSequenceClassification</span><span class="pun">.</span><span class="pln">from_pretrained</span><span class="pun">(</span><span class="pln">model_name_or_path</span><span class="pun">,</span><span class="pln"> num_labels</span><span class="pun">=</span><span class="lit">1</span><span class="pun">)</span></li><li class="L5"><span class="com"># Add special tokens if necessary</span></li><li class="L6"><span class="pln">tokenizer</span><span class="pun">.</span><span class="pln">pad_token </span><span class="pun">=</span><span class="pln"> tokenizer</span><span class="pun">.</span><span class="pln">eos_token</span></li><li class="L7"><span class="pln">model</span><span class="pun">.</span><span class="pln">config</span><span class="pun">.</span><span class="pln">pad_token_id </span><span class="pun">=</span><span class="pln"> model</span><span class="pun">.</span><span class="pln">config</span><span class="pun">.</span><span class="pln">eos_token_id</span></li><li class="L8"><span class="com"># Define the maximum length</span></li><li class="L9"><span class="pln">max_length </span><span class="pun">=</span><span class="pln"> </span><span class="lit">1024</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-42">Copied!</span></button></pre></td>
</tr>
<tr class="even">
<td>add_combined_columns</td>
<td>This code snippet combines the prompt with chosen and rejected
responses in a data set example. It combines with the 'Human:' and
'Assistant:' for clarity. This function modifies each example in the
'train' split the data set by creating new columns 'prompt_chosen' and
'prompt_rejected' with the combined text.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li><li>7</li><li>8</li><li>9</li><li>10</li></ol><ol class="linenums"><li class="L0"><span class="com"># Define a function to combine 'prompt' with 'chosen' and 'rejected' responses</span></li><li class="L1"><span class="kwd">def</span><span class="pln"> add_combined_columns</span><span class="pun">(</span><span class="pln">example</span><span class="pun">):</span></li><li class="L2"><span class="pln"> </span><span class="com"># Combine 'prompt' with 'chosen' response, formatting it with "Human:" and "Assistant:" labels</span></li><li class="L3"><span class="pln"> example</span><span class="pun">[</span><span class="str">'prompt_chosen'</span><span class="pun">]</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="str">"\n\nHuman: "</span><span class="pln"> </span><span class="pun">+</span><span class="pln"> example</span><span class="pun">[</span><span class="str">"prompt"</span><span class="pun">]</span><span class="pln"> </span><span class="pun">+</span><span class="pln"> </span><span class="str">"\n\nAssistant: "</span><span class="pln"> </span><span class="pun">+</span><span class="pln"> example</span><span class="pun">[</span><span class="str">"chosen"</span><span class="pun">]</span></li><li class="L4"><span class="pln"> </span><span class="com"># Combine 'prompt' with 'rejected' response, formatting it with "Human:" and "Assistant:" labels</span></li><li class="L5"><span class="pln"> example</span><span class="pun">[</span><span class="str">'prompt_rejected'</span><span class="pun">]</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> </span><span class="str">"\n\nHuman: "</span><span class="pln"> </span><span class="pun">+</span><span class="pln"> example</span><span class="pun">[</span><span class="str">"prompt"</span><span class="pun">]</span><span class="pln"> </span><span class="pun">+</span><span class="pln"> </span><span class="str">"\n\nAssistant: "</span><span class="pln"> </span><span class="pun">+</span><span class="pln"> example</span><span class="pun">[</span><span class="str">"rejected"</span><span class="pun">]</span></li><li class="L6"><span class="pln"> </span><span class="com"># Return the modified example</span></li><li class="L7"><span class="pln"> </span><span class="kwd">return</span><span class="pln"> example</span></li><li class="L8"><span class="com"># Apply the function to each example in the 'train' split of the dataset</span></li><li class="L9"><span class="pln">dataset</span><span class="pun">[</span><span class="str">'train'</span><span class="pun">]</span><span class="pln"> </span><span class="pun">=</span><span class="pln"> dataset</span><span class="pun">[</span><span class="str">'train'</span><span class="pun">].</span><span class="pln">map</span><span class="pun">(</span><span class="pln">add_combined_columns</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-43">Copied!</span></button></pre></td>
</tr>
<tr class="odd">
<td>RetrievalQA</td>
<td>This code snippet creates an example for 'RetrievalQA' using a language model and document retriever.</td>
<td><pre class="prettyprint linenums prettyprinted" style=""><ol class="formatted-line-numbers"><li>1</li><li>2</li><li>3</li><li>4</li></ol><ol class="linenums"><li class="L0"><span class="pln">qa </span><span class="pun">=</span><span class="pln"> </span><span class="typ">RetrievalQA</span><span class="pun">.</span><span class="pln">from_chain_type</span><span class="pun">(</span><span class="pln">llm</span><span class="pun">=</span><span class="pln">flan_ul2_llm</span><span class="pun">,</span><span class="pln"> </span></li><li class="L1"><span class="pln"> chain_type</span><span class="pun">=</span><span class="str">"stuff"</span><span class="pun">,</span><span class="pln"> retriever</span><span class="pun">=</span><span class="pln">docsearch</span><span class="pun">.</span><span class="pln">as_retriever</span><span class="pun">(),</span><span class="pln"> return_source_documents</span><span class="pun">=</span><span class="kwd">False</span><span class="pun">)</span></li><li class="L2"><span class="pln">query </span><span class="pun">=</span><span class="pln"> </span><span class="str">"what is mobile policy?"</span></li><li class="L3"><span class="pln">qa</span><span class="pun">.</span><span class="pln">invoke</span><span class="pun">(</span><span class="pln">query</span><span class="pun">)</span></li></ol><button title="Copy" class="action-code-block copy-code-block multiple-lines"><i class="fa fa-copy" aria-hidden="true"></i><span class="popuptext" id="md-code-block-copy-44">Copied!</span></button></pre></td>
</tr>
</tbody>
</table><footer>
<img src="Fundamentals%20of%20Building%20AI%20Agents%20using%20RAG%20and%20LangChain_files/footerlogo.png"> </footer></body></html>