You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/getstarted/evals.md
+7-42
Original file line number
Diff line number
Diff line change
@@ -166,58 +166,23 @@ Output
166
166
3 summarise given text\nSupply chain challenges ... Supply chain challenges in North America, caus... 1
167
167
```
168
168
169
-
Viewing the sample-level results in a CSV file, as shown above, is fine for quick checks but not ideal for detailed analysis or comparing results across evaluation runs. For a better experience, use [app.ragas.io](https://app.ragas.io/) to view, analyze, and compare evaluation results interactively.
169
+
Viewing the sample-level results in a CSV file, as shown above, is fine for quick checks but not ideal for detailed analysis or comparing results across evaluation runs.
170
170
171
+
### Want help in improving your AI application using evals?
171
172
172
-
## Analyzing Results
173
+
In the past 2 years, we have seen and helped improve many AI applications using evals.
173
174
174
-
For this you may sign up and setup [app.ragas.io](https://app.ragas.io) easily. If not, you may use any alternative tools available to you.
175
+
We are compressing this knowledge into a product to replace vibe checks with eval loops so that you can focus on building great AI applications.
175
176
176
-
In order to use the [app.ragas.io](http://app.ragas.io) dashboard, you need to have an account on [app.ragas.io](https://app.ragas.io/). If you don't have one, you can sign up for one [here](https://app.ragas.io/login). You will also need to generate a [Ragas APP token](https://app.ragas.io/dashboard/settings/app-tokens).
177
-
178
-
Once you have the API key, you can use the `upload()` method to export the results to the dashboard.
179
-
180
-
```python
181
-
import os
182
-
os.environ["RAGAS_APP_TOKEN"] ="your_app_token"
183
-
```
184
-
185
-
Now you can view the results in the dashboard by following the link in the output of the `upload()` method.
186
-
187
-
```python
188
-
results.upload()
189
-
```
190
-
191
-

177
+
If you want help with improving and scaling up your AI application using evals.
In the example above, we can see that the LLM-based metric mistakenly marks some summary as accurate, even though it missed critical details like growth numbers and market domain. Such mistakes can occur when the metric does not align with your specific evaluation preferences. For example,
183
+

198
184
199
-

200
-
201
-
202
-
To fix these results, ragas provides a way to align the metric with your preferences, allowing it to learn like a machine learning model. Here's how you can do this in three simple steps:
203
-
204
-
1.**Annotate**: Accept, reject, or edit evaluation results to create training data (at least 15-20 samples).
205
-
2.**Download**: Save the annotated data using the `Annotated JSON` button in [app.ragas.io](https://app.ragas.io/).
206
-
3.**Train**: Use the annotated data to train your custom metric.
207
-
208
-
To learn more about this, refer to how to [train your own metric guide](./../howtos/customizations/metrics/train_your_own_metric.md)
Once trained, you can re-evaluate the same or different test datasets. You should notice that the metric now aligns with your preferences and makes fewer mistakes, improving its accuracy.
### Want help in improving your AI application using evals?
180
180
181
-
Once you have evaluated, you may want to view, analyse and share results. This is important to interpret the results and understand the performance of your RAG system. For this you may sign up and setup [app.ragas.io]() easily. If not, you may use any alternative tools available to you.
181
+
In the past 2 years, we have seen and helped improve many AI applications using evals.
182
182
183
-
In order to use the [app.ragas.io](http://app.ragas.io) dashboard, you need to have an account on [app.ragas.io](https://app.ragas.io/). If you don't have one, you can sign up for one [here](https://app.ragas.io/login). You will also need to generate a [Ragas APP token](https://app.ragas.io/dashboard/settings/app-tokens).
183
+
We are compressing this knowledge into a product to replace vibe checks with eval loops so that you can focus on building great AI applications.
184
184
185
-
Once you have the API key, you can use the `upload()` method to export the results to the dashboard.
185
+
If you want help with improving and scaling up your AI application using evals.
186
186
187
-
```python
188
-
import os
189
-
os.environ["RAGAS_APP_TOKEN"] ="your_app_token"
190
-
```
191
187
192
-
Now you can view the results in the dashboard by following the link in the output of the `upload()` method.
Copy file name to clipboardExpand all lines: docs/getstarted/rag_testset_generation.md
+2-15
Original file line number
Diff line number
Diff line change
@@ -58,21 +58,8 @@ dataset.to_pandas()
58
58
Output
59
59

60
60
61
-
You can also use other tools like [app.ragas.io](https://app.ragas.io/) or any other similar tools available for you in the [Integrations](./../howtos/integrations/index.md) section.
62
-
63
-
In order to use the [app.ragas.io](https://app.ragas.io/) dashboard, you need to have an account on [app.ragas.io](https://app.ragas.io/). If you don't have one, you can sign up for one [here](https://app.ragas.io/login). You will also need to have a [Ragas APP token](https://app.ragas.io/settings/api-keys).
64
-
65
-
Once you have the API key, you can use the `upload()` method to export the results to the dashboard.
66
-
67
-
```python
68
-
import os
69
-
os.environ["RAGAS_APP_TOKEN"] ="your_app_token"
70
-
dataset.upload()
71
-
```
72
-
73
-
Now you can view the results in the dashboard by following the link in the output of the `upload()` method.
74
-
75
-

61
+
!!! note
62
+
Generating synthetic test data can be confusing and hard, but if you need we are happy to help you with it. We have built pipelines to generate test data for various use cases. If you need help with it, please talk to us by booking a [slot](https://bit.ly/3EBYq4J) or drop us a line: [[email protected]](mailto:[email protected]).
0 commit comments