Skip to content

Commit

Permalink
fix combine script and generate files
Browse files Browse the repository at this point in the history
  • Loading branch information
rick committed Sep 28, 2023
1 parent f0fc4eb commit 2c5742b
Show file tree
Hide file tree
Showing 12 changed files with 201 additions and 1,055 deletions.
22 changes: 14 additions & 8 deletions .github/workflows/combine_nb_to_docs_testing.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ rm -rf alltools.ipynb
# IF MOVING ANY IPYNB, MAKE SURE TO RE-SYMLINK. MANY IPYNB REFERENCED HERE LIVE IN OTHER PATHS
nbmerge langchain_quickstart.ipynb logging.ipynb custom_feedback_functions.ipynb >> all_tools.ipynb

# Colab quickstarts
# IF MOVING ANY IPYNB, MAKE SURE TO RE-SYMLINK. MANY IPYNB REFERENCED HERE LIVE IN OTHER PATHS
nbmerge colab_dependencies.ipynb langchain_quickstart.ipynb >> langchain_quickstart_colab.ipynb
nbmerge colab_dependencies.ipynb llama_index_quickstart.ipynb >> llama_index_quickstart_colab.ipynb
nbmerge colab_dependencies.ipynb text2text_quickstart.ipynb >> text2text_quickstart_colab.ipynb

# Create pypi page documentation

cat intro.md > README.md
Expand All @@ -26,18 +32,18 @@ jupyter nbconvert --to script --output-dir=./py_script_quickstarts/ all_tools.ip
SED=`which -a gsed sed | head -n1`

# Fix nbmerge ids field invalid for ipynb
$SED -i '' "/id\"\:/d" all_tools.ipynb langchain_quickstart_colab.ipynb llama_index_quickstart_colab.ipynb text2text_quickstart_colab.ipynb
$SED -i "/id\"\:/d" all_tools.ipynb langchain_quickstart_colab.ipynb llama_index_quickstart_colab.ipynb text2text_quickstart_colab.ipynb

## Remove ipynb JSON calls
$SED -i '' "/JSON/d" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
$SED -i "/JSON/d" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
## Replace jupyter display with python print
$SED -i '' "s/display/print/g" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
$SED -i "s/display/print/g" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
## Remove cell metadata
$SED -i '' "/\# In\[/d" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
$SED -i "/\# In\[/d" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
## Remove single # lines
$SED -i '' "/\#$/d" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
$SED -i "/\#$/d" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
## Collapse multiple empty line from sed replacements with a single line
$SED -i '' -e "/./b" -e ":n" -e "N;s/\\n$//;tn" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
$SED -i -e "/./b" -e ":n" -e "N;s/\\n$//;tn" ./py_script_quickstarts/langchain_quickstart.py ./py_script_quickstarts/llama_index_quickstart.py ./py_script_quickstarts/text2text_quickstart.py ./py_script_quickstarts/all_tools.py
# Move generated files to their end locations
# EVERYTHING BELOW IS LINKED TO DOCUMENTATION OR TESTS; MAKE SURE YOU UPDATE LINKS IF YOU CHANGE
# IF NAMES CHANGED; CHANGE THE LINK NAMES TOO
Expand All @@ -49,7 +55,7 @@ mv TOP_README.md ../../README.md
# Links are referenced in intro.md and gh_intro.md
# There are symlinks from ../../trulens_eval/generated_files/ to these scripts for testing
mkdir -p ../../trulens_eval/examples/quickstart/py_script_quickstarts/
mv *.py ../../trulens_eval/examples/quickstart/py_script_quickstarts/
mv ./py_script_quickstarts/*.py ../../trulens_eval/examples/quickstart/py_script_quickstarts/

# Trulens tests run off of these files
mv all_tools* ../../trulens_eval/generated_files/
mv all_tools* ../../trulens_eval/generated_files/
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,24 +43,24 @@ TruLens supports the evaluation of tracking for any LLM app framework. Choose a

**Langchain**

[langchain_quickstart.ipynb](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.13.0/trulens_eval/examples/quickstart/langchain_quickstart.ipynb).
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/truera/trulens/blob/releases/rc-trulens-eval-0.13.0/trulens_eval/examples/colab/quickstarts/langchain_quickstart_colab.ipynb)
[langchain_quickstart.ipynb](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.14.0/trulens_eval/examples/quickstart/langchain_quickstart.ipynb).
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/truera/trulens/blob/releases/rc-trulens-eval-0.14.0/trulens_eval/examples/colab/quickstarts/langchain_quickstart_colab.ipynb)

[langchain_quickstart.py](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.13.0/trulens_eval/examples/quickstart/py_script_quickstarts/langchain_quickstart.py).
[langchain_quickstart.py](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.14.0/trulens_eval/examples/quickstart/py_script_quickstarts/langchain_quickstart.py).

**Llama-Index**

[llama_index_quickstart.ipynb](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.13.0/trulens_eval/examples/quickstart/llama_index_quickstart.ipynb).
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/truera/trulens/blob/releases/rc-trulens-eval-0.13.0/trulens_eval/examples/colab/quickstarts/llama_index_quickstart_colab.ipynb)
[llama_index_quickstart.ipynb](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.14.0/trulens_eval/examples/quickstart/llama_index_quickstart.ipynb).
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/truera/trulens/blob/releases/rc-trulens-eval-0.14.0/trulens_eval/examples/colab/quickstarts/llama_index_quickstart_colab.ipynb)

[llama_index_quickstart.py](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.13.0/trulens_eval/examples/quickstart/py_script_quickstarts/llama_index_quickstart.py)
[llama_index_quickstart.py](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.14.0/trulens_eval/examples/quickstart/py_script_quickstarts/llama_index_quickstart.py)

**No Framework**

[text2text_quickstart.ipynb](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.13.0/trulens_eval/examples/quickstart/text2text_quickstart.ipynb).
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/truera/trulens/blob/releases/rc-trulens-eval-0.13.0/trulens_eval/examples/colab/quickstarts/text2text_quickstart_colab.ipynb)
[text2text_quickstart.ipynb](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.14.0/trulens_eval/examples/quickstart/text2text_quickstart.ipynb).
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/truera/trulens/blob/releases/rc-trulens-eval-0.14.0/trulens_eval/examples/colab/quickstarts/text2text_quickstart_colab.ipynb)

[text2text_quickstart.py](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.13.0/trulens_eval/examples/quickstart/py_script_quickstarts/text2text_quickstart.py)
[text2text_quickstart.py](https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.14.0/trulens_eval/examples/quickstart/py_script_quickstarts/text2text_quickstart.py)


## TruLens-Explain
Expand Down
Loading

0 comments on commit 2c5742b

Please sign in to comment.