From 34f1d96aa1d8e1cf6dc713fce06bbefd0c6d0bf7 Mon Sep 17 00:00:00 2001 From: sharmax-vandana Date: Sun, 15 Sep 2024 21:55:28 +0530 Subject: [PATCH 1/2] Update deepseed-flan-t5-summarization.ipynb fixed typos --- training/deepseed-flan-t5-summarization.ipynb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/training/deepseed-flan-t5-summarization.ipynb b/training/deepseed-flan-t5-summarization.ipynb index 8fcc537..b00efd1 100644 --- a/training/deepseed-flan-t5-summarization.ipynb +++ b/training/deepseed-flan-t5-summarization.ipynb @@ -1,4 +1,4 @@ -{ + { "cells": [ { "attachments": {}, @@ -136,7 +136,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We know now that our documents can be “500” tokens long to fit our `template_prompt` still correctly. In addition to our input, we need to understand better our “target” sequence length meaning and how long are the summarization ins our dataset. Therefore we iterate over the dataset and calculate the max input length (at max 500) and the max target length. (takes a few minutes)" + "We know now that our documents can be “500” tokens long to fit our `template_prompt` still correctly. In addition to our input, we need to understand better our “target” sequence length meaning and how long are the summarizations in our dataset. Therefore we iterate over the dataset and calculate the max input length (at max 500) and the max target length. (takes a few minutes)" ] }, { @@ -367,7 +367,7 @@ "|-------------------|------------|--------------|--------------------|-----------|----------|--------|\n", "| FLAN-T5-XL (3B) | No | 4x V100 16GB | OOM | fp32 | - | - |\n", "| FLAN-T5-XL (3B) | No | 8x V100 16GB | 1 | fp32 | 105h | ~$2570 |\n", - "| FLAN-T5-XL (3B) | No | 8x A100 40GB | 72 | bf16 | 2,5h | ~$81 |\n", + "| FLAN-T5-XL (3B) | No | 8x A100 40GB | 72 | bf16 | 2.5h | ~$81 |\n", "| FLAN-T5-XL (3B) | Yes | 4x V100 16GB | 8 | fp32 | 69h | ~$828 |\n", "| FLAN-T5-XL (3B) | Yes | 8x V100 16GB | 8 | fp32 | 32h | ~$768 |\n", "| FLAN-T5-XXL (11B) | No | 8x A100 40GB | 8 | bf16 | 10h | ~$322 |\n", From cb8f5acad4a539feecacec37347771161d5743a6 Mon Sep 17 00:00:00 2001 From: sharmax-vandana Date: Sun, 15 Sep 2024 21:57:19 +0530 Subject: [PATCH 2/2] Update deepseed-flan-t5-summarization.ipynb fixed typos --- training/deepseed-flan-t5-summarization.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/training/deepseed-flan-t5-summarization.ipynb b/training/deepseed-flan-t5-summarization.ipynb index b00efd1..e5a98a2 100644 --- a/training/deepseed-flan-t5-summarization.ipynb +++ b/training/deepseed-flan-t5-summarization.ipynb @@ -1,4 +1,4 @@ - { +{ "cells": [ { "attachments": {},