Merge pull request #1694 from XiangJinyu/main

Modify some files, about AFlow and SPO

README.md

@@ -27,6 +27,8 @@
 </p>
 
 ## News
+🚀 Jan. 22, 2025: Our paper [AFlow: Automating Agentic Workflow Generation](https://openreview.net/forum?id=z5uVAKwmjf) accepted for **oral presentation (top 1.8%)** at ICLR 2025, **ranking #2** in the LLM-based Agent category.
+
 🚀 Oct. 29, 2024: We introduced three papers: [AFLOW](https://arxiv.org/abs/2410.10762), [FACT](https://arxiv.org/abs/2410.21012), and [SELA](https://arxiv.org/abs/2410.17238), check the [code](examples)!
 
 🚀 Mar. 29, 2024: [v0.8.0](https://github.com/geekan/MetaGPT/releases/tag/v0.8.0) released. Now you can use Data Interpreter ([arxiv](https://arxiv.org/abs/2402.18679), [example](https://docs.deepwisdom.ai/main/en/DataInterpreter/), [code](https://github.com/geekan/MetaGPT/tree/main/examples/di)) via pypi package import. Meanwhile, we integrated the RAG module and supported multiple new LLMs.

examples/spo/README.md

@@ -2,6 +2,7 @@
 
 [![Paper](https://img.shields.io/badge/Paper-arXiv-red)](https://arxiv.org/pdf/2502.06855)
 [![Demo](https://img.shields.io/badge/Demo-Hugging%20Face-yellow)](https://huggingface.co/spaces/XiangJinYu/SPO)
+[![ModelScope](https://img.shields.io/badge/Demo-ModelScope-blue)](https://modelscope.cn/studios/AI-ModelScope/SPO)
 
 An automated prompt engineering tool for Large Language Models (LLMs), designed for universal domain adaptation.
 
@@ -21,7 +22,9 @@ A next-generation prompt engineering system implementing **Self-Supervised Promp
 ## 🔗 Quick Links
 
 - [📝 Read our paper](https://arxiv.org/pdf/2502.06855)
-- [🤗 Try our demo](https://huggingface.co/spaces/XiangJinYu/SPO)
+- [🤗 Try our Hugging Face demo](https://huggingface.co/spaces/XiangJinYu/SPO)
+- [🔮 Try our ModelScope demo](https://modelscope.cn/studios/AI-ModelScope/SPO)
+
 
 ## 📊 Experiment
 

metagpt/ext/aflow/scripts/prompts/optimize_prompt.py

@@ -17,7 +17,7 @@
 Here is a graph and the corresponding prompt (prompt only related to the custom method) that performed excellently in a previous iteration (maximum score is 1). You must make further optimizations and improvements based on this graph. The modified graph must differ from the provided example, and the specific differences should be noted within the <modification>xxx</modification> section.\n
 <sample>
     <experience>{experience}</experience>
-    <modification>(such as:add a review step/delete a operator/modify a prompt)</modification>
+    <modification>(such as:add /delete /modify/ ...)</modification>
     <score>{score}</score>
     <graph>{graph}</graph>
     <prompt>{prompt}</prompt>(only prompt_custom)

metagpt/ext/spo/utils/evaluation_utils.py

@@ -1,3 +1,4 @@
+import asyncio
 from pathlib import Path
 from typing import Any, List, Optional, Tuple
 
@@ -49,9 +50,15 @@ async def evaluate_prompt(
             succeed = True
         else:
             evaluation_results = []
-            for _ in range(EVALUATION_REPETITION):
-                result = await evaluator.prompt_evaluate(samples=samples, new_samples=new_samples)
-                evaluation_results.append(result)
+
+            evaluation_results.extend(
+                await asyncio.gather(
+                    *(
+                        evaluator.prompt_evaluate(samples=samples, new_samples=new_samples)
+                        for _ in range(EVALUATION_REPETITION)
+                    )
+                )
+            )
 
             logger.info(f"Evaluation Results {evaluation_results}")