awesome_llm_agents.md

Awesome llm agents

• Awesome llm agents
  • Survey
  • LLM OS
  • Agents
    • Other
  • AutoGPT
    • Other
  • Augmented LLM
    • Other
  • Web browsing
    • Other
  • Retrieval agumented generation
    • Other
  • Code Interpreter
  • GPTs
    • Plugins
    • Other
  • Evaluation
  • Other
  • Extra reference

Survey

• Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security, arXiv, 2401.05459, arxiv, pdf, cication: -1

  Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun · (Personal_LLM_Agents_Survey - MobileLLM)

• Retrieval-Augmented Generation for Large Language Models: A Survey, arXiv, 2312.10997, arxiv, pdf, cication: -1

  Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Haofen Wang · (rag-survey - tongji-kgllm)

• Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives, arXiv, 2312.11970, arxiv, pdf, cication: -1

  Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, Yong Li · (mp.weixin.qq)

• LLM Powered Autonomous Agents | Lil'Log

  · (mp.weixin.qq)

LLM OS

• At the Intersection of LLMs and Kernels - Research Roundup

• llama2.c - trholding

  Llama 2 Everywhere (L2E) · (jiqizhixin)

• MemGPT - cpacker

  Teaching LLMs memory management for unbounded context 📚🦙

  · (jiqizhixin)

Agents

• Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception, arXiv, 2401.16158, arxiv, pdf, cication: -1

  Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, Jitao Sang · (MobileAgent - X-PLUG)

• SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents, arXiv, 2401.10935, arxiv, pdf, cication: -1

  Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, Zhiyong Wu · (SeeClick - njucckevin)

• Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-Evolution, arXiv, 2401.13996, arxiv, pdf, cication: -1

  Cheng Qian, Shihao Liang, Yujia Qin, Yining Ye, Xin Cong, Yankai Lin, Yesai Wu, Zhiyuan Liu, Maosong Sun

• ChatQA: Building GPT-4 Level Conversational QA Models, arXiv, 2401.10225, arxiv, pdf, cication: -1

  Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Mohammad Shoeybi, Bryan Catanzaro

• Tool-LMM: A Large Multi-Modal Model for Tool Agent Learning, arXiv, 2401.10727, arxiv, pdf, cication: -1

  Chenyu Wang, Weixin Luo, Qianyu Chen, Haonan Mai, Jindi Guo, Sixun Dong, Xiaohua, Xuan, Zhengxin Li, Lin Ma · (Tool-LMM?tab=readme-ov-file - Tool-LMM)

• Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk, arXiv, 2401.05033, arxiv, pdf, cication: -1

  Dennis Ulmer, Elman Mansimov, Kaixiang Lin, Justin Sun, Xibin Gao, Yi Zhang

• GitAgent: Facilitating Autonomous Agent with GitHub by Tool Extension, arXiv, 2312.17294, arxiv, pdf, cication: -1

  Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Yujia Qin, Yining Ye, Yaxi Lu, Zhong Zhang, Yukun Yan, Yankai Lin

• Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning, arXiv, 2312.14878, arxiv, pdf, cication: -1

  Filippos Christianos, Georgios Papoudakis, Matthieu Zimmer, Thomas Coste, Zhihao Wu, Jingxuan Chen, Khyati Khandelwal, James Doran, Xidong Feng, Jiacheng Liu

• AppAgent: Multimodal Agents as Smartphone Users, arXiv, 2312.13771, arxiv, pdf, cication: -1

  Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, Gang Yu

  · (AppAgent - mnotgod96)

• KwaiAgents: Generalized Information-seeking Agent System with Large Language Models, arXiv, 2312.04889, arxiv, pdf, cication: -1

  Haojie Pan, Zepeng Zhai, Hao Yuan, Yaojia Lv, Ruiji Fu, Ming Liu, Zhongyuan Wang, Bing Qin · (kwaiagents - kwaikeg)

• CogAgent: A Visual Language Model for GUI Agents, arXiv, 2312.08914, arxiv, pdf, cication: -1

  Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding

  · (CogVLM - THUDM)

• Creative Agents: Empowering Agents with Imagination for Creative Tasks, arXiv, 2312.02519, arxiv, pdf, cication: -1

  Chi Zhang, Penglin Cai, Yuhui Fu, Haoqi Yuan, Zongqing Lu

  · (Creative-Agents - PKU-RL) · (mp.weixin.qq)

• An LLM Compiler for Parallel Function Calling, arXiv, 2312.04511, arxiv, pdf, cication: -1

  Sehoon Kim, Suhong Moon, Ryan Tabrizi, Nicholas Lee, Michael W. Mahoney, Kurt Keutzer, Amir Gholami · (llmcompiler - squeezeailab)

• Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses, arXiv, 2312.00763, arxiv, pdf, cication: -1

  Xiao Ma, Swaroop Mishra, Ariel Liu, Sophie Su, Jilin Chen, Chinmay Kulkarni, Heng-Tze Cheng, Quoc Le, Ed Chi

• taskweaver - microsoft

  A code-first agent framework for seamlessly planning and executing data analytics tasks.

  · (jiqizhixin)

• Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents, arXiv, 2311.11797, arxiv, pdf, cication: -1

  Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, Zhiwei He, Yiming Wang, Mark Gerstein, Rui Wang, Gongshen Liu · (CoT-Igniting-Agent - Zoeyyao27)

• ToolTalk: Evaluating Tool-Usage in a Conversational Setting, arXiv, 2311.10775, arxiv, pdf, cication: -1

  Nicholas Farn, Richard Shin · (ToolTalk - microsoft)

• TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems, arXiv, 2311.11315, arxiv, pdf, cication: -1

  Yilun Kong, Jingqing Ruan, Yihong Chen, Bin Zhang, Tianpeng Bao, Shiwei Shi, Guoqing Du, Xiaoru Hu, Hangyu Mao, Ziyue Li

• multi-agent-postgres-data-analytics - disler

  The way we interact with our data is changing.

• ProAgent - OpenBMB

  · (ProAgent - OpenBMB)

• JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models, arXiv, 2311.05997, arxiv, pdf, cication: -1

  Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang · (craftjarvis-jarvis1.github)

• Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs, arXiv, 2311.05657, arxiv, pdf, cication: -1

  Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, Bill Yuchen Lin · (lumos - allenai) · (allenai.github)

• OpenAI_Agent_Swarm - daveshap

  HAAS = Hierarchical Autonomous Agent Swarm - "Resistance is futile!"

• LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents, arXiv, 2311.05437, arxiv, pdf, cication: -1

  Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu

• Octopus: Embodied Vision-Language Programmer from Environmental Feedback, arXiv, 2310.08588, arxiv, pdf, cication: -1

  Jingkang Yang, Yuhao Dong, Shuai Liu, Bo Li, Ziyue Wang, Chencheng Jiang, Haoran Tan, Jiamu Kang, Yuanhan Zhang, Kaiyang Zhou · (Octopus - dongyh20) · (mp.weixin.qq)

• War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars, arXiv, 2311.17227, arxiv, pdf, cication: -1

  Wenyue Hua, Lizhou Fan, Lingyao Li, Kai Mei, Jianchao Ji, Yingqiang Ge, Libby Hemphill, Yongfeng Zhang · (mp.weixin.qq)

• Neural MMO 2.0: A Massively Multi-task Addition to Massively Multi-agent Learning, arXiv, 2311.03736, arxiv, pdf, cication: -1

  Joseph Suárez, Phillip Isola, Kyoung Whan Choe, David Bloomin, Hao Xiang Li, Nikhil Pinnaparaju, Nishaanth Kanna, Daniel Scott, Ryan Sullivan, Rose S. Shuman

• From Copilot to CoOrchestration

• OpenAgents: An Open Platform for Language Agents in the Wild, arXiv, 2310.10634, arxiv, pdf, cication: -1

  Tianbao Xie, Fan Zhou, Zhoujun Cheng, Peng Shi, Luoxuan Weng, Yitao Liu, Toh Jing Hua, Junning Zhao, Qian Liu, Che Liu

• agenttuning - thudm

  AgentTuning: Enabling Generalized Agent Abilities for LLMs

• Humanoid Agents: Platform for Simulating Human-like Generative Agents, arXiv, 2310.05418, arxiv, pdf, cication: 1

  Zhilin Wang, Yu Ying Chiu, Yu Cheung Chiu · (humanoidagents - humanoidagents)

• XAgent - OpenBMB

  An Autonomous LLM Agent for Complex Task Solving · (jiqizhixin)

• Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency, arXiv, 2309.17382, arxiv, pdf, cication: -1

  Zhihan Liu, Hao Hu, Shenao Zhang, Hongyi Guo, Shuqi Ke, Boyi Liu, Zhaoran Wang

• Humanoid Agents: Platform for Simulating Human-like Generative Agents, arXiv, 2310.05418, arxiv, pdf, cication: 1

  Zhilin Wang, Yu Ying Chiu, Yu Cheung Chiu · (mp.weixin.qq)

• A Zero-Shot Language Agent for Computer Control with Structured Reflection, arXiv, 2310.08740, arxiv, pdf, cication: -1

  Tao Li, Gang Li, Zhiwei Deng, Bryan Wang, Yang Li

• Lemur: Harmonizing Natural Language and Code for Language Agents, arXiv, 2310.06830, arxiv, pdf, cication: 1

  Yiheng Xu, Hongjin Su, Chen Xing, Boyu Mi, Qian Liu, Weijia Shi, Binyuan Hui, Fan Zhou, Yitao Liu, Tianbao Xie

• EcoAssistant: Using LLM Assistant More Affordably and Accurately, arXiv, 2310.03046, arxiv, pdf, cication: -1

  Jieyu Zhang, Ranjay Krishna, Ahmed H. Awadallah, Chi Wang

• khoj - khoj-ai

  An AI personal assistant for your digital brain

• AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn, arXiv, 2306.08640, arxiv, pdf, cication: -1

  Difei Gao, Lei Ji, Luowei Zhou, Kevin Qinghong Lin, Joya Chen, Zihan Fan, Mike Zheng Shou

• Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4, arXiv, 2309.17277, arxiv, pdf, cication: -1

  Jiaxian Guo, Bo Yang, Paul Yoo, Bill Yuchen Lin, Yusuke Iwasawa, Yutaka Matsuo

• autogen - microsoft

  Enable Next-Gen Large Language Model Applications. Join our Discord: https://discord.gg/pAbnFJrkgZ

• How FaR Are Large Language Models From Agents with Theory-of-Mind?, arXiv, 2310.03051, arxiv, pdf, cication: 2

  Pei Zhou, Aman Madaan, Srividya Pranavi Potharaju, Aditya Gupta, Kevin R. McKee, Ari Holtzman, Jay Pujara, Xiang Ren, Swaroop Mishra, Aida Nematzadeh · (qbitai)

• AutoAgents - LinkSoul-AI

  Generate different roles for GPTs to form a collaborative entity for complex tasks.

• LASER: LLM Agent with State-Space Exploration for Web Navigation, arXiv, 2309.08172, arxiv, pdf, cication: -1

  Kaixin Ma, Hongming Zhang, Hongwei Wang, Xiaoman Pan, Dong Yu

• Agents: An Open-source Framework for Autonomous Language Agents, arXiv, 2309.07870, arxiv, pdf, cication: 4

  Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang · (agents - aiwaves-cn)

• MindAgent: Emergent Gaming Interaction - Microsoft Research

  · (qbitai)

• The Rise and Potential of Large Language Model Based Agents: A Survey, arXiv, 2309.07864, arxiv, pdf, cication: 23

  Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou · (jiqizhixin) · (LLM-Agent-Paper-List - WooooDyy)

• Cognitive Architectures for Language Agents, arXiv, 2309.02427, arxiv, pdf, cication: 11

  Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, Thomas L. Griffiths · (awesome-language-agents - ysymyth)

• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors, arXiv, 2308.10848, arxiv, pdf, cication: 13

  Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian · (agentverse - openbmb)

• AI-town - a16z-infra

  A MIT-licensed, deployable starter kit for building and customizing your own version of AI town - a virtual town where AI characters live, chat and socialize.

• TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage, arXiv, 2308.03427, arxiv, pdf, cication: 11

  Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Ziyue Li, Xingyu Zeng

• SHOW-1 and Showrunner Agents in Multi-Agent Simulations

  · (fablestudio.github) · (mp.weixin.qq)

• Building Cooperative Embodied Agents Modularly with Large Language Models, arXiv, 2307.02485, arxiv, pdf, cication: -1

  Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B. Tenenbaum, Tianmin Shu, Chuang Gan

• autotab-starter - Planetary-Computers

  Build browser agents for real world tasks

• openagents - xlang-ai

  OpenAgents: An Open Platform for Language Agents in the Wild

• octopus - dongyh20

  🐙Octopus, an embodied vision-language model trained with RLEF, emerging superior in embodied visual planning and programming.

• gollie - hitz-zentroa

  Guideline following Large Language Model for Information Extraction

• NexusRaven-13B: Surpassing the state-of-the-art in open-source function calling LLMs.

  · (nexusflow)

• ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models, arXiv, 2309.00986, arxiv, pdf, cication: 2

  Chenliang Li, Hehong Chen, Ming Yan, Weizhou Shen, Haiyang Xu, Zhikai Wu, Zhicheng Zhang, Wenmeng Zhou, Yingda Chen, Chen Cheng · (modelscope-agent - modelscope)

• trl-text-environment - trl-lib 🤗

• awesome-ai-devtools - jamesmurdza

  Curated list of AI-powered developer tools.

• TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage, arXiv, 2308.03427, arxiv, pdf, cication: 11

  Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Ziyue Li, Xingyu Zeng

• functionary - musabgultekin

  Chat language model that can interpret and execute functions/plugins

• Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models, arXiv, 2308.00675, arxiv, pdf, cication: 4

  Cheng-Yu Hsieh, Si-An Chen, Chun-Liang Li, Yasuhisa Fujii, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, Tomas Pfister

• gorilla - ShishirPatil

  Gorilla: An API store for LLMs

• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs, arXiv, 2307.16789, arxiv, pdf, cication: 33

  Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian · (ToolBench - OpenBMB)

• Android in the Wild: A Large-Scale Dataset for Android Device Control, arXiv, 2307.10088, arxiv, pdf, cication: 4

  Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, Timothy Lillicrap · (google-research - google-research)

• amadeusgpt - adaptivemotorcontrollab

  We turn natural language descriptions of behaviors into machine-executable code

• Towards Language Models That Can See: Computer Vision Through the LENS of Natural Language, arXiv, 2306.16410, arxiv, pdf, cication: -1

  William Berrios, Gautam Mittal, Tristan Thrush, Douwe Kiela, Amanpreet Singh · (lens - contextualai)

• ViperGPT: Visual Inference via Python Execution for Reasoning, arXiv, 2303.08128, arxiv, pdf, cication: 76

  Dídac Surís, Sachit Menon, Carl Vondrick

• HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face, arXiv, 2303.17580, arxiv, pdf, cication: 233

  Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, Yueting Zhuang

• LOVM: Language-Only Vision Model Selection, arXiv, 2306.08893, arxiv, pdf, cication: -1

  Orr Zohar, Shih-Cheng Huang, Kuan-Chieh Wang, Serena Yeung

• CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models, arXiv, 2305.14318, arxiv, pdf, cication: 7

  Cheng Qian, Chi Han, Yi R. Fung, Yujia Qin, Zhiyuan Liu, Heng Ji · (jiqizhixin)

• gorilla - ShishirPatil

  Gorilla: An API store for LLMs · (jiqizhixin) · (mp.weixin.qq)

• ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models, arXiv, 2305.18323, arxiv, pdf, cication: 10

  Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, Dongkuan Xu · (rewoo - billxbf)

• OlaGPT: Empowering LLMs With Human-like Problem-Solving Abilities, arXiv, 2305.16334, arxiv, pdf, cication: 2

  Yuanzhen Xie, Tao Xie, Mingxiong Lin, WenTao Wei, Chenglin Li, Beibei Kong, Lei Chen, Chengxiang Zhuo, Bo Hu, Zang Li · (mp.weixin.qq)

• Natural Language Commanding via Program Synthesis, arXiv, 2306.03460, arxiv, pdf, cication: 1

  Apurva Gandhi, Thong Q. Nguyen, Huitian Jiao, Robert Steen, Ameya Bhatawdekar

• Think Before You Act: Decision Transformers with Internal Working Memory, arXiv, 2305.16338, arxiv, pdf, cication: -1

  Jikun Kang, Romain Laroche, Xindi Yuan, Adam Trischler, Xue Liu, Jie Fu · (qbitai)

• Visual Programming: Compositional visual reasoning without training, arXiv, 2211.11559, arxiv, pdf, cication: -1

  Tanmay Gupta, Aniruddha Kembhavi

Other

• Open-source LLMs as LangChain Agents

• 从第一性原理看大模型Agent技术

• AI最大赛道Agent机遇全解析

• Chat 向左，Agent 向右 - 知乎

• 功能超全的AI Agents开源库来了，能写小说，还能当导购、销售 | 机器之心

• AI革新之路：14篇AI Agents论文，探讨人工智能未来

• 数字身份智能体的基本原理及应用前景展望

AutoGPT

• crewAI - joaomdmoura

  Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

• self-operating-computer - OthersideAI

• open-interpreter - KillianLucas

  OpenAI's Code Interpreter in your terminal, running locally.

• ChatDev - OpenBMB

  Create Customized Software using Natural Language Idea (through Multi-Agent Collaboration)

• gpt-researcher - assafelovic

  GPT based autonomous agent that does online comprehensive research on any given topic

• gpt-llm-trainer - mshumer

  · (qbitai)

• MetaGPT - geekan

  The Multi-Agent Meta Programming Framework: Given one line Requirement, return PRD, Design, Tasks, Repo | 多智能体元编程框架：给定老板需求，输出产品文档、架构设计、任务列表、代码

• Toward Actionable Generative AI

• PromptAppGPT - mleoking

  A rapid prompt app development framework based on GPT · (mp.weixin.qq)

• Responsible Task Automation: Empowering Large Language Models as Responsible Task Automators, arXiv, 2306.01242, arxiv, pdf, cication: 2

  Zhizheng Zhang, Xiaoyi Zhang, Wenxuan Xie, Yan Lu · (jiqizhixin)

• Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions, arXiv, 2306.02224, arxiv, pdf, cication: 10

  Hui Yang, Sifu Yue, Yunzhong He · (mp.weixin.qq)

• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society, arXiv, 2303.17760, arxiv, pdf, cication: -1

  Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, Bernard Ghanem

• Language Models can Solve Computer Tasks, arXiv, 2303.17491, arxiv, pdf, cication: 50

  Geunwoo Kim, Pierre Baldi, Stephen McAleer

• SuperAGI - TransformerOptimus

  <⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

• babyagi - yoheinakajima

• Re3: Generating Longer Stories With Recursive Reprompting and Revision, arXiv, 2210.06774, arxiv, pdf, cication: 55

  Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein

• Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents, ICML, 2022, arxiv, pdf, cication: 341

  Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch · (huangwl18.github)

Other

• Godmode.space

  · (mp.weixin.qq) · (cognosys) · (doanythingmachine)

• AgentGPT

Augmented LLM

• Efficient Tool Use with Chain-of-Abstraction Reasoning, arXiv, 2401.17464, arxiv, pdf, cication: -1

  Silin Gao, Jane Dwivedi-Yu, Ping Yu, Xiaoqing Ellen Tan, Ramakanth Pasunuru, Olga Golovneva, Koustuv Sinha, Asli Celikyilmaz, Antoine Bosselut, Tianlu Wang

• LLM Augmented LLMs: Expanding Capabilities through Composition, arXiv, 2401.02412, arxiv, pdf, cication: -1

  Rachit Bansal, Bidisha Samanta, Siddharth Dalmia, Nitish Gupta, Shikhar Vashishth, Sriram Ganapathy, Abhishek Bapna, Prateek Jain, Partha Talukdar

• ProTIP: Progressive Tool Retrieval Improves Planning, arXiv, 2312.10332, arxiv, pdf, cication: -1

  Raviteja Anantha, Bortik Bandyopadhyay, Anirudh Kashi, Sayantan Mahinder, Andrew W Hill, Srinivas Chappidi

• Memory Augmented Language Models through Mixture of Word Experts, arXiv, 2311.10768, arxiv, pdf, cication: -1

  Cicero Nogueira dos Santos, James Lee-Thorp, Isaac Noble, Chung-Ching Chang, David Uthus

• ControlLLM: Augment Language Models with Tools by Searching on Graphs, arXiv, 2310.17796, arxiv, pdf, cication: -1

  Zhaoyang Liu, Zeqiang Lai, Zhangwei Gao, Erfei Cui, Zhiheng Li, Xizhou Zhu, Lewei Lu, Qifeng Chen, Yu Qiao, Jifeng Dai

• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language, arXiv, 2204.00598, arxiv, pdf, cication: 202

  Andy Zeng, Maria Attarian, Brian Ichter, Krzysztof Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani · (socraticmodels.github)

• Understanding Retrieval Augmentation for Long-Form Question Answering, arXiv, 2310.12150, arxiv, pdf, cication: 1

  Hung-Ting Chen, Fangyuan Xu, Shane Arora, Eunsol Choi

• Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model, arXiv, 2310.09520, arxiv, pdf, cication: 1

  Haikang Deng, Colin Raffel

• RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation, arXiv, 2310.04408, arxiv, pdf, cication: -1

  Fangyuan Xu, Weijia Shi, Eunsol Choi

• InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining, arXiv, 2310.07713, arxiv, pdf, cication: -1

  Boxin Wang, Wei Ping, Lawrence McAfee, Peng Xu, Bo Li, Mohammad Shoeybi, Bryan Catanzaro

• RA-DIT: Retrieval-Augmented Dual Instruction Tuning, arXiv, 2310.01352, arxiv, pdf, cication: -1

  Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Rich James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis

• Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization, arXiv, 2308.02151, arxiv, pdf, cication: 6

  Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh Murthy, Zeyuan Chen, Jianguo Zhang, Devansh Arpit

• Meta-training with Demonstration Retrieval for Efficient Few-shot Learning, arXiv, 2307.00119, arxiv, pdf, cication: -1

  Aaron Mueller, Kanika Narang, Lambert Mathias, Qifan Wang, Hamed Firooz

• AVIS: Autonomous Visual Information Seeking with Large Language Model Agent, arXiv, 2306.08129, arxiv, pdf, cication: -1

  Ziniu Hu, Ahmet Iscen, Chen Sun, Kai-Wei Chang, Yizhou Sun, David A Ross, Cordelia Schmid, Alireza Fathi · (mp.weixin.qq)

• Modular Visual Question Answering via Code Generation, arXiv, 2306.05392, arxiv, pdf, cication: 1

  Sanjay Subramanian, Medhini Narasimhan, Kushal Khangaonkar, Kevin Yang, Arsha Nagrani, Cordelia Schmid, Andy Zeng, Trevor Darrell, Dan Klein

• Reimagining Retrieval Augmented Language Models for Answering Queries, arXiv, 2306.01061, arxiv, pdf, cication: -1

  Wang-Chiew Tan, Yuliang Li, Pedro Rodriguez, Richard James, Xi Victoria Lin, Alon Halevy, Scott Yih

Other

• 陈丹琦ACL学术报告来了！详解大模型「外挂」数据库7大方向3大挑战，3小时干货满满 | 量子位

Web browsing

• search_with_lepton - leptonai

  Building a quick conversation-based search demo with Lepton AI.

• WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models, arXiv, 2401.13919, arxiv, pdf, cication: -1

  Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, Dong Yu

• GPT-4V(ision) is a Generalist Web Agent, if Grounded, arXiv, 2401.01614, arxiv, pdf, cication: -1

  Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, Yu Su · (SeeAct - OSU-NLP-Group) · (osu-nlp-group.github)

• webglm - thudm

  WebGLM: An Efficient Web-enhanced Question Answering System (KDD 2023)

• FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation, arXiv, 2310.03214, arxiv, pdf, cication: 2

  Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le · (jiqizhixin)

• GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation, arXiv, 2311.07562, arxiv, pdf, cication: -1

  An Yan, Zhengyuan Yang, Wanrong Zhu, Kevin Lin, Linjie Li, Jianfeng Wang, Jianwei Yang, Yiwu Zhong, Julian McAuley, Jianfeng Gao · (MM-Navigator - zzxslp)

  · (qbitai)

• vimGPT - ishan0102

  Browse the web with GPT-4V and Vimium

• A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis, arXiv, 2307.12856, arxiv, pdf, cication: 13

  Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, Aleksandra Faust

• WebGLM - THUDM

  WebGLM: An Efficient Web-enhanced Question Answering System (KDD 2023)

• WebArena: A Realistic Web Environment for Building Autonomous Agents

  · (twitter)

• Query2doc: Query Expansion with Large Language Models, arXiv, 2303.07678, arxiv, pdf, cication: 23

  Liang Wang, Nan Yang, Furu Wei · (mp.weixin.qq)

Other

• GPT-4V学会用键鼠上网，人类眼睁睁看着它发帖玩游戏 | 量子位

Retrieval agumented generation

• RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval, arXiv, 2401.18059, arxiv, pdf, cication: -1

  Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, Christopher D. Manning

• Corrective Retrieval Augmented Generation, arXiv, 2401.15884, arxiv, pdf, cication: -1

  Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, Zhen-Hua Ling

• flagembedding - flagopen

  Dense Retrieval and Retrieval-augmented LLMs

• autollm - safevideo

  Ship RAG based LLM web apps in seconds.

• The Power of Noise: Redefining Retrieval for RAG Systems, arXiv, 2401.14887, arxiv, pdf, cication: -1

  Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, Fabrizio Silvestri

• RAGatouille - bclavie

• simple-rag - lamini-ai

• pdftochat - Nutlope

  Chat with your PDFs with AI · (pdftochat)

• RAGxplorer - gabrielchua

  Visualise and explore your RAG documents

• RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture, arXiv, 2401.08406, arxiv, pdf, cication: -1

  Aman Gupta, Anup Shirgaonkar, Angels de Luis Balaguer, Bruno Silva, Daniel Holstein, Dawei Li, Jennifer Marsman, Leonardo O. Nunes, Mahsa Rouzbahman, Morris Sharp

• Improving Text Embeddings with Large Language Models, arXiv, 2401.00368, arxiv, pdf, cication: -1

  Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei

• QAnything - netease-youdao

  Question and Answer based on Anything.

• embedchain - embedchain

  The Open Source RAG framework

• Retrieval-Augmented Generation for Large Language Models: A Survey, arXiv, 2312.10997, arxiv, pdf, cication: -1

  Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Haofen Wang · (rag-survey - tongji-kgllm)

  · (mp.weixin.qq)

• CodeFuse-DevOps-Model - codefuse-ai

  DevOps-Models is a series of industrial-first LLMs for theDevOps domain. Asking it for any question in the DevOps domain to get solution!

• codefuse-chatbot - codefuse-ai

  An open-sourced AI assistant/agents for the full-life cycle of AI native software developing, supporting chat interactions plus knowledge base, invoking tools, sandbox execution, etc. · (qbitai)

• Context Tuning for Retrieval Augmented Generation, arXiv, 2312.05708, arxiv, pdf, cication: -1

  Raviteja Anantha, Tharun Bethi, Danil Vodianik, Srinivas Chappidi

• TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents, arXiv, 2312.01279, arxiv, pdf, cication: -1

  James Enouen, Hootan Nakhost, Sayna Ebrahimi, Sercan O Arik, Yan Liu, Tomas Pfister

• LongContext_vs_RAG_NeedleInAHaystack - A-Roucher

  Comparing retrieval abilities from GPT4-Turbo and a RAG system on a toy example for various context lengths

• Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models, arXiv, 2311.09210, arxiv, pdf, cication: -1

  Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, Dong Yu

• Learning to Filter Context for Retrieval-Augmented Generation, arXiv, 2311.08377, arxiv, pdf, cication: -1

  Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md Rizwan Parvez, Graham Neubig · (filco - zorazrw)

• gpt-crawler - BuilderIO

  Crawl a site to generate knowledge files to create your own custom GPT from a URL

• Langchain-Chatchat - chatchat-space

  Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM 等语言模型的本地知识库问答 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM) QA app with langchain

• privateGPT - imartinez

  Interact with your documents using the power of GPT, 100% privately, no data leaks

• KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval, arXiv, 2310.15511, arxiv, pdf, cication: -1

  Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Ghosh Peshawaria, Ranjita Naik, Besmira Nushi

• Langchain-Chatchat - chatchat-space

  Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM 等语言模型的本地知识库问答 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM) QA app with langchain

• DocsGPT - arc53

  GPT-powered chat for documentation, chat with your documents · (qbitai)

• LMDX: Language Model-based Document Information Extraction and Localization, arXiv, 2309.10952, arxiv, pdf, cication: -1

  Vincent Perot, Kai Kang, Florian Luisier, Guolong Su, Xiaoyu Sun, Ramya Sree Boppana, Zilong Wang, Jiaqi Mu, Hao Zhang, Nan Hua

• PDFTriage: Question Answering over Long, Structured Documents, arXiv, 2309.08872, arxiv, pdf, cication: 3

  Jon Saad-Falcon, Joe Barrow, Alexa Siu, Ani Nenkova, David Seunghyun Yoon, Ryan A. Rossi, Franck Dernoncourt

• sec-insights - run-llama

  A real world full-stack application using LlamaIndex

• simplyretrieve - rcgai

  An Easy-to-use Private and Lightweight Retrieval-Centric Generative AI Tool. Create chat tool with your documents and open-source LLMs, highly customizable.

• FastGPT - labring

  A platform that uses the OpenAI API to quickly build an AI knowledge base, supporting many-to-many relationships.

• factool - gair-nlp

  A fact-checking tool that detects factual errors.

• Llama-2-Open-Source-LLM-CPU-Inference - kennethleungty

  Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

• danswer - danswer-ai

  Ask Questions in natural language and get Answers backed by private sources. Connects to tools like Slack, GitHub, Confluence, etc.

• quivr - StanGirard

  🧠 Dump all your files and thoughts into your private GenerativeAI Second Brain and chat with it 🧠

• chatgpt-retrieval - techleadhd

• localGPT - PromtEngineer

  Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.

• privateGPT - imartinez

  Interact privately with your documents using the power of GPT, 100% privately, no data leaks

Embedding

• contrastors - nomic-ai

  Train Models Contrastively in Pytorch · (huggingface) · (mp.weixin.qq)

Other

• A Cheat Sheet and Some Recipes For Building Advanced RAG | by Andrei | Jan, 2024 | LlamaIndex Blog

• Build a search engine, not a vector DB

• 大模型RAG的迭代路径

• 大模型RAG问答技术架构及核心模块回顾

• 【大模型外挂知识库(RAG)优化】如何炼成强大的向量化召回模型 - 知乎

• RAG调优方案

• RAG+GPT-4 Turbo让模型性能飙升！更长上下文不是终局，「大海捞针」实验成本仅4%

• 问答场景常用大模型解决方案

Code Interpreter

• open-interpreter - KillianLucas

  OpenAI's Code Interpreter in your terminal, running locally

GPTs

• GPTs - linexjlin

  leaked prompts of GPTs

• rags - run-llama

• GPT-Baker - abidlabs 🤗

• gpts-works - all-in-aigc

  A Third-party GPTs store

• gpt-crawler - BuilderIO

  Crawl a site to generate knowledge files to create your own custom GPT from a URL

• Awesome-GPTs - ai-boost

  Curated list of awesome GPTs 👍.

• Awesome-GPT-Agents - fr0gger

  A curated list of GPT agents for cybersecurity

• Awesome-GPT-Store - Anil-matcha

  A collection of major GPTS available in public

• awesome-gpts - taranjeet

  Collection of all the GPTs created by the community

• opengpts - langchain-ai

Plugins

• GPT-4调用插件40次都没成功，果断放弃，无效调用、拒绝回答时有发生 | 机器之心k

Other

• Featured GPTs | Best Curated Custom GPTs List for your Daily Tasks
• Discover the Best GPTs
• AI of the day by SamurAI
• 各路大神献出自定义GPT，24小时Top 9名单在这 | 机器之心

Evaluation

• AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents, arXiv, 2401.13178, arxiv, pdf, cication: -1

  Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, Junxian He · (AgentBoard - hkust-nlp)

• codefuse-devops-eval - codefuse-ai

  Industrial-first evaluation benchmark for LLMs in the DevOps/AIOps domain.

• GAIA: a benchmark for General AI Assistants, arXiv, 2311.12983, arxiv, pdf, cication: -1

  Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, Thomas Scialom · (huggingface)

• Testing Language Model Agents Safely in the Wild, arXiv, 2311.10538, arxiv, pdf, cication: -1

  Silen Naihin, David Atkinson, Marc Green, Merwane Hamadi, Craig Swift, Douglas Schonholtz, Adam Tauman Kalai, David Bau

• BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents, arXiv, 2308.05960, arxiv, pdf, cication: 7

  Zhiwei Liu, Weiran Yao, Jianguo Zhang, Le Xue, Shelby Heinecke, Rithesh Murthy, Yihao Feng, Zeyuan Chen, Juan Carlos Niebles, Devansh Arpit · (BOLAA - salesforce)

• mlagentbench - snap-stanford

• smartplay - microsoft

  SmartPlay is a benchmark for Large Language Models (LLMs). It is designed to be easy to use, and to provide a wide variety of games to test agents on.

• AgentBench: Evaluating LLMs as Agents, arXiv, 2308.03688, arxiv, pdf, cication: 9

  Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang

Other

• Nexus_Function_Calling_Leaderboard - Nexusflow 🤗

• Learning few-shot imitation as cultural transmission | Nature Communications

  · (mp.weixin.qq)

• Rapidly build an application in Gradio power by a Generative AI Agent | Google Cloud Blog

• 从第一性原理看大模型Agent技术

• 万字长文！何谓Agent，为何Agent？

• 首个获得驾照的AI！Agent担任私人助理样样精通，还能帮助考试作弊

• 多智能体(Agents)协作框架：人工智能的下一个方向和挑战

• Agent 将是 AI 最大的赛道！

Vector Database

• awesome-vector-database - dangkhoasdc

  A curated list of awesome works related to high dimensional structure/vector search & database

• How to choose your vector database in 2023?

  · (youtube)

Other

• GPT成功背后的秘密--向量数据库简介 - 知乎
• 7个向量数据库对比：Milvus、Pinecone、Vespa、Weaviate、Vald、GSI 和 Qdrant - 墨天轮

Extra reference

• llm-agent-survey - paitesanshi

• awesome-ai-agents - e2b-dev

  A list of AI autonomous agents

• generative_agents - joonspk-research

  Generative Agents: Interactive Simulacra of Human Behavior