Skip to content

Latest commit

 

History

History
468 lines (331 loc) · 40.9 KB

awesome_openllm.md

File metadata and controls

468 lines (331 loc) · 40.9 KB

Awesome opengpt

English

Foundation

  • miqu-1-70b - miqudev 🤗

  • H2O-Danube-1.8B Technical Report, arXiv, 2401.16818, arxiv, pdf, cication: -1

    Philipp Singer, Pascal Pfeiffer, Yauhen Babakhin, Maximilian Jeblick, Nischay Dhankhar, Gabor Fodor, Sri Satish Ambati

  • Smaug-34B-v0.1 - abacusai 🤗

  • bagel-34b-v0.2 - jondurbin 🤗

  • TinyLlama: An Open-Source Small Language Model, arXiv, 2401.02385, arxiv, pdf, cication: -1

    Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, Wei Lu · (TinyLlama - jzhang38) Star

  • TigerBot: An Open Multilingual Multitask LLM, arXiv, 2312.08688, arxiv, pdf, cication: -1

    Ye Chen, Wei Cai, Liangmin Wu, Xiaowei Li, Zhanxuan Xin, Cong Fu

  • DeciLM-7B - Deci 🤗

  • DeciLM-7B-instruct - Deci 🤗

    · (huggingface)

  • LLM360: Towards Fully Transparent Open-Source LLMs, arXiv, 2312.06550, arxiv, pdf, cication: -1

    Zhengzhong Liu, Aurick Qiao, Willie Neiswanger, Hongyi Wang, Bowen Tan, Tianhua Tao, Junbo Li, Yuqi Wang, Suqi Sun, Omkar Pangarkar

  • GPT4All: An Ecosystem of Open Source Compressed Language Models, arXiv, 2311.04931, arxiv, pdf, cication: -1

    Yuvanesh Anand, Zach Nussbaum, Adam Treat, Aaron Miller, Richard Guo, Ben Schmidt, GPT4All Community, Brandon Duderstadt, Andriy Mulyar · (gpt4all - nomic-ai) Star

  • OpenChat: Advancing Open-source Language Models with Mixed-Quality Data, arXiv, 2309.11235, arxiv, pdf, cication: -1

    Guan Wang, Sijie Cheng, Xianyuan Zhan, Xiangang Li, Sen Song, Yang Liu · (openchat - imoneoi) Star · (huggingface) · (openchat)

  • Zephyr: Direct Distillation of LM Alignment, arXiv, 2310.16944, arxiv, pdf, cication: 1

    Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib · (alignment-handbook - huggingface) Star

  • H2O Open Ecosystem for State-of-the-art Large Language Models, arXiv, 2310.13012, arxiv, pdf, cication: -1

    Arno Candel, Jon McKinney, Philipp Singer, Pascal Pfeiffer, Maximilian Jeblick, Chun Ming Lee, Marcos V. Conde

  • BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model, arXiv, 2309.11568, arxiv, pdf, cication: -1

    Nolan Dey, Daria Soboleva, Faisal Al-Khateeb, Bowen Yang, Ribhu Pathria, Hemant Khachane, Shaheer Muhammad, Zhiming, Chen, Robert Myers

  • OpenBA: An Open-sourced 15B Bilingual Asymmetric seq2seq Model Pre-trained from Scratch, arXiv, 2309.10706, arxiv, pdf, cication: -1

    Juntao Li, Zecheng Tang, Yuyang Ding, Pinzheng Wang, Pei Guo, Wangjie You, Dan Qiao, Wenliang Chen, Guohong Fu, Qiaoming Zhu · (openba - opennlg) Star

  • XGen-7B Technical Report, arXiv, 2309.03450, arxiv, pdf, cication: 3

    Erik Nijkamp, Tian Xie, Hiroaki Hayashi, Bo Pang, Congying Xia, Chen Xing, Jesse Vig, Semih Yavuz, Philippe Laban, Ben Krause

  • FLM-101B: An Open LLM and How to Train It with $100K Budget, arXiv, 2309.03852, arxiv, pdf, cication: 3

    Xiang Li, Yiqun Yao, Xin Jiang, Xuezhi Fang, Xuying Meng, Siqi Fan, Peng Han, Jing Li, Li Du, Bowen Qin · (huggingface)

  • adept-inference - persimmon-ai-labs Star

    Inference code for Persimmon-8B

  • WizardLM - nlpxucan Star

    Family of instruction-following LLMs powered by Evol-Instruct: WizardLM, WizardCoder

  • FreeWilly2 - stabilityai 🤗

  • xgen - salesforce Star

    Salesforce open-source LLMs with 8k sequence length.

  • PolyLM: An Open Source Polyglot Large Language Model, arXiv, 2307.06018, arxiv, pdf, cication: 5

    Xiangpeng Wei, Haoran Wei, Huan Lin, Tianhao Li, Pei Zhang, Xingzhang Ren, Mei Li, Yu Wan, Zhiwei Cao, Binbin Xie

  • A Technical Report for Polyglot-Ko: Open-Source Large-Scale Korean Language Models, arXiv, 2306.02254, arxiv, pdf, cication: -1

    Hyunwoong Ko, Kichang Yang, Minho Ryu, Taekyoon Choi, Seungmu Yang, Jiwung Hyun, Sungho Park, Kyubyong Park

  • Examining User-Friendly and Open-Sourced Large GPT Models: A Survey on Language, Multimodal, and Scientific GPT Models, arXiv, 2308.14149, arxiv, pdf, cication: -1

    Kaiyuan Gao, Sunan He, Zhenyu He, Jiacheng Lin, QiZhi Pei, Jie Shao, Wei Zhang · (gpt_alternatives - GPT-Alternatives) Star · (jiqizhixin)

OLMo

  • OLMo: Accelerating the Science of Language Models, arXiv, 2402.00838, arxiv, pdf, cication: -1

    Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang · (OLMo - allenai) Star · (allenai) · (allenai)

  • OLMo-7B - allenai 🤗

Phi

Mistral

StripedHyena-7B

BLOOM

Mosaic pretrained transformers (MPT)

GitHub - mosaicml/llm-foundry: LLM training code for MosaicML foundation models

h2oGPT

  • h2oGPT: Democratizing Large Language Models, arXiv, 2306.08161, arxiv, pdf, cication: -1

    Arno Candel, Jon McKinney, Philipp Singer, Pascal Pfeiffer, Maximilian Jeblick, Prithvi Prabhu, Jeff Gambera, Mark Landry, Shivam Bansal, Ryan Chesler

LLaMA

Falcon

Pythia

[2304.01373] Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

 · ([pythia](https://github.com/EleutherAI/pythia) - EleutherAI) ![Star](https://img.shields.io/github/stars/EleutherAI/pythia.svg?style=social&label=Star)

Other

Finetuning

Vicuna

  • Flacuna: Unleashing the Problem Solving Power of Vicuna using FLAN Fine-Tuning, arXiv, 2307.02053, arxiv, pdf, cication: 3

    Deepanway Ghosal, Yew Ken Chia, Navonil Majumder, Soujanya Poria

  • FastChat - lm-sys Star

    An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and FastChat-T5.

Alpaca

Dolly

  • dolly - databrickslabs Star

    Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform · (huggingface) · (databricks)

Misc

Mulitlingual (chinese)

Foundation

  • MiniCPM - OpenBMB Star

    MiniCPM-2.4B: An end-side LLM outperforms Llama2-13B.

    · (huggingface)

  • iFlytekSpark-13B: 讯飞星火开源-13B(iFlytekSpark-13B)

  • Orion-14B: Open-source Multilingual Large Language Models, arXiv, 2401.12246, arxiv, pdf, cication: -1

    Du Chen, Yi Huang, Xiaopu Li, Yongqiang Li, Yongqiang Liu, Haihui Pan, Leichao Xu, Dacheng Zhang, Zhipeng Zhang, Kun Han

  • Orion - OrionStarAI Star

    Orion-14B is a family of models includes a 14B foundation LLM, and a series of models: a chat model, a long context model, a quantized model, a RAG fine-tuned model, and an Agent fine-tuned model. Orion-14B 系列模型包括一个具有140亿参数的多语言基座大模型以及一系列相关的衍生模型,包括对话模型,长文本模型,量化模型,RAG微调模型,Agent微调模型等。 · (Orion - OrionStarAI) Star

  • TeleChat Technical Report, arXiv, 2401.03804, arxiv, pdf, cication: -1

    Zihan Wang, Xinzhang Liu, Shixuan Liu, Yitong Yao, Yuyao Huang, Zhongjiang He, Xuelong Li, Yongxiang Li, Zhonghao Che, Zhaoxi Zhang

  • YAYI 2: Multilingual Open-Source Large Language Models, arXiv, 2312.14862, arxiv, pdf, cication: -1

    Yin Luo, Qingchao Kong, Nan Xu, Jia Cao, Bao Hao, Baoyu Qu, Bo Chen, Chao Zhu, Chenyang Zhao, Donglei Zhang

  • SeaLLMs -- Large Language Models for Southeast Asia, arXiv, 2312.00738, arxiv, pdf, cication: -1

    Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu

    · (SeaLLMs - DAMO-NLP-SG) Star

  • YUAN 2.0: A Large Language Model with Localized Filtering-based Attention, arXiv, 2311.15786, arxiv, pdf, cication: -1

    Shaohua Wu, Xudong Zhao, Shenling Wang, Jiangang Luo, Lingjun Li, Xi Chen, Bing Zhao, Wei Wang, Tong Yu, Rongguo Zhang · (Yuan-2.0 - IEIT-Yuan) Star

  • Ziya2: Data-centric Learning is All LLMs Need, arXiv, 2311.03301, arxiv, pdf, cication: -1

    Ruyi Gan, Ziwei Wu, Renliang Sun, Junyu Lu, Xiaojun Wu, Dixiang Zhang, Kunhao Pan, Ping Yang, Qi Yang, Jiaxing Zhang

    · (huggingface)

  • Skywork: A More Open Bilingual Foundation Model, arXiv, 2310.19341, arxiv, pdf, cication: 1

    Tianwen Wei, Liang Zhao, Lichang Zhang, Bo Zhu, Lijie Wang, Haihua Yang, Biye Li, Cheng Cheng, Weiwei Lü, Rui Hu · (jiqizhixin) · (qbitai) · (skywork - skyworkai) Star

  • Aquila2 - FlagAI-Open Star

    The official repo of Aquila2 series proposed by BAAI, including pretrained & chat large language models. · (mp.weixin.qq)

  • ColossalAI - hpcaitech Star

    Making large AI models cheaper, faster and more accessible · (qbitai)

  • VisCPM - OpenBMB Star

    基于CPM基础模型的中英双语多模态大模型系列 · (jiqizhixin)

Yi-01

  • Yi - 01-ai Star

    A series of large language models trained from scratch by developers @01-ai

    · (jiqizhixin)

InterLM

DeepSeek

  • DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence, arXiv, 2401.14196, arxiv, pdf, cication: -1

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li

  • DeepSeek-MoE - deepseek-ai Star

    · (huggingface)

  • DeepSeek-LLM - deepseek-ai Star

    DeepSeek LLM: Let there be answers · (huggingface) · (mp.weixin.qq)

Xverse

Qwen

Baichuan

  • Baichuan 2: Open Large-scale Language Models, arXiv, 2309.10305, arxiv, pdf, cication: 16

    Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan · (Baichuan2 - baichuan-inc) Star · (cdn.baichuan-ai) · (mp.weixin.qq) · (jiqizhixin)

  • Baichuan-13B - baichuan-inc Star

    A 13B large language model developed by Baichuan Intelligent Technology · (mp.weixin.qq)

  • baichuan-7B - baichuan-inc Star

    A large-scale 7B pretraining language model developed by BaiChuan-Inc.

ChatGLM

  • ChatGLM3 - THUDM Star

    ChatGLM3 series: Open Bilingual Chat LLMs | 开源双语对话语言模型 · (qbitai)

  • ChatGLM2-6B - THUDM Star

    ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型 · (qbitai)

  • chatglm.cpp - li-plus Star

    C++ implementation of ChatGLM-6B & ChatGLM2-6B

  • TigerBot - TigerResearch Star

    TigerBot: A multi-language multi-task LLM · (qbitai)

Finetuning

  • Aurora:Activating Chinese chat capability for Mistral-8x7B sparse Mixture-of-Experts through Instruction-Tuning, arXiv, 2312.14557, arxiv, pdf, cication: -1

    Rongsheng Wang, Haoming Chen, Ruizhe Zhou, Yaofei Duan, Kunyan Cai, Han Ma, Jiaxi Cui, Jian Li, Patrick Cheong-Iao Pang, Yapeng Wang

    · (Aurora - WangRongsheng) Star

  • Taiwan-LLaMa - MiuLab Star

    Traditional Mandarin LLMs for Taiwan

  • Chinese-LLaMA-Alpaca-2 - ymcui Star

    中文LLaMA-2 & Alpaca-2大语言模型 (Chinese LLaMA-2 & Alpaca-2 LLMs)

  • TransGPT - DUOMO Star

    · (jiqizhixin)

  • Llama2-Chinese - FlagAlpha Star

    Llama中文社区,最好的中文Llama大模型,完全开源可商用

  • Chinese-Llama-2-7b - LinkSoul-AI Star

    开源社区第一个能下载、能运行的中文 LLaMA2 模型!

  • ChatGLM-Efficient-Tuning - hiyouga Star

    Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调

  • BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models, arXiv, 2306.10968, arxiv, pdf, cication: -1

    Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma, Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen · (jiqizhixin) · (BayLing - ictnlp) Star · (huggingface)

Other

Extra

  • CroissantLLM: A Truly Bilingual French-English Language Model, arXiv, 2402.00786, arxiv, pdf, cication: -1

    Manuel Faysse, Patrick Fernandes, Nuno Guerreiro, António Loison, Duarte Alves, Caio Corro, Nicolas Boizard, João Alves, Ricardo Rei, Pedro Martins

  • MaLA-500: Massive Language Adaptation of Large Language Models, arXiv, 2401.13303, arxiv, pdf, cication: -1

    Peiqin Lin, Shaoxiong Ji, Jörg Tiedemann, André F. T. Martins, Hinrich Schütze · (huggingface)

  • Multilingual Instruction Tuning With Just a Pinch of Multilinguality, arXiv, 2401.01854, arxiv, pdf, cication: -1

    Uri Shaham, Jonathan Herzig, Roee Aharoni, Idan Szpektor, Reut Tsarfaty, Matan Eyal

  • LLaMA Beyond English: An Empirical Study on Language Capability Transfer, arXiv, 2401.01055, arxiv, pdf, cication: -1

    Jun Zhao, Zhihao Zhang, Qi Zhang, Tao Gui, Xuanjing Huang

  • 2023, year of open LLMs

  • FinGPT: Large Generative Models for a Small Language, arXiv, 2311.05640, arxiv, pdf, cication: -1

    Risto Luukkonen, Ville Komulainen, Jouni Luoma, Anni Eskelinen, Jenna Kanerva, Hanna-Mari Kupari, Filip Ginter, Veronika Laippala, Niklas Muennighoff, Aleksandra Piktus · (turkunlp)

Toolkits

  • LLMZoo - FreedomIntelligence Star

    ⚡LLM Zoo is a project that provides data, models, and evaluation benchmark for large language models.⚡

Extra reference