# 상업적으로 이용할 수 있는 라이선스를 가진 오픈소스 LLM 목록

**URL:** https://discuss.pytorch.kr/t/llm/1510
**Category:** 읽을거리&정보공유
**Tags:** llm-for-commercial, license, opensource, llm
**Created:** [5월 9, 2023, 4:39오전 UTC](https://discuss.pytorch.kr/t/llm/1510 "2023-05-09T04:39:42Z")
**Posts on this page:** 1
**Page:** 1

<div class="post-metadata">

### Author: ![9bow](https://discuss.pytorch.kr/user_avatar/discuss.pytorch.kr/9bow/32/16301_2.png) [@9bow](https://discuss.pytorch.kr/u/9bow)
#### Post date: [5월 9, 2023, 4:39오전 UTC](https://discuss.pytorch.kr/t/llm/1510/1 "2023-05-09T04:39:42Z")

</div>

[아래 소개해드린 MPT-7B](https://discuss.pytorch.kr/t/gn-mpt-7b-llm/1509/2)에 이어, 상업적으로 사용 가능한 (라이선스를 가진) LLM들이 속속 등장하는 가운데, 이를 정리한 (& 하고 있는) [GitHub 저장소](https://github.com/eugeneyan/open-llms)가 있어 공유드립니다.

> **[GitHub - eugeneyan/open-llms: 📋 A list of open LLMs available for commercial...](https://github.com/eugeneyan/open-llms)**
>
> 📋 A list of open LLMs available for commercial use.

아래는 오늘(2023년 05월 09일) 기준, MIT, Apache 2.0, OpenRAIL-M 라이선스를 가진 LLM들의 목록입니다.

* * *

# Open LLMs

These LLMs are all licensed for commercial use (e.g., Apache 2.0, MIT, OpenRAIL-M). Contributions welcome!

| Language Model | Checkpoints | Paper/Blog | Size | Context Length | Licence |
| --- | --- | --- | --- | --- | --- |
| T5 | [T5 & Flan-T5](https://github.com/google-research/t5x/blob/main/docs/models.md#flan-t5-checkpoints), [Flan-T5-xxl (HF)](https://huggingface.co/google/flan-t5-xxl) | [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://github.com/google-research/text-to-text-transfer-transformer#released-model-checkpoints) | 60M - 11B | [512](https://discuss.huggingface.co/t/does-t5-truncate-input-longer-than-512-internally/3602) | Apache 2.0 |
| UL2 | [UL2 & Flan-UL2](https://github.com/google-research/google-research/tree/master/ul2#checkpoints), [Flan-UL2 (HF)](https://huggingface.co/google/flan-ul2) | [UL2 20B: An Open Source Unified Language Learner](https://ai.googleblog.com/2022/10/ul2-20b-open-source-unified-language.html) | 20B | [512, 2048](https://huggingface.co/google/flan-ul2#tldr) | Apache 2.0 |
| Cerebras-GPT | [Cerebras-GPT](https://huggingface.co/cerebras) | [Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models](https://www.cerebras.net/blog/cerebras-gpt-a-family-of-open-compute-efficient-large-language-models/) ([Paper](https://arxiv.org/abs/2304.03208)) | 111M - 13B | [2048](https://huggingface.co/cerebras/Cerebras-GPT-13B#model-details) | Apache 2.0 |
| Pythia | [pythia 70M - 12B](https://github.com/EleutherAI/pythia) | [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://arxiv.org/abs/2304.01373) | 70M - 12B | [2048](https://arxiv.org/pdf/2304.01373.pdf) | Apache 2.0 |
| Dolly | [dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) | [Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm) | 3B, 7B, 12B | [2048](https://github.com/databrickslabs/dolly#dolly) | MIT |
| RWKV | [RWKV, ChatRWKV](https://github.com/BlinkDL/RWKV-LM#rwkv-parallelizable-rnn-with-transformer-level-llm-performance-pronounced-as-rwakuv-from-4-major-params-r-w-k-v) | [The RWKV Language Model (and my LM tricks)](https://github.com/BlinkDL/RWKV-LM) | 100M - 14B | [infinity (RNN)](https://github.com/BlinkDL/RWKV-LM#rwkv-parallelizable-rnn-with-transformer-level-llm-performance-pronounced-as-rwakuv-from-4-major-params-r-w-k-v) | Apache 2.0 |
| GPT-J-6B | [GPT-J-6B](https://github.com/kingoflolz/mesh-transformer-jax/#gpt-j-6b), [GPT4All-J](https://github.com/nomic-ai/gpt4all#raw-model) | [GPT-J-6B: 6B JAX-Based Transformer](https://arankomatsuzaki.wordpress.com/2021/06/04/gpt-j/) | 6B | [2048](https://github.com/kingoflolz/mesh-transformer-jax/#gpt-j-6b) | Apache 2.0 |
| GPT-NeoX-20B | [GPT-NEOX-20B](https://huggingface.co/EleutherAI/gpt-neox-20b) | [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://arxiv.org/abs/2304.04165) | 20B | [2048](https://huggingface.co/EleutherAI/gpt-neox-20b) | Apache 2.0 |
| Bloom | [Bloom](https://huggingface.co/bigscience/bloom) | [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://arxiv.org/abs/2211.05100) | 176B | [2048](https://huggingface.co/bigscience/bloom) | [OpenRAIL-M v1](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) |
| StableLM-Alpha | [StableLM-Alpha](https://github.com/Stability-AI/StableLM#stablelm-alpha) | [Stability AI Launches the First of its StableLM Suite of Language Models](https://stability.ai/blog/stability-ai-launches-the-first-of-its-stablelm-suite-of-language-models) | 3B - 65B | [4096](https://github.com/Stability-AI/StableLM#stablelm-alpha) | CC BY-SA-4.0 |
| FastChat-T5 | [fastchat-t5-3b-v1.0](https://huggingface.co/lmsys/fastchat-t5-3b-v1.0) | [We are excited to release FastChat-T5: our compact and commercial-friendly chatbot!](https://twitter.com/lmsysorg/status/1652037026705985537?s=20) | 3B | 512 | Apache 2.0 |
| h2oGPT | [h2oGPT](https://github.com/h2oai/h2ogpt) | [Building the World’s Best Open-Source Large Language Model: H2O.ai’s Journey](https://h2o.ai/blog/building-the-worlds-best-open-source-large-language-model-h2o-ais-journey/) | 12B - 20B | [256 - 2048](https://huggingface.co/h2oai) | Apache 2.0 |
| MPT-7B | [MPT-7B](https://huggingface.co/mosaicml/mpt-7b), [MPT-7B-Instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) | [Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs](https://www.mosaicml.com/blog/mpt-7b) | 7B | [84k (ALiBi)](https://huggingface.co/mosaicml/mpt-7b#how-is-this-model-different) | Apache 2.0, CC BY-SA-3.0 |
| RedPajama-INCITE | [RedPajama-INCITE](https://huggingface.co/togethercomputer) | [Releasing 3B and 7B RedPajama-INCITE family of models including base, instruction-tuned & chat models](https://www.together.xyz/blog/redpajama-models-v1) | 3B - 7B | ? | Apache 2.0 |
| OpenLLaMA | [OpenLLaMA-7b-preview-300bt](https://huggingface.co/openlm-research/open_llama_7b_preview_300bt) | [OpenLLaMA: An Open Reproduction of LLaMA](https://github.com/openlm-research/open_llama) | 7B | [2048](https://huggingface.co/h2oai) | Apache 2.0 |

## LLMs for code

| Language Model | Checkpoints | Paper/Blog | Size | Context Length | Licence |
| --- | --- | --- | --- | --- | --- |
| SantaCoder | [santacoder](https://huggingface.co/bigcode/santacoder) | [SantaCoder: don't reach for the stars!](https://arxiv.org/abs/2301.03988) | 1.1B | [2048](https://huggingface.co/bigcode/santacoder/blob/main/README.md#model-summary) | [OpenRAIL-M v1](https://huggingface.co/spaces/bigcode/) |
| StarCoder | [starcoder](https://huggingface.co/bigcode/starcoder) | [StarCoder: A State-of-the-Art LLM for Code](https://huggingface.co/blog/starcoder), [StarCoder: May the source be with you!](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | 15B | [8192](https://huggingface.co/bigcode/starcoder#model-summary) | [OpenRAIL-M v1](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) |
| Replit Code | [replit-code-v1-3b](https://huggingface.co/replit/replit-code-v1-3b) | [Training a SOTA Code LLM in 1 week and Quantifying the Vibes — with Reza Shabani of Replit](https://www.latent.space/p/reza-shabani#details) | 2.7B | [infinity? (ALiBi)](https://huggingface.co/replit/replit-code-v1-3b#model-description) | CC BY-SA-4.0 |
| CodeGen2 | [codegen2 1B-16B](https://github.com/salesforce/CodeGen2) | [CodeGen2: Lessons for Training LLMs on Programming and Natural Languages](https://arxiv.org/abs/2305.02309) | 1B - 16B | [2048](https://arxiv.org/abs/2305.02309) | Apache 2.0 |

## Evals on open LLMs

- [Leaderboard by lmsys.org](https://chat.lmsys.org/?leaderboard)
- [Evals by MosaicML](https://twitter.com/jefrankle/status/1654631746506301441)
- [Holistic Evaluation of Language Models (HELM)](https://crfm.stanford.edu/helm/latest/?groups=1)
- [LLM-Leaderboard](https://github.com/LudwigStumpp/llm-leaderboard)

## LLM datasets for fine-tuning

PENDING

_Want to contribute? Just add a row above._

* * *

### What do the licences mean?

- [Apache 2.0](https://en.wikipedia.org/wiki/Apache_License): Allows users to use the software for any purpose, to distribute it, to modify it, and to distribute modified versions of the software under the terms of the license, without concern for royalties.
- [MIT](https://en.wikipedia.org/wiki/MIT_License): Similar to Apache 2.0 but shorter and simpler. Also, in contrast to Apache 2.0, does not require stating any significant changes to the original code.
- [CC BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/): Allows (i) copying and redistributing the material and (ii) remixing, transforming, and building upon the material for any purpose, even commercially. But if you do the latter, you **must distribute your contributions under the same license as the original.** (Thus, may not be viable for internal teams.)
- [OpenRAIL-M v1](https://www.bigcode-project.org/docs/pages/model-license/): Allows royalty-free access and flexible downstream use and sharing of the model and modifications of it, and comes with a set of use restrictions (see [Attachment A](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement))

**Disclaimer:** The information provided in this repo does not, and is not intended to, constitute legal advice. Maintainers of this repo are not responsible for the actions of third parties who use the models. Please consult an attorney before using models for commercial purposes.