GPT Based Technologies

Last modified by Munoz Matéo on 2023/06/22 20:22

Research on relevant GPT based technologies

Since February 2023, it seems some new Open Source GPT work is coming out every day, as well as examples of Chat-GPT prompts and usages.

Therefore this list will need to be updated to continue to be relevant. As it's hosted on the xwiki.org wiki, you are welcome to register on https://xwiki.org and start contributing to this document.

General Info

Notes on AI state of the art: https://github.com/sw-yx/ai-notes

Some links for knowledge: https://gist.github.com/rain-1/eebd5e5eb2784feecf450324e3341c8d

Fully Open Source Models & Datasets & Code

Tools

cformers

Tool to run models on CPU both as python apis and in chat mode (chat.py). Based on llama.cpp

Supports already multiple architectures and models including Open Source models (bloom,

https://github.com/NolanoOrg/cformers

Following Architectures are supported:

  • GPT-J
  • BLOOM
  • GPT-NeoX/Pythia/Open-Assistant/Open-Chat-Kit
  • CodeGen

Currently following huggingface models are supported:

  • EleutherAI/gpt-j-6B
  • bigscience/bloom-560m
  • bigscience/bloom-1b1
  • bigscience/bloom-1b7
  • bigscience/bloom-3b
  • BigScience/bloom-7b1
  • OpenAssistant/oasst-sft-1-pythia-12b
  • togethercomputer/GPT-NeoXT-Chat-Base-20B (thanks to @HCBlackFox)
  • Salesforce/codegen-350M-mono
  • Salesforce/codegen-2B-mono
  • Salesforce/codegen-6B-mono
  • Salesforce/codegen-16B-mono
  • gpt2 (thanks to @kamalojasv181)

Tested on PC and got results (not sure all models work)

Seems quite powerful. Just started March 19th so it's moving fast. Looks like a good option to work and compare model behaviors

lit-llama

Fully open source implementation of the llama code

https://github.com/Lightning-AI/lit-llama

Uses NanoGPT as the model

This would be more implementing the architecture specific aspects of llama. The model not being the llama model which is not open source, might not be as powerful. But it could mean some technical progress of llama could be accessible to Open Source models

Nvidia NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

https://github.com/NVIDIA/NeMo-Guardrails

LangChain (framework)

LangChain is a framework for developing applications powered by language models

https://docs.langchain.com/docs/

FAISS

A library for efficient similarity search and clustering of dense vectors. - useful for giving LLMs additional "memories"

https://github.com/facebookresearch/faiss

Elasticsearch also supports vector fields.

Other open source vector databases: Pinecone, Milvis ... -> https://pynomial.com/2021/10/open-source-vector-databases-overview/

Mojo - (programming language designed for AI development)

(not fully released yet but worth keeping an eye on it)

"Mojo combines the usability of Python with the performance of C, unlocking unparalleled programmability of AI hardware and extensibility of AI models."

https://www.modular.com/mojo

Cameleon llm (framework)

Chameleon is a plug-and-play compositional reasoning framework that augments LLMs with various types of tools.

https://github.com/lupantech/chameleon-llm

Untested

Gradio

Gradio is the fastest way to demo your machine learning model with a friendly web interface so that anyone can use it, anywhere!

https://www.gradio.app/

powers oobabooga/text-generation-webui

Text generation web UI

A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA.

Its goal is to become the AUTOMATIC1111/stable-diffusion-webui of text generation.

https://github.com/oobabooga/text-generation-webui

has one click installers for linux, mac and linux

LocalAI

Self-hosted, community-driven, local OpenAI-compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware. No GPU required. LocalAI is a RESTful API to run ggml compatible models: llama.cpp, alpaca.cpp, gpt4all.cpp, rwkv.cpp, whisper.cpp, vicuna, koala, gpt4all-j, cerebras and many others!

https://github.com/go-skynet/LocalAI

Installed at https://llmapi.ai.devxwiki.com. Very interesting project. Implements OpenAI API including also Chat API, Image API, and Embeddings API

Open Chat Kit

OpenChatKit provides a powerful, open-source base to create both specialized and general purpose chatbots for various applications. The kit includes an instruction-tuned 20 billion parameter language model, a 6 billion parameter moderation model, and an extensible retrieval system for including up-to-date responses from custom repositories. It was trained on the OIG-43M training dataset, which was a collaboration between Together, LAION, and Ontocord.ai. Much more than a model release, this is the beginning of an open source project. We are releasing a set of tools and processes for ongoing improvement with community contributions.

https://github.com/togethercomputer/OpenChatKit

Model based on GPT-NeoX-Chat-Base-20B

Bloom

Bloom

Largest open source model 176B Parameters (as of March 2023). Training in France:

https://huggingface.co/bigscience/bloom

BLOOM is an autoregressive Large Language Model (LLM), trained to continue text from a prompt on vast amounts of text data using industrial-scale computational resources. As such, it is able to output coherent text in 46 languages and 13 programming languages that is hardly distinguishable from text written by humans. BLOOM can also be instructed to perform text tasks it hasn't been explicitly trained for, by casting them as text generation tasks.

Bloomz

We present BLOOMZ & mT0, a family of models capable of following human instructions in dozens of languages zero-shot. We finetune BLOOM & mT5 pretrained multilingual language models on our crosslingual task mixture (xP3) and find the resulting models capable of crosslingual generalization to unseen tasks & languages.

https://huggingface.co/bigscience/bloomz

bloomz.cpp

Modified version of llama.cpp to run bloom on a CPU

https://github.com/NouamaneTazi/bloomz.cpp

Limitations: Did not fully make it work Ubuntu Linux (garbage output). Made it work on Mac. Cannot run the very large models (memory limitations probably).

Alpacoom

Fine-tuned bloom using Alpaca:

https://huggingface.co/mrm8488/Alpacoom

Untested, as it does not seem to exist in CPU version. Claims to have good results

StableML

Models from stability.ai (makes of Stable Diffusion). Annoucement here: https://stability.ai/blog/stability-ai-launches-the-first-of-its-stablelm-suite-of-language-models

https://github.com/stability-AI/stableLM/

As a proof-of-concept, we also fine-tuned the model with Stanford Alpaca's procedure using a combination of five recent datasets for conversational agents: Stanford's Alpaca, Nomic-AI's gpt4all, RyokoAI's ShareGPT52K datasets, Databricks labs' Dolly, and Anthropic's HH. We will be releasing these models as StableLM-Tuned-Alpha.

  • Base model checkpoints (StableLM-Base-Alpha) are licensed under the Creative Commons license (CC BY-SA-4.0). Under the license, you must give credit to Stability AI, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the Stability AI endorses you or your use.
  • Fine-tuned checkpoints (StableLM-Tuned-Alpha) are licensed under the Non-Commercial Creative Commons license (CC BY-NC-SA-4.0), in-line with the original non-commercial license specified by Stanford Alpaca.
  • All code in this repository is licensed under the Apache License 2.0 license.

Chat demo on HuggingFace: https://huggingface.co/spaces/stabilityai/stablelm-tuned-alpha-chat

Cerebras-GPT

Released on March 28th, 2023

https://www.cerebras.net/blog/cerebras-gpt-a-family-of-open-compute-efficient-large-language-models/

https://huggingface.co/cerebras

Untested

GPT4All

Released on March 28th, 2023 by https://home.nomic.ai/

Demo, data and code to train an assistant-style large language model with ~800k GPT-3.5-Turbo Generations based on LLaMa

Runnable on laptop with code based on alpaca.cpp:

https://github.com/nomic-ai/gpt4all

Tested using LocalAI.

MosaicML

They recently released a series of what they call MPT models under an Apache-2.0 license.
They also apparently support very long inputs, and performance wise is about the same as llama.

Base: https://huggingface.co/mosaicml/mpt-7b
They also released a few interesting fine tuned models

instruct:    https://huggingface.co/mosaicml/mpt-7b-instruct
chat:        https://huggingface.co/mosaicml/mpt-7b-chat
storywriter: https://huggingface.co/mosaicml/mpt-7b-storywriter

Tested mpt-7b-chat using FastChat. Seems quite fast, but sometimes gives short answers. Maybe should test instruct instead even in a chat context.

Open Chat Kit

See tools section

Not Fully Open Source

llama.cpp

Works on Ubuntu. However this code is using the llama model which is not Open-Source

Vicuna

An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality

Made by Universities. The models are based on LLAMA so they are not open source

Chat tool: https://chat.lmsys.org/

https://vicuna.lmsys.org/

Source code of the chat tool: https://github.com/lm-sys/FastChat

Stable-vicuna

(considered slightly better than the base vicuna)
StableVicuna-13B is a Vicuna-13B v0 model fine-tuned using reinforcement learning from human feedback (RLHF) via Proximal Policy Optimization (PPO) on various conversational and instructional datasets. 

https://huggingface.co/CarperAI/stable-vicuna-13b-delta

Tested a quantized variation of it: https://huggingface.co/Aitrepreneur/stable-vicuna-13B-GPTQ-4bit-128g  with TextgenerationwebUI

ColossalChat

Based on LLAMA also, so it's not open source

https://medium.com/@yangyou_berkeley/colossalchat-an-open-source-solution-for-cloning-chatgpt-with-a-complete-rlhf-pipeline-5edf08fb538b

Chat Tool is open source: https://chat.colossalai.org/

https://github.com/hpcaitech/ColossalAI

Alpaca-Lora

Alpaca is research work from Stanford: https://github.com/tatsu-lab/stanford_alpaca

Open Source code to reproduce the alpaca system (on other models): https://github.com/tloen/alpaca-lora

Could be used to fine-tune models.

It is however to verify if the training approach used in Alpaca-Lora can be used for commercial usage. In particular if OpenAI GPT is used for training, this could be unauthorized by OpenAI conditions.
Chat-GPT OpenAI Interesting Projects

Dolly

Fine tuning of GPT-J on Alpaca

Only the training data is available: https://github.com/databrickslabs/dolly

https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html

As it's based on Alpaca it's non commercial. However it can serve as an example on how to fine-tune an Open Source model

Interesting AI Projects

Document Indexing / Querying in Natural Language

LlamaIndex

LlamaIndex (GPT Index) is a data framework for your LLM applications

Already has community indexers for many softwares

Seems to be more a framework than a ready to go software.

Could be an interesting base for WAISE

https://github.com/jerryjliu/llama_index

Markprompt

A tool to create prompts for a collection of Markdown documents: https://markprompt.com/

It's Open Source: https://github.com/motifland/markprompt

It's unclear how it scales on volume, but it will be very interesting to study it.

Private GPT

Python code showing how to index content in a vector database and then query it in natural language. Also works with LocaAI

https://github.com/imartinez/privateGPT

Service to index documents (not open source)

https://libraria.dev/

https://klavier.ai/

AI Implementations

Notion AI

Notion (https://notion.so) has launched AI features using GPT (https://www.notion.so/product/ai)

Their prompts have been reversed engineered: https://www.latent.space/p/reverse-prompt-eng published here: https://github.com/sw-yx/ai-notes/blob/main/Resources/Notion%20AI%20Prompts.md

NextCloud

NextCloud has launched AI

https://nextcloud.com/blog/hub-4-pioneers-ethical-ai-integration-for-a-more-productive-and-collaborative-future/

Also includes the "Ethical AI Rating" in order to tell users which AI features are based on components that are Open (both the code, the data sets or the end weight of the model), and whether it's possible to self-host them. This is important because AI systems have risks, such as privacy and security of data, discrimination and bias or energy usage. In order to solve these risks, Open access will be key. This concept is very interesting and it would make a lot of sense to join them and do something similar and ideally the same.

More info at: https://nextcloud.com/blog/nextcloud-ethical-ai-rating/

Google

Google workspace AI announcement. This should be relevant in terms of using AI in collaborative tools

https://workspace.google.com/blog/product-announcements/generative-ai

Microsoft

Microsoft Office 365 / Teams AI announcement.

https://www.theverge.com/2023/3/16/23642833/microsoft-365-ai-copilot-word-outlook-teams

https://blogs.microsoft.com/blog/2023/05/04/announcing-the-next-wave-of-ai-innovation-with-microsoft-bing-and-edge/

GitHub Copilot

Relevant to writing code in the wiki:

https://github.com/features/copilot

Image, Audio, Speech

Stable Diffusion (image generation)

Online: https://stablediffusionweb.com/

Install on local computer: https://stable-diffusion-ui.github.io/ (takes 3 minutes to generate on image without GPU)

SAM - segment anything

https://github.com/facebookresearch/segment-anything

Bark - transformer-based text-to-audio model

https://huggingface.co/suno/bark

(not tested yet)

Uberduck -Text to speech

Online: https://app.uberduck.ai/ (a bit robotic but has lots of popular character voices)

Github: https://github.com/uberduck-ai (not tested locally)

Automation

AutoGPT

 https://github.com/Significant-Gravitas/Auto-GPT

AgentGPT

https://agentgpt.reworkd.ai/

BabyAGI

https://github.com/yoheinakajima/babyagi

(will test soon)

Other

 

About

About

Support

Platform

User Guide

Admin Guide

Developer Guide

Projects

XWiki

Extensions

Other

Contribute

Status

Practices

Under the Hood

Get Involved

Get Connected