Question 1

What is OpenChatKit and what does it provide?

Accepted Answer

OpenChatKit is an open-source project that provides a powerful base to create both specialized and general purpose chatbots for various applications. It consists of four key components: an instruction-tuned large language model, customization recipes to fine-tune the model, an extensible retrieval system to augment the model with live-updating information, and a moderation model to filter inappropriate or out-of-domain questions.

Question 2

Who are the collaborators behind OpenChatKit and the training datasets?

Accepted Answer

OpenChatKit is a collaboration between Together, LAION and Ontocord. Together is a company that provides open-source foundation models for natural language understanding and generation. LAION is a company that provides high-quality data annotation and curation services. Ontocord is a company that provides data engineering and machine learning solutions.

Question 3

How can I try out OpenChatKit and give feedback?

Accepted Answer

You can try out OpenChatKit on https://openchatkit.net and give feedback through the OpenChatKit feedback app. You can also join the OpenChatKit community on GitHub, Discord, Twitter and Medium, and share your ideas, suggestions and questions.

Question 4

What is the base model of OpenChatKit and how is it fine-tuned?

Accepted Answer

The base model of OpenChatKit is GPT-NeoXT-Chat-Base-20B, a 20 billion parameter large language model based on EleutherAI’s GPT-NeoX model. It is fine-tuned with the OIG-43M dataset, focusing on several tasks such as multi-turn dialogue, question answering, classification, extraction, and summarization.

Question 5

How does OpenChatKit perform on different natural language tasks?

Accepted Answer

OpenChatKit performs well on a broad set of natural language tasks, especially those involving question and answering, extraction and classification. It also does well on few-shot prompts, where it can leverage its instruction-tuning to adapt to different tasks. However, there are also some areas where OpenChatKit needs improvement, such as knowledge-based closed question and answering, coding tasks, repetition, context switching, and creative writing and longer answers.

Question 6

How can I access the source code, model weights and training datasets of OpenChatKit?

Accepted Answer

You can access the source code, model weights and training datasets of OpenChatKit on GitHub. You can also download the model weights and the datasets from Hugging Face.