Skip to content

Latest commit

 

History

History
299 lines (232 loc) · 12.4 KB

README_en.md

File metadata and controls

299 lines (232 loc) · 12.4 KB

AITuberKit

Notice: This project has adopted a custom license from version v2.0.0 onwards. If you are using it for commercial purposes, please check the Terms of Use section.

GitHub Last Commit GitHub Top Language GitHub Tag License: Custom

GitHub stars GitHub forks GitHub contributors GitHub issues CodeRabbit Pull Request Reviews

X (Twitter) Discord GitHub Sponsor

🌟 Demo Site 🌟

📚 Documentation Site 📚

日本語｜ 中文｜ 한국어｜ Polski

Overview

AITuberKit is an open-source toolkit that allows anyone to easily build a web application for chatting with AI characters. It features various extensions centered around interaction with AI characters and AITuber streaming functionality. It supports a wide range of AI services, character models, and voice synthesis engines, with high customization options centered around dialogue and AITuber streaming functionality.

AITuberKit Architecture

For detailed usage and configuration instructions, please visit the Documentation Site.

Star History

Star History Chart

Main Features

1. Interaction with AI Characters

  • Easy conversation with AI characters using API keys for various LLMs
  • Multimodal support for recognizing camera footage and uploaded images to generate responses
  • Retention of recent conversations as memory

2. AITuber Streaming

  • Retrieves YouTube stream comments for automatic responses from AI characters
  • Conversation continuation mode allows spontaneous speech even without comments
  • Feature to ignore comments starting with "#"

3. Other Features

  • External Integration Mode: Connect with server applications via WebSocket for advanced functionality
  • Slide Mode: Mode where AI characters automatically present slides
  • Realtime API: Low-latency dialogue and function execution using OpenAI's Realtime API
  • Audio Mode: Natural voice dialogue utilizing OpenAI's Audio API features
  • Message Reception Function: Accept instructions from external sources through a dedicated API to make AI characters speak

Supported Models & Services

Character Models

  • 3D Models: VRM files
  • 2D Models: Live2D files (Cubism 3 and later)

Supported LLMs

  • OpenAI
  • Anthropic
  • Google Gemini
  • Azure OpenAI
  • Groq
  • Cohere
  • Mistral AI
  • Perplexity
  • Fireworks
  • Local LLM
  • Dify

Supported Voice Synthesis Engines

  • VOICEVOX
  • Koeiromap
  • Google Text-to-Speech
  • Style-Bert-VITS2
  • AivisSpeech
  • GSVI TTS
  • ElevenLabs
  • OpenAI
  • Azure OpenAI
  • Niji Voice

Quick Start

Development Environment

  • Node.js: ^20.0.0
  • npm: ^10.0.0

Installation Steps

  1. Clone the repository locally.
git clone https://github.com/tegnike/aituber-kit.git
  1. Open the folder.
cd aituber-kit
  1. Install packages.
npm install
  1. Start the application in development mode.
npm run dev

  1. Open the URL: http://localhost:3000

  2. Create a .env file as needed.

cp .env.example .env

For detailed configuration and usage instructions, please visit the Documentation Site.

⚠️ Important Security Notice

This repository is intended for personal use and development in local environments, as well as commercial use with appropriate security measures. However, please note the following when deploying to a web environment:

  • API Key Handling: The system is designed to call AI services (OpenAI, Anthropic, etc.) and TTS services via a backend server, so proper management of API keys is necessary.

For Production Use

When using in a production environment, we recommend one of the following approaches:

  1. Backend Server Implementation: Manage API keys on the server side to avoid direct API access from clients
  2. Appropriate Explanation to Users: If users are using their own API keys, explain security considerations to them
  3. Access Restriction Implementation: Implement appropriate authentication and authorization mechanisms as needed

Sponsorship

We are seeking sponsors to continue development.
Your support greatly contributes to the development and improvement of AITuberKit.

GitHub Sponsor

"Buy Me A Coffee"

Contributors (in order of support)

morioki3 hodachi-axcxept coderabbitai ai-bootcamp-tokyo wmoto-ai JunzoKamahara darkgaldragon usagi917 ochisamu mo0013 tsubouchi bunkaich seiki-aliveland rossy8417 gijigae takm-reason haoling FoundD-oka terisuke konpeita MojaX2 micchi99 nekomeowww yfuku 8484ff_42 sher1ock-jp uwaguchi M1RA_A_Project

Plus multiple private sponsors

Terms of Use

License

This project has adopted a custom license from version v2.0.0 onwards.

  • Free Use

    • Free for personal use, educational purposes, and non-profit purposes that are not for commercial purposes.

  • Commercial License

    • A separate commercial license is required for commercial use.
    • For details, please check About the License.

Others

Priority Implementation

This project accepts paid priority implementation of features.

  • Features requested by companies or individuals can be implemented with priority.
  • Implemented features will be published as part of this OSS project.
  • Fees are individually quoted based on the complexity of the feature and the time required for implementation.
  • This priority implementation is separate from the commercial license. If you want to use the implemented features for commercial purposes, you need to obtain a commercial license separately.

For details, please contact [email protected].