Vecy On Device Ai Llm App For Rag

Android AI app for private LLM chats with local files using RAG.

Vecy is an Android AI app that allows users to chat with fully-private LLMs and local files (documents, photos). It automatically processes and indexes local files on the phone, and users can also directly upload files. Vecy offers RAG tools for processing local files to enhance LLM responses. Users can select from a range of small LLMs (from 0.5B to 3.5B parameters) and engage in conversations with documents stored locally. The app automatically processes files (PDFs, DOCX documents, images, etc.) within the Documents, Download, and DCIM directories to generate embedding vectors, which are stored locally on the device. These embedding vectors are crucial for RAG (retrieval-augmented generation) capabilities, enhancing the LLM's ability to provide more accurate and relevant responses when queries are related to the local files.