Colloqio
On-device AI - private, fast, always available
133 followers
On-device AI - private, fast, always available
133 followers
Meet Colloqio, the revolutionary AI companion that runs 100% on your device. No cloud servers, no data collection, complete privacy. Your personal AI that remembers you, works offline, and keeps your conversations truly private. Coming soon to iOS.
Colloqio
@invalidrun Hi Daniel, I love this. how did you get it to work technically?
Colloqio
@zolani_matebese Thanks for the question! Under the hood the model running is Gemma 3n, I did test various other models before settling on Gemma 3n and this model came out on top in terms of what I wanted Colloqio to do, memory usage, speed and how up to date the information was that it could reference. Then built the chat system and noticed quite quickly there were some limitations around long conversions, I would get maybe 12-15 long messages in and was running into issues so I implemented a system that basically compacts and summarises the current chat whilst it's the users turn to reply and then feed that back into the AI to carry on the conversion, I then implemented "Memories" basically the AI is told to pick out what it thinks might be important conversations to remember and then keeps them in memory for future conversions. Those were the two big hurdles I struggled with but everything else was relatively straight forward, there maybe still some issues with context and long conversations but I'm currently working through them as they arise.
Thank you for the question and I hope this gives you a brief insight into the technical parts of the app.
Raycast
Really cool! What’s the biggest limitation users should expect from offline models?
Colloqio
@lak7 Hey thanks for the comment! In terms of limitations I would say it works well for simple tasks, chatting, braining storming ideas, asking questions but if you try to get into complex chats and start getting into more detail of complex tasks it just isn't built for that if you use it for drafting messages, asking questions about a subject, home work (not complex homework) it's fine but go beyond that and you will see some limitations, initially I also had issues with long conversations but I've managed to get around some of those issues (not all) by compacting similar to other LLM's do when the context gets too long.
@invalidrun I see! Pretty cool man, I do have a lot of use cases for such simple and offline tasks. Currently I am using runanywhere as an offline ai, but Colloqio's ui is definitely seems better, hope it soon comes to android!
Congrats on the launch on-device AI is a strong angle. I noticed one spot where first-time users might pause around setup/expectations. Happy to share a concrete example if useful.
Colloqio
@invalidrun I went through the onboarding flow and this is the screen I hit.
One thing I noticed: being asked to download a ~3GB model before seeing any example of the product might cause hesitation, especially on mobile data. I personally paused here to think about it.
Maybe a lightweight preview (sample interaction, short demo, or clearer “what you’ll get after download”) before this step could help users feel more confident continuing.
The whole privacy first, no internet angle is exactly what we need more of. Excited to see where you take this! Quick question: which models are you running in the background?
Colloqio
Better if it will use voice commands only and talk to you like a support person.
Colloqio
@heyromix Thanks for the feedback currently it does have mic support so you can talk to the AI and send a message but it doesn't have a voice feature, I am already keeping this in mind for future releases.
Congrats on the launch. Have you created this AI companion for general purpose or targeting any vertical.
Colloqio
Sounds good so far.Will check this out
I think on-device AI is the future. Great idea. What are the minimum requirements for a device to run this?
Colloqio