Show HN: I built an on-device LLM based chatbot for iPhones

20 points by woadwarrior01 a year ago

I’ve been playing around with local LLMs for the past couple of months and decided to build something that can run on an iPhone. It’s a universal app built with SwiftUI and the excellent ggml library. The model is an SFT fine tuned and 4 bit quantised version of the RedPajama-INCITE-Chat-3B-v1 OSS LLM. It works reasonably well on recent-ish (~3 year old) iPhones, iPads and Macs. It was launched on the App Store yesterday[1] and Product Hunt today[2]. It seems to be reasonably ok at natural language interactions, but given its size, does pretty badly at coding and reasoning. Also, it sometimes hallucinates after a few back and forth messages. The fine tuning for natural language Q&A seems to have made the base model’s already bad performance on coding tasks, even worse. I’ll add a few App Store promo codes in a comment below, if anyone would like to try it out.

[1]: https://apple.co/43n6BsW [2]: https://www.producthunt.com/posts/personal-gpt