good thread walking through the tech here as well @jvaleska.eth

paddy (@0xPD33) on XThe challenge of optimizing LLM inference on phones is way greater than it should be. This is partly because not many people are doing it, but also because on most systems it seems incredibly difficult to get proper access to the hardware underneath the OS. I'll break down what I've tried and wh…x.com