Post
17
MNN-Pocket-0.3B-Dec on a Galaxy S26: your app can ask its own multiple-choice questions and get each option's probability, with no LLM API. The NPU kept every tested probability within 0.02 of my PyTorch run of the original model.
litert-community/MNN-Pocket-0.3B-Dec-LiteRT
A model from Alibaba's MNN team for multiple-choice, yes/no and rating questions, converted to LiteRT.
In the clip, a test app answers questions in 13.5, 15.6 and 13.7 ms on the NPU, plus 9.5 to 17.6 ms in its own code.
To run it: the card's Python host; for Android, port it as the test app did.
litert-community/MNN-Pocket-0.3B-Dec-LiteRT
A model from Alibaba's MNN team for multiple-choice, yes/no and rating questions, converted to LiteRT.
In the clip, a test app answers questions in 13.5, 15.6 and 13.7 ms on the NPU, plus 9.5 to 17.6 ms in its own code.
To run it: the card's Python host; for Android, port it as the test app did.