small-language-model

This is a Phi Family of SLMs book for getting started with Phi Models. Phi a family of open sourced AI models developed by Microsoft. Phi models are the most capable and cost-effective small language models (SLMs) available, outperforming models of the same size and next size up across a variety of language, reasoning, coding, and math benchmarks

Awesome list of TypeSafe AI Jev use cases: 74 demos ranked by likes, 150+ GitHub repos, limits, cost and API examples. CC0

A small proof-of-concept language model (not an LLM) that aims to incorporate latent-space prediction, internal state using recurrent trace units, and byte-by-byte output, built with MLX.

A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apache-2.0.

A 124.6M LLM trained from scratch on 13B tokens on a single RTX 4090 — tokenizer, pretraining, post-training, evaluation, and reproducible inference.

BarunLM-35M: a parameter-efficient 35M base language model trained on 5.7B tokens

Wonderful Matrices to Build Small Language Models

Sparse-upcycle a dense Gemma-3-270m into a Mixture-of-Experts (gemma3moe) and serve it on llama.cpp (CPU). Upcycle, train, inspect routing, GGUF.

SLM-SQL: An Exploration of Small Language Models for Text-to-SQL

Zest. Natural Language to CLI commands tool from Spicy Lemonade.

An Android App recreating the Simon Says game. Uses MediaPipe to run an LLM on device

Small foundation models of human cognition and behaviour

This repository provides everything you need to perform Supervised Fine-Tuning (SFT) of the Qwen2.5-Coder-1.5B-Instruct model—or any of its larger variants (7B, 14B, 32B)—on the Qwen Models, using the nvidia/OpenCodeReasoning dataset.

A tool-calling layer, not a language model — your schemas in, validated calls out, at 48M parameters. Malformed JSON, invented parameter names, and undeclared tools are structurally unreachable on any catalog. Adapt it to your own catalog; 11 negative results included.

Online Self-Distillation Fine-Tuning (SDFT) for Small Language Models (SLM)

NIRNAY 450M: open-weight calibrated decision model (Apache-2.0). Banking77 intent classification 87.9%, fitted ECE 0.045. Laya fork plus concept bottleneck, coarse-to-fine pointer and RLCD. Built by Eulogik.

Code for paper "Achieving Sparse Activation in Small Language Models"

Prajna — on-device memory-augmented reasoning: a 6.7M Cognitive Resonance Network inside frozen Gemma 4 E2B. Passes CEHRI 60/60 via retrieval memory, 91.7% on unseen reworded exams. Trained on a Mac Mini M4, no GPU, no API.

A 35M-parameter local compiler for typed personal actions

A Streamlit demo using an SLM (Phi) and RAG to showcase how an AI can help users with learning functionality on a website