Running Open-Weight LLMs Locally Cheatsheet: Quantization, Ollama, llama.cpp, vLLM

A practical cheatsheet for running open-weight LLMs on your own hardware: quantization formats (GGUF k-quants such as Q4_K_M, GPTQ, AWQ, MXFP4), VRAM math for weights and KV cache with a Python...