jjang-ai/vmlx

Registry 信息

项目简介

vMLX 是一个基于 MLX 的高级框架，为 MLX Studio 提供连续批处理、前缀缓存、分页注意力、KV 缓存量化和视觉语言支持等功能。它还具备图像生成/编辑能力，并与 OpenAI 和 Anthropic API 兼容，是在 Apple Silicon 上部署本地大语言模型的多功能工具。

要点

• 连续批处理实现高效推理
• 前缀缓存与分页注意力优化
• KV 缓存量化减少内存占用
• 视觉语言（VL）支持多模态任务
• 图像生成与编辑能力
• 兼容 OpenAI 和 Anthropic API

使用场景

• 在 MacBook 上本地运行大语言模型（MLX）
• 构建视觉语言模型的多模态应用
• 通过 KV 缓存重用与压缩优化推理
• 部署 MCP 服务器支持 AI 智能体工作流
• 通过 MLX Studio 生成和编辑图像

README 摘要

<p align="center"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/jjang-ai/vmlx/main/assets/logo-wide-dark.png"> <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/jjang-ai/vmlx/main/assets/logo-wide-light.png"> <img alt="vMLX" src="https://raw.githubusercontent.com/jjang-ai/vmlx/main/assets/logo-wide-light.png" width="400"> </picture> </p> <h3 align="center">MLX Inference Server for Apple Silicon</h3> <p align="center"> Self-hosted inference server for LLMs, VLMs, and image generation on Apple Silicon.<br> OpenAI + Anthropic + Ollama compatible HTTP API. Self-hosted; no third-party API keys required.<br> Native MTP artifact detection and family-specific cache policy gates keep speculative/cache settings explicit and model-safe. </p> <p align="center"> <em>Looking for a native Swift macOS app or Swift inference engine? See <a href="https://osaurus.ai">osaurus.ai</a>.</em> </p> <p align="center"> <a href="https://pypi.org/project/vmlx/"><img src="https://img.shields.io/pypi/v/vmlx?color=%234B8BBE&label=PyPI&logo=python&logoColor=white" alt="PyPI" /></a> <a href="https://github.com/jjang-ai/vmlx/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-green?logo=apache" alt="License" /></a> <a href="https://github.com/jjang-ai/vmlx"><img src="https://img.shields.io/github/stars/jjang-ai/vmlx?style=social" alt="Stars" /></a> <img src="https://img.shields.io/badge/Apple_Silicon-M1%2FM2%2FM3%2FM4-black?logo=apple" alt="Apple Silicon" /> <img src="https://img.shields.io/badge/Python-3.10+-3776AB?logo=python&logoColor=white" alt="Python" /> <img src="https://img.shields.io/badge/Electron-28-47848F?logo=electron&logoColor=white" alt=

话题