OpenClaw, Superpowers, Hermes, ECC, DeepSeek Harness, the skill Claude Code by Andrej Karpathy… On GitHub, agentic tooling currently enjoys unparalleled popularity.
We broaden the perspective here to cover the entire range of Building blocks in the AI ecosystem: heterogeneous inference, routing gateways, knowledge management, etc. Here are a few with active repositories. They are arranged by star count.
MarkItDown
A Python utility under the MIT license that converts various file formats into Markdown. Signed by Microsoft, it is comparable to textract, with a focus on preserving structure (entities, lists, tables, links).
Several plug-ins are available, notably for OCR and for linking with Azure Cognitive Services. These deliver, among other things, video handling and structured field management.
OpenHands Agent Canvas
Front-end self-hosted (package npm) under the MIT license to orchestrate ACP-compatible coding agents. By default it uses the OpenHands agent, but it can leverage others, on a local or remote back-end. It can be augmented with an automation server (GitHub, Slack, Linear…) and an ingress service.
Colibri
Vincenzo Fornaro, a computer vision engineer, is the driving force behind this inference engine focused on mixture-of-experts (MoE) models. Licensed under Apache 2.0, it supports back-ends CPU, CUDA, Metal, and NUMA. Its standout feature is stitching RAM, VRAM, and NVMe into a single storage hierarchy, with a just-in-time distributed operation that allocates parameters. Principal model families supported include: DeepSeek, GLM, Inkling, Kimi, OLMoE and Qwen.
WeKnora
Tencent has developed this knowledge-management tool. Under the MIT license, it combines hybrid search (lexical and semantic; potentially agentive) with automated wiki creation and knowledge graphs.
Models, vector bases, and storage back-ends are interchangeable. An extension enables control of web browsers. Various integrations exist with productivity tools (Confluence, GitLab, Notion…) and instant messaging. WeKnora has an MCP server, a ClawHub skill, and a plug-in for the DeepSeek harness.
9Router
This local LLM gateway is MIT-licensed. It automatically shifts between three billing tiers:
- Claude Code, Codex, GitHub Copilot, and Gemini subscriptions
- Economy models (GLM, Kimi, MiniMax)
- Free versions of Kiro, OpenCode, and Vertex AI
You can use multiple accounts per provider, with automatic load balancing. And you can chain several into a single “virtual provider.” 9Router tracks quotas and automatically refreshes OAuth tokens. It can perform local embedding and compress tool results before sending them to the LLMs.
Obscura
Acknowledged under the Apache 2.0 license for this headless browser. It is pitched as an alternative to Chrome, lighter on memory, with faster initialization and better session isolation.
Obscura is compatible with the Chrome DevTools protocol. It can be connected to Puppeteer and Playwright. Cloudflare cites it as an inspiration in the development of Kitesurf, their own headless browser.
Llamafile
Originally there was Cosmopolitan. This project, based on the APE format (Actually Portable Executable), aimed to make C truly portable by reconfiguring GCC and Clang to produce a POSIX-compatible format usable across multiple OSes and CPU architectures.
Applied to the llama.cpp framework, this yields Llamafile, which enables distributing LLMs (weights plus inference code) in a single, executable file, without installation.
Under Apache 2.0, Llamafile exists in a “lite” version, without bundled CUDA and Vulkan libraries. To circumvent the 4 GB executable size limit on Windows, it allows external weights. The latest server version manages a catalog of about a dozen models, including Mistral and Qwen.
WebLLM
Another inference engine that targets browsers via WebAssembly and WebGPU. Licensed under the Apache 2.0, it handles Web Workers, enabling offloading tasks to separate threads.
It is a project from the MLC community (Machine Learning Compilation), whose ranks include OctoML and Carnegie Mellon. It leverages the ML compilation principle (capturing a tensor operation graph and then transforming it into low-level representations) to foster heterogeneous execution.
ChatGPTBox
This browser extension (Chromium, Firefox, Safari) under MIT license provides several ways to access ChatGPT: floating chat windows, a dedicated page, context-menu actions (summaries, translations, sentiment analysis, code explanations…).
ChatGPTBox has site-specific integrations (GitHub, Reddit, Stack Overflow, YouTube, search engines…). They are disableable. It can connect to third-party OpenAPI-format models and Ollama servers.
Local Deep Research
This local research assistant under the MIT license offers several modes: quick summary, detailed search, report creation, document analysis, and Q&A. It can leverage several strategies, including an autonomous LangGraph-based one.
Local Deep Research includes a REST API, an MCP server, a dashboard, and evaluations. Supposedly able to run on an RTX 3090, it handles models such as DeepSeek, Gemma, Llama, Mistral, and Qwen, via Ollama, LM Studio, and llama.cpp.
OpenWhispr
A voice dictation application under the MIT license for Windows, macOS, and Linux. It can process audio locally (Whisper, Orukeet, NVIDIA Parakeet, and Cohere Transcribe; unlimited usage) or online (GPT-5, Claude, Gemini, Groq, Tinfoil, OpenRouter; up to 2000 words per week).
Speaker diarization is performed locally. OpenWhispr can automatically transcribe Zoom, Teams, and FaceTime meetings, in addition to imported files. Users can feed it screenshots as context. All transcriptions are stored in a SQLite database with optional cloud sync. By default, responses appear at the cursor location; otherwise, they are copied to the clipboard.
Code2Prompt
Olivier D’Arcona, a Swiss ML engineer and PhD student at the University of Fribourg (thesis on document analysis by LLMs in finance and law), is the principal developer of Code2Prompt.
Under the MIT license, the tool promises to optimize converting codebases into prompts, enabling customization via Handlebars templates and Glob patterns. It is available as an API, Python SDK, skill, and MCP server.
stable-diffusion.cpp
The llamafile.cpp approach (pure C/C++ inference) applied to diffusion models—image and video—based on GGML, for a variety of back-ends (CPU, CUDA, Vulkan, Metal, OpenCL, SYCL).
MIT-licensed, stable-diffusion.cpp offers several extensions, including:
- Upscaling with ESRGAN
- SDXL image editing with PhotoMaker
- Pattern transfer with IP-Adapter
- Object removal with ADetailer (which uses YOLOv8)
GodMode
This desktop application (Windows, Mac, Linux) under the MIT license brings together on a single interface the web versions of various chatbots (ChatGPT, Claude, Perplexity, Bing, etc.). Its distinctive feature: it sends prompts to them concurrently.
The web apps are preferred to APIs because some features are integrated there in advance, or even reserved for them. This also allows covering services without APIs (Poe, for instance) and what ones do only provide APIs for certain models (Perplexity).
Lists “Awesome Prompt Engineering” and “Awesome Local LLM”
On GitHub, many topics have their own “Awesome” lists, typically in the form of link compilations.
There are, for instance, lists dedicated to self-hosting, hacking, OSINT, CLI tools, or microservices. Here we spotlight two, dedicated respectively to prompt engineering / context engineering and to local AI. Their contents are similar: reference articles, tools, models, datasets, benchmarks, tutorials…
