The AI Dev Toolkit Landscape
Frameworks, tracing, evals, model hubs, cookbooks, benchmarks, and the other tabs everyone has open.
The AI dev toolkit is not one category. It is several categories wearing the same hoodie.
Frameworks
LangChain, LlamaIndex, Haystack, Semantic Kernel, and Spring AI help glue model calls, prompts, retrieval, and tools into application flows. They save time when your app has repeated patterns. They add weight when all you needed was one API call.
Observability
Langfuse, LangSmith, Braintrust, Laminar, and W&B help you inspect prompts, outputs, costs, latency, and traces. You need this when debugging stops being "print the response."
Evals
promptfoo, Braintrust, custom pytest checks, and human review workflows tell you whether changes made the system better or worse.
Model hubs and benchmark sites
Hugging Face helps you find models, datasets, demos, and libraries. Artificial Analysis and similar sites help compare cost, speed, and quality signals.
Provider cookbooks
OpenAI, Anthropic, Google, and others publish examples that are often more useful than generic blog posts because they show the exact API shape.
The trick is not collecting tools. The trick is knowing which layer is currently hurting.