The daily firehose — everything the toolchain shipped today, already filtered.
// HOW THIS ISSUE IS MADE
We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.
shell-gpt 1.2.0 adds --no-interaction flag for redirectable shell output and stdin support in REPL mode.
└──▷ GET THIS VERSION
$ git clone --branch 1.2.0 https://github.com/TheR1D/shell_gpt.git
# already have the repo? check out this version:$ git checkout 1.2.0
└──▷ TRY IT
Seed a REPL session with a source file as context, then ask follow-up questions interactively.
$ sgpt --repl temp < my_app.py
›Adds --no-interaction flag (used with --shell) to print the suggested command to stdout instead of interactive mode, enabling shell pipeline use like sgpt -s "say hi" | pbcopy.
›REPL mode now accepts stdin, a PROMPT argument, or both simultaneously, allowing initial context to be piped in alongside an interactive session.
└──▷ BREAKING ON UPGRADE
!Shell integration in ~/.bashrc or ~/.zshrc will stop working on upgrade; run sgpt --install-integration and manually remove the old integration function from your shell profile.
1 more release in this issue
· 2024-01-09 → 2024-01-28
shell-gpt 1.1.0 adds OpenAI function calling, letting the LLM execute shell commands and AppleScripts via new --install-functions and --functions flags.
└──▷ GET THIS VERSION
$ git clone --branch 1.1.0 https://github.com/TheR1D/shell_gpt.git
# already have the repo? check out this version:$ git checkout 1.1.0
└──▷ TRY IT
Install default functions so the LLM can run shell commands and AppleScripts on your machine.
$ sgpt --install-functions
Point shell-gpt at a custom functions directory and enable function calling by default in your config.
›Adds --install-functions flag to download and install default functions, enabling the LLM to execute shell commands and AppleScripts (macOS) directly on your system.
›Adds --functions flag to enable or disable OpenAI function calling at invocation time.
›Adds OPENAI_FUNCTIONS_PATH config variable to specify the directory where custom function definitions are loaded from.
›Adds OPENAI_USE_FUNCTIONS config variable to enable or disable function calling globally in the config.
›Adds SHOW_FUNCTIONS_OUTPUT config variable to control whether function execution output is displayed.
+2 moreshow less
›Adds shortcut -c as an alias for --code, -lc for --list-chats, and -lr for --list-roles.
›Integrates the OpenAI Python library for API requests, providing more descriptive error messages with suggested solutions.
Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
CrewAI v0.1.32 adds per-agent iteration limits, RPM throttling, and initial i18n support
└──▷ GET THIS VERSION
$ git clone --branch v0.1.32 https://github.com/crewAIInc/crewAI.git
# already have the repo? check out this version:$ git checkout v0.1.32
›Adds ability to limit the maximum number of iterations for an agent, preventing runaway agent loops.
›Adds Request Per Minute (RPM) throttling configurable for both individual Agents and Crews.
›Adds initial internationalization (i18n) support with a Greek translation included.
›Adds EmbeddingRetriever support for Amazon Bedrock embedding models, including amazon.titan-embed-text-v1 and Cohere models, via an aws_config parameter accepting aws_access_key_id, aws_secret_access_key, and aws_session_token.
›Adds an optional webdriver parameter to Crawler.__init__ to supply a pre-configured custom WebDriver instead of the default Chrome driver.
›Adds model_kwargs argument to FARMReader to support loading the model in fp16 at inference time.
›Adds model_kwargs argument to SentenceTransformersRanker to pass HuggingFace Transformers loading options.
›Makes JoinDocuments sensitive to the weights parameter and adds score normalization when join_mode is reciprocal rank fusion.
+1 moreshow less
›Optimizes PineconeDocumentStore.write_documents upserts with asynchronous requests.
$ git clone --branch v0.1.3 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:$ git checkout v0.1.3
›Adds MlflowEmbeddings support for additional kwargs, enabling compatibility with the Cohere API.
›Adds ElasticsearchStore relevance function selector, allowing callers to choose the scoring function at query time.
›Adds max inner product support to ElasticsearchStore as a new distance/similarity option.
›Enables vector length definition at PGVector init time, allowing index creation with an explicit dimension without needing to infer it from the first document.
›Adds DeepInfra as a supported provider for chat models via a new DeepInfra chat model integration.
+9 moreshow less
›Enables LangChain built-in tools inside Gemini function calling via langchain_google_vertexai.
›Re-enables streaming support for GPT4All models.
›Adds support for Amazon Titan Express as a chat model via BedrockChat.
›Adds async methods to Bedrock LLM integration.
›Adds TiDB as a message history store backend.
›Adds TigerGraph as a supported graph database integration.
›Adds a new document loader for Visio files (.vsdx extension).
›Updates Memgraph integration with expanded support.
LangChain v0.1.2 adds function calling on VertexAI, MistralAI embeddings, astream_events on Runnables, and more new integrations.
└──▷ GET THIS VERSION
$ git clone --branch v0.1.2 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:$ git checkout v0.1.2
└──▷ USE IT
Stream granular chain/agent events in real time — useful for building responsive UIs or detailed observability pipelines.
python
async for event in chain.astream_events({"input": "What is LangChain?"}, version="v1"):
print(event)
Tag a dataset evaluation run with the current git revision so results are traceable to an exact commit.
python
from langchain.smith import run_on_dataset
run_on_dataset(
client=client,
dataset_name="my-dataset",
llm_or_chain_factory=chain,
revision_identifier="v1.2.0-4-gabcdef1",
)
Apply Gemini safety settings at the wrapper level to enforce content policies across all requests.
python
from langchain_google_vertexai import ChatVertexAI
from vertexai.generative_models import HarmCategory, HarmBlockThreshold
llm = ChatVertexAI(
model_name="gemini-pro",
safety_settings={
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
},
)
›Adds astream_events method to Runnables (with required version parameter while in beta) for streaming granular event data from chains and agents.
›Adds safety_settings property to the Gemini wrapper in google-vertexai.
›Adds revision_identifier parameter to run_on_dataset; falls back to the LANGCHAIN_REVISION_ID environment variable or git describe when not passed explicitly.
›Adds support for function calling on VertexAI via the google-vertexai partner package.
›Adds SystemMessage support for the Gemini chat model in langchain_google_vertexai.
+11 moreshow less
›Adds MistralAI embeddings via the mistralai partner package.
›Adds a Cassandra document loader (CassandraLoader) in langchain_community.
›Adds PolygonLastQuote tool and toolkit to langchain_community.
›Adds KoNLPy-based text splitter for Korean-language text in langchain.
›Adds neo4j timeout and value sanitization options to the Neo4j integration.
›Adds streaming logprobs support for OpenAI models.
›Adds basic logging and human-input capability to ShellTool in langchain_community.
›Supports more comparators in the Milvus self-querying retriever.
›Allows the OpenSearch Query Translator to correctly handle Date types.
›Uses MetadataVectorCassandraTable in the Cassandra vector store for improved metadata handling.
›Improves PGVector insert performance via SQLAlchemy's bulk_save_objects method.
Letta 0.3 moves all agent and user state into database storage and adds a hosted multi-user server mode.
└──▷ GET THIS VERSION
$ git clone --branch 0.3 https://github.com/letta-ai/letta.git
# already have the repo? check out this version:$ git checkout 0.3
└──▷ TRY IT
Preserve access to pre-0.3 agents and data sources by migrating them into the new database storage before doing anything else.
$ memgpt migrate
Spin up a local multi-user Letta server that exposes a REST API (with generated OpenAPI spec) for integrating agents into downstream applications.
$ memgpt server
›Adds memgpt server command to run Letta as a hosted service on http://localhost:8283, serving multiple users and emitting an openapi.json spec on startup.
›Adds memgpt migrate command to move existing agent state and data sources from ~/.memgpt/config into the new database-backed storage layer.
›All agent, user, and system state is now persisted in database storage (local SQLite and Chroma by default, configurable), enabling multi-user deployments.
└──▷ BREAKING ON UPGRADE
!Existing agents and data sources in ~/.memgpt/config are inaccessible after upgrading to 0.3 until migrated with memgpt migrate.
1 more release in this issue
· 2024-01-03 → 2024-01-30
AutoGen v0.2.8 adds Redis caching, a web surfer agent, and human-input initiate_chat with no message required.
└──▷ GET THIS VERSION
$ git clone --branch v0.2.8 https://github.com/microsoft/autogen.git
# already have the repo? check out this version:$ git checkout v0.2.8
└──▷ USE IT
Kick off a multi-agent conversation that prompts a human for the opening message instead of hard-coding one.
python
human_proxy.initiate_chat(assistant)
›Adds Redis cache support (alongside existing diskcache) for agent chat and LLM client inference via initiate_chat and client-level caching APIs.
›Allows initiate_chat to be called without passing a message, enabling the agent conversation to begin with human input instead.
›Adds a new web surfer agent capable of searching and browsing the web autonomously.
›Adds a dev container for AutoGen Studio to streamline development environment setup.
└──▷ BREAKING ON UPGRADE
!use_docker now defaults to True; setups that previously relied on the False default will begin attempting to run code in Docker containers.
!last_n_messages now defaults to 'auto'; setups that relied on the previous numeric default may see different conversation-context truncation behavior.
!In the next release (not this one), the default value of use_docker in code_execution_config will change to True; set it to False or None explicitly now to avoid docker being enabled automatically on upgrade.
AutoGen v0.2.4 adds teachability for any agent, OpenAI tool-call support, and AutoBuild agent-library construction.
└──▷ GET THIS VERSION
$ git clone --branch v0.2.4 https://github.com/microsoft/autogen.git
# already have the repo? check out this version:$ git checkout v0.2.4
›Adds OpenAI tool-call support to conversable agents, enabling agents to invoke tool calls returned by the API.
›Introduces a generic extensibility mechanism that lets any conversable agent become teachable — not just built-in agent types — as demonstrated by the new GPTAssistantAgent teachability example.
›Extends AutoBuild to support building agents from an agent library and auto-generating agent descriptions for group chat.
└──▷ BREAKING ON UPGRADE
!GPT-4 is no longer the default model; callers that relied on the implicit default will now receive an error — the model must be set explicitly whenever an LLM is used.
Semantic Kernel 1.2.0 adds Function and Prompt Filters, extended FunctionResult, and OpenAPI payload default values.
└──▷ GET THIS VERSION
$ git clone --branch dotnet-1.2.0 https://github.com/microsoft/semantic-kernel.git
# already have the repo? check out this version:$ git checkout dotnet-1.2.0
›Introduces Function and Prompt Filters as the new interception model, replacing the now-deprecated Kernel events (SKEXP0003/SKEXP0004).
›Adds FunctionResultExtended to expose richer metadata from kernel function invocations.
›Supports DefaultValue for OpenAPI payload properties, improving plugin reliability when callers omit optional fields.
›Updates FlowOrchestrator to use YAML plugins for defining orchestration steps.
›Adds an example demonstrating how to use the OpenAI response_format property for structured outputs.
+1 moreshow less
›Extends chat message parsing to handle a broader range of message shapes.
└──▷ BREAKING ON UPGRADE
!Kernel events are marked deprecated in favor of Filters; CancelKernelEventArgs is now attributed SKEXP0003 (was SKEXP0004), which may affect experimental-feature suppressions.
!The NCalc Plugin has been removed from the plugin library.
!Polly has been removed as a dependency, so any code relying on Polly being transitively available through Semantic Kernel will need to add it directly.
4 more releases in this issue
· 2024-01-05 → 2024-01-24
Semantic Kernel Python renames Skills to Plugins and completion settings to execution_settings for .NET alignment
└──▷ GET THIS VERSION
$ git clone --branch python-0.4.6.dev https://github.com/microsoft/semantic-kernel.git
# already have the repo? check out this version:$ git checkout python-0.4.6.dev
›Renames all Skills references to Plugins across class names, variable names, filenames, and directory names to align with SK .NET conventions.
›Renames completion settings to execution_settings in PromptTemplateConfig and AIRequestSettings to match SK .NET behavior.
└──▷ BREAKING ON UPGRADE
!All Skills-named classes, variables, filenames, and directories are renamed to Plugins — any code referencing the old Skills names will break on upgrade.
!The completion settings key in PromptTemplateConfig and AIRequestSettings is renamed to execution_settings — existing configurations using completion will break on upgrade.
Semantic Kernel 1.1.0 adds agent tool support, instruction templating, DI-resolved OpenAI clients, and a new ResponseFormat setting.
└──▷ GET THIS VERSION
$ git clone --branch dotnet-1.1.0 https://github.com/microsoft/semantic-kernel.git
# already have the repo? check out this version:$ git checkout dotnet-1.1.0
└──▷ USE IT
Force the OpenAI completion to return a JSON object by setting ResponseFormat on execution settings.
csharp
var settings = new OpenAIPromptExecutionSettings
{
ResponseFormat = "json_object"
};
var result = await kernel.InvokePromptAsync(prompt, new(settings));
Resolve a pre-configured OpenAIClient from the DI container instead of passing credentials explicitly.
csharp
builder.Services.AddSingleton<OpenAIClient>(sp => new OpenAIClient(new Uri(endpoint), new AzureKeyCredential(apiKey)));
builder.Services.AddAzureOpenAIChatCompletion(deploymentName: "gpt-4");
›Adds OpenAIPromptExecutionSettings.ResponseFormat property to control the response format returned by OpenAI completions.
›Adds support for agent tools code-interpreter and retrieval on OpenAI Assistants-based agents.
›Adds support for instruction templating on Agents, enabling dynamic prompt construction at the agent level.
›Adds previous plan and error context to Handlebars planner retry logic, improving iterative planning recovery.
›Restores FlowOrchestrator support for multi-step flow orchestration workflows.
+1 moreshow less
›Function Calling Planner now catches exceptions and outputs error messages into chat history for observability.
Semantic Kernel Python 0.4.5.dev adds an Ollama connector and debug logging for StepwisePlanner.
└──▷ GET THIS VERSION
$ git clone --branch python-0.4.5.dev https://github.com/microsoft/semantic-kernel.git
# already have the repo? check out this version:$ git checkout python-0.4.5.dev
›Adds Ollama connector, enabling local LLM inference via Ollama as a new backend for Python SK applications.
›Adds debug logging for StepwisePlanner's next-step thought, making planner reasoning observable at runtime.
Semantic Kernel for Python gains AIRequestSettings with three configuration methods for AI service request management.
└──▷ GET THIS VERSION
$ git clone --branch python-0.4.4.dev https://github.com/microsoft/semantic-kernel.git
# already have the repo? check out this version:$ git checkout python-0.4.4.dev
›Adds AIRequestSettings base class for storing settings across multiple services via a single extension_data field, with dynamic creation of service-specific request setting classes at call time.
›Adds kernel-based request settings generation that returns a pre-configured class with service_id and ai_model_id pre-filled based on the registered service.
›Adds richer exceptions when Azure OpenAI content filtering is triggered, surfacing filtering events as structured errors.
Jan v0.4.4 adds GPU detection, Linux AppImage support, Swagger docs, model-switching mid-thread, and a collapsible sidebar with hotkey.
└──▷ GET THIS VERSION
$ git clone --branch v0.4.4 https://github.com/janhq/jan.git
# already have the repo? check out this version:$ git checkout v0.4.4
└──▷ TRY IT
Explore and test Jan's local inference API interactively without writing any code.
$ open http://localhost:1337/docs
›Adds Swagger API documentation at localhost:1337/docs, giving practitioners a live, interactive reference for Jan's local API.
›Adds engine settings panel in the UI, exposing inference engine parameters directly to users.
›Adds a keyboard shortcut list in the Settings page so users can discover and reference all hotkeys.
›Makes left sidebar collapsible via hotkey, improving screen real-estate during active sessions.
›Enables switching models mid-thread without starting a new conversation.
+5 moreshow less
›Adds GPU detection for Windows and Linux, including CUDA version detection, so the app can surface hardware compatibility information.
›Adds Linux AppImage format support, broadening the supported distribution packaging options.
›Adds compatibility and recommendation labels (with color coding) to model cards in the Hub, based on total system RAM.
›Makes model.json optional for model loading, reducing friction when importing models without metadata.
›Deprecates the model.json ready state in favor of a .download file extension to track download status.
└──▷ BREAKING ON UPGRADE
!The model.json ready state is deprecated in favor of the .download file extension; existing tooling or scripts that rely on the ready-state field in model.json will need to be updated.
›New --usevulkan <gpu id> flag enables an early Vulkan GPU backend, now included in Windows and Linux prebuilt binaries (note: Mixtral on Vulkan not fully supported).
›Adds dynatemp_exponent parameter (previously hard-coded to 1.0), now configurable via API and in Kobold Lite.
›Adds XTTS API Server support in Kobold Lite for local AI-powered text-to-speech.
›Adds Old CPU fallback build targets (NoAVX2 and Failsafe modes) to the Linux prebuilt binary and koboldcpp.sh.
›Adds HD image generation options in Kobold Lite.
+3 moreshow less
›Adds popup-on-complete browser notification options in Kobold Lite.
›Adds option in Kobold Lite to let the AI impersonate the user for a turn in chat.
›Merges the new GGML backend rework from upstream, preserving support for earlier non-GGUF models via a fossilized earlier version of the library.
2 more releases in this issue
· 2024-01-01 → 2024-01-27
KoboldCpp v1.55.1 adds Dynamic Temperature sampling with dynatemp_range control and exposes latest seed in the perf endpoint.
└──▷ GET THIS VERSION
$ git clone --branch v1.55.1 https://github.com/LostRuins/koboldcpp.git
# already have the repo? check out this version:$ git checkout v1.55.1
└──▷ TRY IT
Use Dynamic Temperature to let the sampler vary between 0.3 and 0.5 per token, balancing creativity and coherence without manual tuning.
$ curl -s http://localhost:5001/api/v1/generate -H 'Content-Type: application/json' -d '{"prompt": "Once upon a time", "temperature": 0.4, "dynatemp_range": 0.1, "max_length": 200}'
Retrieve the most recently used seed from the perf endpoint to reproduce a specific generation.
$ curl -s http://localhost:5001/perf
›Adds dynatemp_range parameter enabling Dynamic Temperature (DynaTemp) sampling, where the actual temperature is automatically adjusted between temperature ± dynatemp_range at inference time (e.g., temperature=0.4 and dynatemp_range=0.1 yields a 0.3–0.5 range).
›Exposes the most recently used seed in the /perf endpoint, making it easier to reproduce or audit generation runs.
›Adds a min/max temperature UI in Lite for configuring DynaTemp directly, with both input styles auto-syncing to each other.
KoboldCpp v1.54 adds logit_bias support for OpenAI and Kobold APIs, custom background images in Lite, and GUI launcher tooltips.
└──▷ GET THIS VERSION
$ git clone --branch v1.54 https://github.com/LostRuins/koboldcpp.git
# already have the repo? check out this version:$ git checkout v1.54
└──▷ TRY IT
Bias specific token IDs up or down during generation to steer or suppress output tokens — works with both the Kobold and OpenAI-compatible endpoints.
$ curl -X POST http://localhost:5001/api/v1/generate -H 'Content-Type: application/json' -d '{"prompt": "Once upon a time", "logit_bias": {"1234": 2.0, "5678": -100.0}}'
›Adds logit_bias parameter to both the OpenAI and Kobold APIs, accepting a dictionary of token ID (int) to logit bias (float) pairs in the same object format as the official OpenAI implementation.
›Adds support for custom background images in KoboldCpp Lite.
›Adds customizable stepcount and cfgscale settings for Horde/A1111 image generation in Lite.
›Adds mouseover tooltips for all labels in the GUI launcher.
LocalAI v2.5.0 adds phi-2 and more embedded models, plus URL-based YAML model loading at startup.
└──▷ GET THIS VERSION
$ git clone --branch v2.5.0 https://github.com/mudler/LocalAI.git
# already have the repo? check out this version:$ git checkout v2.5.0
└──▷ TRY IT
Run a popular embedded model by short-hand name without any manual config — useful for quick local inference.
$ docker run -ti-p 8080:8080 localai/localai:v2.5.0-ffmpeg-core phi-2
Load a model at startup from a remote YAML config URL (e.g. a GitHub Gist), enabling community-shared model definitions without rebuilding your image.
$ docker run -ti-p 8080:8080 localai/localai:v2.5.0-ffmpeg-core https://raw.githubusercontent.com/mudler/LocalAI/master/embedded/models/llava.yaml
›Supports passing a URL pointing to a valid YAML model config file (e.g. a GitHub Gist) directly as a startup argument to load models like llava without pre-bundling them.
›Adds phi-2 and additional embedded models launchable by name as a direct CLI argument (e.g. phi-2) when starting LocalAI.
›Adds model usage and description metadata to embedded model definitions.
LocalAI v2.4.1 ships embedded model configurations with popular model examples ready to use out of the box.
└──▷ GET THIS VERSION
$ git clone --branch v2.4.1 https://github.com/mudler/LocalAI.git
# already have the repo? check out this version:$ git checkout v2.4.1
›Adds embedded model configurations bundled directly into LocalAI, including pre-built examples for popular models, removing the need to author config files from scratch.
SGLang v0.1.6 adds an OpenAI-compatible API server with streaming Completions and ChatCompletions endpoints.
└──▷ GET THIS VERSION
$ git clone --branch v0.1.6 https://github.com/sgl-project/sglang.git
# already have the repo? check out this version:$ git checkout v0.1.6
└──▷ TRY IT
Stream completions from a running SGLang server using the new OpenAI-compatible endpoint.
$ curl http://localhost:30000/v1/completions -H 'Content-Type: application/json' -d '{"model": "default", "prompt": "The capital of France is", "stream": true}'
Send a chat-style request to the new ChatCompletion endpoint for OpenAI-compatible clients.
Adds prompt_lookup_num_tokens, multimodal content arrays, RGBA image support, and trust_remote_code for DeepSpeed loading.
└──▷ GET THIS VERSION
$ git clone --branch snapshot-2024-01-28 https://github.com/oobabooga/textgen.git
# already have the repo? check out this version:$ git checkout snapshot-2024-01-28
›Adds prompt_lookup_num_tokens parameter for prompt lookup decoding.
›Adds trust_remote_code support when loading models with DeepSpeed.
›Updates n_gpu_layers default to 256 to support larger models.
›Supports content arrays in multimodal OpenAI API requests.
›Supports RGBA color format for image inputs.
3 more releases in this issue
· 2024-01-07 → 2024-01-28
Adds dynatemp parameters, past-chat sidebar, and Tab-key navigation between tabs.
└──▷ GET THIS VERSION
$ git clone --branch snapshot-2024-01-21 https://github.com/oobabooga/textgen.git
# already have the repo? check out this version:$ git checkout snapshot-2024-01-21
›Adds dynatemp_low, dynatemp_high, and dynatemp_exponent parameters for dynamic temperature control during generation.
›Adds a past chat histories sidebar on desktop for quick access to previous conversations.
›Adds Tab key shortcut to switch between the current tab and the Parameters tab.
Adds dynamic temperature parameters and a desktop chat history sidebar to oobabooga text-generation-webui.
└──▷ GET THIS VERSION
$ git clone --branch snapshot-2024-01-14 https://github.com/oobabooga/textgen.git
# already have the repo? check out this version:$ git checkout snapshot-2024-01-14
›Adds dynatemp_low, dynatemp_high, and dynatemp_exponent parameters for dynamic temperature control during text generation.
›New sidebar on desktop displays past chat histories for quick access.
›Press Tab to switch between the current tab and the Parameters tab in the UI.
Adds dynamic_temperature_low parameter and Dynamic Temperature support for the HF loader.
└──▷ GET THIS VERSION
$ git clone --branch snapshot-2024-01-07 https://github.com/oobabooga/textgen.git
# already have the repo? check out this version:$ git checkout snapshot-2024-01-07
›Adds dynamic_temperature_low parameter for finer control over dynamic temperature sampling ranges.
›Adds Dynamic Temperature sampling support for the HuggingFace (HF) loader.
vLLM v0.2.7 adds SSL to API servers, CUDA graph support for GPTQ/SqueezeLLM, and up to 70% distributed inference throughput gains via NCCL.
└──▷ GET THIS VERSION
$ git clone --branch v0.2.7 https://github.com/vllm-project/vllm.git
# already have the repo? check out this version:$ git checkout v0.2.7
›Adds SSL arguments to API servers, enabling TLS-secured inference endpoints.
›Enables CUDA graph capture for GPTQ and SqueezeLLM quantized models, accelerating inference for those quantization formats.
›Switches distributed control-plane communication from Ray to NCCL, removing serialization/deserialization overhead for up to 70% throughput improvement in distributed inference.
›Adds support for DeciLM-7B and DeciLM-7B-instruct model architectures.
›Adds support for GPT-NeoX models without attention biases.
Ollama v0.1.21 adds conversation save/load, MESSAGE Modelfile command, Python/JS libraries, and broader CPU support
└──▷ GET THIS VERSION
$ git clone --branch v0.1.21 https://github.com/ollama/ollama.git
# already have the repo? check out this version:$ git checkout v0.1.21
└──▷ TRY IT
Seed a model with Chain-Of-Thought examples at build time so every session starts with pre-loaded conversation history.
$ # Modelfile
FROM llama2
SYSTEM You are a helpful assistant.
MESSAGE user Is Toronto in Canada?
MESSAGE assistant yes
MESSAGE user Is Sacramento in Canada?
MESSAGE assistant no
# Then build and run:
ollama create -f Modelfile yesno
ollama run yesno
›Adds /save <model> and /load <model> commands inside ollama run to persist and restore conversations and model settings (including /set parameter and /set system changes) as a named model.
›Adds MESSAGE Modelfile command to pre-seed conversation history when building a model with ollama create, enabling techniques like Chain-Of-Thought prompting.
›Publishes first-release official Python (ollama-python) and JavaScript (ollama-js) client libraries for Ollama.
›Extends CPU support to processors without AVX instructions, enabling Ollama to run in virtual machines, Rosetta, and GitHub Actions environments.
›Delivers ~10% model inference speed boost on CPUs with AVX2 support.
+3 moreshow less
›Adds GPU-to-CPU automatic fallback when a GPU detection error is encountered at model load time.
›Adds four new models to the library: Qwen (1.8B–72B), DuckDB-NSQL (text-to-SQL for DuckDB), Stable Code, and Nous Hermes 2 Mixtral.
›Improves Nvidia GPU detection, especially under WSL.
Set a 32K context window interactively in an ollama run session before sending a long prompt.
$ ollama run mistral
/set parameter num_ctx 32678
›Sets context window size via num_ctx in /set parameter num_ctx (CLI) or the options.num_ctx field in the POST /api/generate JSON body — enabling up to 32K context with models like Mistral.
Triton v2.42.0 adds a Python in-process API, model-load retry, OpenTelemetry context propagation, pinned-memory metrics, and experimental PyTorch 2.0 serving.
└──▷ GET THIS VERSION
$ git clone --branch v2.42.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:$ git checkout v2.42.0
›Adds a command-line option to retry loading failed models a configurable number of attempts.
›Adds Triton Python API for in-process integration within a Python environment.
›Adds support for OpenTelemetry context propagation in trace mode.
›Adds pinned memory pool usage reporting to Triton metrics.
›Adds experimental support for serving PyTorch 2.0 models via the PyTorch backend.
+3 moreshow less
›Improves HTTP endpoint error responses so that status codes other than 400 may be returned to align with the actual error type.
›Model Analyzer now loads and optimizes ensemble models.
›Model Analyzer now supports optimizing a model on a remote Triton server without requiring a local GPU.
└──▷ BREAKING ON UPGRADE
!The FasterTransformer backend is deprecated as of 24.01 and is no longer supported or released with this and future versions of Triton.
Langfuse v2.0 rebuilds LLM cost tracking with custom model definitions, per-project pricing, and score filtering on generations.
└──▷ GET THIS VERSION
$ git clone --branch v2.0.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:$ git checkout v2.0.0
›Adds custom model/price definitions at the project level, enabling tracking of newly emerging models and price changes over time via the new model definition abstraction.
›Supports setting token usage and cost directly via the API when ingesting traces.
›Surfaces usage and cost information across all UI tables and APIs.
›Adds score-based filtering to the generations table in the UI.
›Adds an improved prompt UI with better versioning support.
└──▷ BREAKING ON UPGRADE
!Self-hosted deployments require a one-off (non-blocking) migration script on historical data to ensure accurate LLM costs; see https://langfuse.com/changelog/2024-01-29-custom-model-prices#upgrade-path for the upgrade path.
15 more releases in this issue
· 2024-01-02 → 2024-01-30
Langfuse v1.31.1 adds tokenization support for Azure-style gpt-35* model names.
└──▷ GET THIS VERSION
$ git clone --branch v1.31.1 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:$ git checkout v1.31.1
›Tokenization now recognizes gpt-35* as an alternative model name alias for gpt-3.5*, enabling accurate token counting for Azure OpenAI deployments that use the hyphen-less naming convention.
›Adds exist_ok option to create_table in the Python SDK, allowing idempotent table creation without raising an error if the table already exists.
›Adds Gemini text embedding function to the Python embedding API, joining existing OpenAI embeddings support.
›Adds basic Polars integration for the Python SDK, including support for ingesting Polars DataFrames and converting an entire table to a Polars DataFrame.
›Supports passing the API key as an environment variable, in addition to explicit parameter passing.
›Updates the Node.js SDK to support OpenAI SDK version ^4.24.1 embeddings API.
+5 moreshow less
›Reworks the Node.js SDK using napi for improved native performance and compatibility.
›Adds a new createIndex API in the napi-based Node.js SDK.
›Improves the Rust create index API and table query API.
›Adds a helper function in the JavaScript SDK to create an Arrow Table with a schema.
›Changes create_table to accept an Arrow Table directly as input.
Milvus 2.3.4 adds access logs, Parquet bulk import, and binlog indexes on growing segments for faster search.
└──▷ GET THIS VERSION
$ git clone --branch v2.3.4 https://github.com/milvus-io/milvus.git
# already have the repo? check out this version:$ git checkout v2.3.4
›Adds access logs for monitoring external gRPC interfaces, recording method names, user requests, response times, and error codes.
›Adds Parquet file import support for bulk ingestion, including arrays and JSON data types, superseding the prior JSON and NumPy-only limitation.
›Introduces binlog index on growing segments, enabling advanced index types (IVF, Fast Scann) and up to 10x faster searches on growing segments.
›Expands cluster support to 10,000 collections/partitions, benefiting multi-tenant environments via timetick mechanism and goroutine management improvements.
›Adds MMap support for index loading.
+2 moreshow less
›Adds partition-level privileges.
›Implements balance channel in querycoord for improved query shard management.
└──▷ BREAKING ON UPGRADE
!Regular expression searches in partitions are discontinued by default to reduce resource consumption; the feature can be re-enabled via configuration.
THE AI TOOLCHAIN