◆ AI OBSERVABILITY

PromptLayer

snapshot-20260821 commercial

PromptLayer is a platform that logs, manages, and analyzes LLM prompts and API calls for debugging and optimization.

source ↗ rss ↗

Summary

PromptLayer is a platform that logs, manages, and analyzes LLM prompts and API calls for debugging and optimization.

Release history

snapshot-20260821 seen Aug 21, 2026 · issue 003

PromptLayer adds trace I/O previews and a redesigned tabbed span details panel alongside Gemini 3.7 Flash support.
- ›Adds trace input/output previews at the top level of the trace detail view, automatically extracted from the first user message and final assistant response, visible above the span tree — especially useful for multi-step traces with tool calls.
- ›Redesigns the span details panel with a tabbed interface separating metadata, inputs/outputs, and entity references, plus hover cards for quick span previews without opening the full panel.
- ›Adds support for the Gemini 3.7 Flash model with updated pricing and reasoning capabilities.
- ›Highlights trace entities (prompts, workflows, evaluations) in the trace view for better context.
1.0.0 Aug 20, 2026 · issue 002

PromptLayer now publishes an API — 143 endpoints across 22 areas: External Ids, Tables, Smart Tables, …
- ›External Ids (22 endpoints) — create, read, update, delete
- ›Tables (22 endpoints) — create, read, update, delete
- ›Smart Tables (21 endpoints) — create, read, update, delete
- ›Reports (11 endpoints) — create, read, update, delete
- ›Datasets (9 endpoints) — create, read
+4 moreshow less
- ›Env Vars (8 endpoints) — create, read, update, delete
- ›Tracking (8 endpoints) — create, read
- ›Prompt Templates (7 endpoints) — create, read, update, delete
- ›14 more areas: Workflow, Folders, Skill Collections, Tool Registry, Evaluations, Rest, Group, Metadata, Otlp, Prompt, Prompts, Request, Score, Spans
snapshot-20260820 seen Aug 20, 2026 · issue 002

PromptLayer adds real-time eval progress tracking, granular scorecard status, and centralized MCP server management for workspaces.
- ›Adds MCP Servers page in workspace settings to centrally connect, configure, test, and manage external MCP servers — configure once per workspace and reference tools across multiple prompt templates.
- ›Supports automatic tool discovery and import from connected MCP servers, with connectivity testing before deployment and credential management via workspace environment variables.
- ›Adds real-time evaluation progress tracking via WebSocket delivery: SDK can now publish planned row counts before writing data, and the dashboard displays 'X of Y rows complete' with explicit lifecycle states (running, completed, aborted).
- ›Adds granular scorecard status reporting with per-row verdict-based counters (passed, failed, warned, running, queued) instead of simple completion percentages, calculated from database truth rather than Redis approximations.
August 20, 2026 Aug 20, 2026 · issue 005

PromptLayer adds trace input/output previews and a redesigned tabbed span details panel for faster trace inspection.
└──▷ HOW TO FIND IT
Quickly grasp what a complex multi-step agent trace accomplished without drilling into every span — open any trace and read the input/output preview at the top of the trace summary section.
📍In the PromptLayer UI, open a trace in the trace detail view. The initial user input and final assistant response are now displayed in the trace summary section above the span tree.
Inspect span metadata, inputs/outputs, and entity references side-by-side without losing context — use the new tabbed panel and hover cards to review span details inline.
📍In the PromptLayer UI, click any span in the span tree to open the span details panel. Use the tabs to switch between metadata, inputs/outputs, and entity references. Hover over a span to preview details without fully opening the panel.
- ›Adds trace input/output previews to the trace detail view, automatically extracting the first user message and final assistant response and displaying them above the span tree — especially useful for multi-step traces with tool calls.
August 18, 2026 Aug 18, 2026 · issue 005

PromptLayer adds real-time evaluation progress tracking with WebSocket delivery and granular per-row scorecard status breakdowns.
- ›SDK can now publish planned row counts before writing data, enabling 'X of Y rows complete' live progress display in the dashboard during evaluations.
- ›WebSocket delivery pushes evaluation progress updates to the UI immediately, eliminating the need for polling.
- ›Scorecard progress now provides granular per-row verdict-based counters — passed, failed, warned, running, and queued — beyond simple completion percentages.
- ›Scorecard progress calculations now use database truth instead of Redis approximations for accuracy.
- ›Partial scores remain visible in scorecards during long-running evaluations.
August 12, 2026 Aug 12, 2026 · issue 005

PromptLayer adds centralized MCP Server Management and Smart Tables column import/conversion capabilities.
└──▷ HOW TO FIND IT
Configure a new MCP server workspace-wide so its tools are available in all prompt templates.
📍In PromptLayer, go to Workspace Settings › MCP Servers, click 'Add Server', enter the server URL and credentials (referencing workspace environment variables), then click 'Test Connection' to discover and verify available tools before saving.
Import an existing request log into a Smart Table anchor column to enrich analysis with nested execution metadata.
📍In a Smart Table, locate a Request column cell, click 'Import', select the existing request log by its source ID, and the table will auto-populate nested columns for latency, cost, model, and tokens.
- ›Adds an MCP Servers page in workspace settings to centrally configure, test, and manage external Model Context Protocol servers, with credentials stored in workspace environment variables.
- ›Enables MCP servers configured once per workspace to be referenced across multiple prompt templates, with automatic tool discovery and import from connected servers.
- ›Adds 'Import' action to Request and Trace column cells in Smart Tables to link existing request logs or traces as anchor columns, auto-populating nested metadata (latency, cost, model, tokens).
- ›Adds column conversion in Smart Tables: right-click a column header to convert Prompt Template columns to Request columns, or Workflow columns to Trace columns, with downstream columns auto-recalculating.
August 10, 2026 Aug 10, 2026 · issue 005

PromptLayer Playground gains OpenRouter audio, image generation, and TTS model support with streaming playback
└──▷ HOW TO FIND IT
Try an OpenRouter audio model in the Playground to generate and immediately play back a spoken response in the browser.
📍In the Playground, go to Model Selection › choose an OpenRouter audio-capable model (e.g., GPT-4 Audio) › open the Model Parameters panel › select a voice option from the pre-populated voice list › run your prompt › use the playback controls to stream the audio response.
Generate a vector SVG image via an OpenRouter image generation model (e.g., Recraft) and download the result directly from the Playground.
📍In the Playground, go to Model Selection › choose an OpenRouter image generation model (e.g., Recraft) › configure aspect ratio, quality, and output format (SVG will surface automatically for vector-capable models) › run your prompt › use the inline preview to inspect then download the output.
- ›Adds OpenRouter audio-capable chat model support (e.g., GPT-4 Audio) in the Playground with real-time streaming voice output, pre-configured voice options in the model parameters panel, and browser-playable playback controls.
- ›Adds OpenRouter image generation models (including vector-capable models like Recraft) to the Playground with configurable aspect ratio, quality, and output format parameters; SVG output models automatically surface appropriate format options, and images display inline with download and preview.
- ›Adds OpenRouter text-to-speech model support (MiniMax Speech, Deepgram, MAI Voice) in the Playground with model-specific voice selection, automatic response format adaptation (MP3/PCM) per provider, and audio speed control where the underlying model supports it.
- ›Model parameter controls now intelligently show or hide based on actual API support for the selected model.
- ›Playground streaming toggle automatically locks when streaming is required or disallowed by the selected model.
August 05, 2026 Aug 5, 2026 · issue 005

PromptLayer adds AI-generated trace summaries and empty tool trajectory validation for agent flow analysis.
└──▷ HOW TO FIND IT
Validate that a fallback workflow made zero tool calls by asserting an empty tool trajectory in a trajectory scenario.
📍In the trajectory evaluator, open your scenario and set the expected tool list to an empty array [] to assert that no tools were called during execution.
- ›Adds AI-generated natural language summaries on the trace detail page, automatically describing what each trace accomplishes by analyzing spans and tool calls, with totals for tool calls, LLM requests, tokens, and cost at a glance.
- ›Summaries regenerate automatically as traces grow to stay accurate.
- ›Trajectory evaluators now support empty tool lists in trajectory scenarios to assert a workflow made zero tool calls, enabling validation of fallback paths and flows that must avoid external API calls.
August 03, 2026 Aug 3, 2026 · issue 005

PromptLayer adds OpenRouter integration, enabling access to hundreds of models across OpenAI, Anthropic, Google, and Meta via a single API key.
└──▷ HOW TO FIND IT
When you want to route LLM requests through OpenRouter to access hundreds of models without managing separate provider credentials.
📍In the PromptLayer dashboard, go to your provider settings and add OpenRouter as a provider using your OpenRouter API key. Then select any OpenRouter-hosted model (e.g. from Anthropic, Google, or Meta) in the model selector when creating or running a prompt template.
- ›Integrates with OpenRouter as a new provider, routing requests across OpenAI, Anthropic, Google, Meta, and other AI labs through a single API key with unified cost and usage tracking in the PromptLayer dashboard.
- ›Supports specialized OpenRouter model types including chat, image generation, video generation, and text-to-speech through the integration.
- ›Enhances the model selector UI to display available API types per provider.
July 30, 2026 Jul 30, 2026 · issue 005

PromptLayer adds Request Volume Tail Analytics with interactive time-series charts filterable by template, invocation site, and custom signals.
└──▷ HOW TO FIND IT
Identify which templates or invocation sites are driving unexpected request spikes over a given time window.
📍In PromptLayer, go to Analytics › Request Volume Tail Analytics, select your time range, then apply a filter by 'Template Name' or 'Invocation Site' to drill into the breakdown.
- ›New Request Volume Tail Analytics view shows request volume trends over time with interactive time-series charts, breakdowns by template name, invocation site, or custom signals, and historical request pattern analysis.
July 29, 2026 Jul 29, 2026 · issue 005

PromptLayer adds Signals for automated traffic labeling, Wrangler scorecard support, and workflow input variables in analytics.
└──▷ HOW TO FIND IT
Filter production request logs to only those where a Signal flagged a result, to triage misbehaving traffic without leaving the logs UI.
📍In the PromptLayer UI, go to Request Logs, open the Signals filter panel, select your Signal, and choose 'Yes' or 'No' to scope the log view to matching results.
Inspect workflow execution inputs inline when debugging a failed workflow run, without opening each execution individually.
📍In the PromptLayer UI, navigate to Workflows › select your workflow › Analytics, locate the 'Input Variables' column in the version analytics table, and click any cell to expand the full variable values in the popover.
- ›Introduces Signals for automated production traffic labeling: evaluate requests with yes/no questions, with reasoning and quote support per evaluation.
- ›Adds ability to test Signals against real logs before enabling them in production.
- ›Adds filtering of request logs by Signal results directly from the UI.
- ›Adds per-hook sampling rate configuration to control Signal evaluation frequency.
- ›Adds a new 'Input Variables' column to the workflow version analytics table, showing all input parameters passed to each workflow execution.
+6 moreshow less
- ›Adds click-to-expand popover in the analytics table to view full workflow input variable values without leaving the table.
- ›Wrangler can now create, retrieve, and recalculate scorecards via natural language commands, including multi-criterion scorecards with STRUCTURE and ASSERT_VALID validators.
- ›Adds JSON Schema validation criteria support alongside structured output prompts in scorecard configuration.
- ›Supports triggering scorecard recalculations for specific rows or evaluation steps programmatically through Wrangler.
- ›Signal bindings can now be individually updated or deleted without replacing the entire configuration.
- ›Supports multiple bindings for the same prompt template with different version or label filters.
July 28, 2026 Jul 28, 2026 · issue 005

PromptLayer adds full trace visibility for Playground/Smart Table runs, Evaluation Drafts, and custom Signal Annotations
└──▷ HOW TO FIND IT
Iterate on a Playground evaluation without affecting your workspace — save as a draft, tune prompt configs and row filters, then publish when ready.
📍In the Playground, run your evaluation and select 'Save as Draft Table'. Map Playground variables to Table columns, configure row filters, then choose 'Export as Permanent Table' and optionally link to a prompt version in your registry.
Tag and filter request logs automatically by defining a custom signal with scope rules, then verify it against historical logs before enabling it in production.
📍In the dashboard, go to Signal Annotations, define a new signal with a custom name, color, and scope rules, then use 'Test against historical request logs' to validate matches before deploying. Filter request logs by signal presence using the signal-based filter in structured search.
- ›Adds Playground Evaluation Drafts, letting users save evaluations as draft Tables before publishing — supports mapping Playground variables to Table columns, configuring row filters, and exporting drafts as permanent Tables optionally linked to prompt versions in the registry.
- ›Adds Signal Annotations to annotate request logs with custom signals (custom names, colors, and scope rules) that automatically apply to matching requests and appear as visual indicators throughout the dashboard.
- ›Enables filtering and searching requests by signal presence in the request logs view.
- ›Supports testing signals against historical request logs before deployment to verify matching behavior.
- ›All Playground runs and Smart Table cell executions now generate complete traces in the Traces view, including per-run traces for batch/completion runs and conversation-level traces for chat sessions.
+2 moreshow less
- ›Smart Table prompt cells now show full execution traces including LLM calls and tool invocations.
- ›Raises the limit on input sets in draft evaluations to up to 50 input sets.
July 27, 2026 Jul 27, 2026 · issue 005

PromptLayer adds custom date range filtering to workflow version and prompt template analytics.
- ›Workflow version analytics now support custom date ranges, enabling comparison of latency, cost, and execution count across specific time periods.
- ›Prompt template analytics gain date range filtering for granular performance insights, including the ability to export analytics data for specific time windows.