This Week's Highlights — issue 002, August 28, 2026

Giskard v3 is a full rewrite structured as a modular monorepo with three installable packages — giskard-checks, giskard-agents, and giskard-core — plus giskard-scan (via pip install 'giskard[scan]') and a new provider-agnostic LLM routing layer giskard-llm replacing litellm, with provider extras such as pip install 'giskard[openai]' and pip install 'giskard[anthropic]'. It requires Python 3.12+, drops v2 APIs (giskard.Model, giskard.Dataset, giskard.testing, Giskard Hub — retained via pip install 'giskard[llm]>2,<3'), removes Scenario.from_sequence and the scenario() factory in favor of the step-based Scenario constructor, makes the Interact injection name-based instead of positional, makes Jinja parsing in Workflow.chat opt-in, drops templating support in conformity rules, and adds opt-out telemetry via DO_NOT_TRACK=1 or GISKARD_TELEMETRY_DISABLED=1.

giskard.scan.vulnerability_scan automates red-teaming of agents across OWASP LLM Top-10 threat categories (prompt injection, harmful content, stereotypes, misinformation), taking target, description, and languages parameters; a minimal OWASP LLM suite generator covers LLM01 indirect injection. generate_suite accepts custom ScenarioGenerator instances via a vulnerability_suite_generator_registry, which now also exposes default registry generators on the public API and integrates third-party ScenarioGenerator instances into the scan pipeline. The built-in prompt-injection dataset was expanded with additional scenarios and templates for broader adversarial coverage.

import asyncio
from giskard.scan import vulnerability_scan

async def my_agent(inputs: str) -> str:
    return f"Echo: {inputs}"  # replace with your real agent

async def main() -> None:
    await vulnerability_scan(
        target=my_agent,
        description="A customer support chatbot for an e-commerce platform.",
        languages=["en"],
    )

asyncio.run(main())

import asyncio
from giskard.scan import vulnerability_scan

async def my_agent(inputs: str) -> str:
    return f"Echo: {inputs}"  # replace with your real agent

async def main() -> None:
    await vulnerability_scan(
        target=my_agent,
        description="A customer support chatbot for an e-commerce platform.",
        languages=["en"],
    )

asyncio.run(main())

giskard-checks ships built-in LLM-as-judge checks Groundedness, Conformity, and LLMJudge (default model openai/gpt-4o-mini), later joined by AnswerRelevance and Toxicity; judge results now enforce non-blank reasons so every verdict carries an explanatory rationale. set_default_generator also accepts plain model name strings (e.g. 'openai/gpt-4o-mini') in addition to generator objects, reducing boilerplate when configuring the default judge/generator.

from giskard.checks import set_default_generator

set_default_generator('openai/gpt-4o-mini')

import asyncio
from giskard.checks import Scenario, Groundedness

def get_answer(inputs: str) -> str:
    return my_agent(inputs)  # replace with your model/agent

async def main() -> None:
    scenario = (
        Scenario("test_capital")
        .interact(inputs="What is the capital of France?", outputs=get_answer)
        .check(
            Groundedness(
                name="answer is grounded",
                context="France is in Western Europe. Its capital is Paris.",
            )
        )
    )
    result = await scenario.run()
    result.print_report()

asyncio.run(main())

import asyncio
from giskard.checks import Scenario, Groundedness

def get_answer(inputs: str) -> str:
    return "Paris"  # replace with your RAG pipeline

async def main() -> None:
    scenario = (
        Scenario("test_capital_grounded")
        .interact(inputs="What is the capital of France?", outputs=get_answer)
        .check(
            Groundedness(
                name="answer is grounded",
                context="France is in Western Europe. Its capital is Paris.",
            )
        )
    )
    result = await scenario.run()
    result.print_report()

asyncio.run(main())

The Scenario API in giskard.checks composes multi-turn eval interactions via chained .interact() and .check() calls with an async .run() entrypoint and .print_report() for human-readable output; scenarios support annotations for attaching metadata to eval steps and can run multiple times within a suite. Suite runs batches of scenarios with dynamic binding and a chainable Suite.append(), and suite reports now include scenario/check error details plus JUnit XML export for SuiteResult, enabling CI integration.

aig-agent-redteam v5.0.0 introduces a refactored mutation engine; the former workflow-attack command is merged into mutation-attack, so any automation or invocation referencing workflow-attack must be updated to use mutation-attack.

Repackages Skill-Scan as a standalone PyPI package aig-skill-scan for CI/CD integration, adding Agent Skill security auditing across 9 risk categories and achieving a SkillTrustBench top score of 0.9848.

v4.5.0 added 4 new detection skills to Agent-Scan; v4.5.1 added 5 OWASP-based skills (agentic-supply-chain, cascading-failure, human-agent-trust, inter-agent-comm, unexpected-code-execution) plus web-exfiltration-detection, bringing Agent-Scan to 10 total detection skills.

The AIG rules library expanded to cover 130 AI components with 1888 rules and new AI component fingerprints (v4.5.0), grew further to 2000+ CVE rules (v4.5.2), and was refreshed with the 2026-07-24 rule set (v4.5.1).

v4.5.0 added 2 new MCP security detection rules plus ATR-derived rules covering additional attack surfaces; v4.5.1 added 4 more MCP detection rules covering hardcoded secrets and insecure deserialization.

Adds Many-Shot, PAIR, GOAT, and ActorAttack multi-turn jailbreak attack methods to PromptSecurity evaluation.

New --transitive flag opts into scanning skills referenced by the target skill, with --transitive-depth, --transitive-allow-prefix, and --transitive-deny-prefix to bound traversal depth and restrict which sources are followed, catching supply-chain risk in referenced dependencies.

$ skillspector scan ./my-skill/ --transitive --transitive-depth 2 --transitive-allow-prefix https://github.com/trusted-org/

Adds canonical inspection-ledger accounting across static and LLM analysis stages, with per-component coverage and explicit out-of-scope records in JSON and SARIF output. New execution_successful and analysis_completeness.ledger_exceptions fields let automation distinguish a complete scan from a partial or failed one; the CLI now exits with code 2 for a fatal execution or accounting failure even when a JSON report was produced, and recursive scans propagate child scan failures into the combined report with a non-zero exit when any child fails. JSON integrations must treat invalid or missing output, a nonzero exit, or execution_successful: false as a blocking validation error.

$ skillspector scan ./my-skill/ --format json --output report.json; python3 -c "import json,sys; r=json.load(open('report.json')); sys.exit(1 if not r.get('execution_successful') else 0)"

New SKILLSPECTOR_TEMPERATURE (values 0–1) and SKILLSPECTOR_SEED (integer) environment variables let you pin LLM sampling for reproducible semantic analysis; both are forwarded to OpenAI-compatible and Azure OpenAI endpoints and left unset to preserve provider defaults.

$ SKILLSPECTOR_PROVIDER=openai OPENAI_API_KEY="$OPENAI_API_KEY" SKILLSPECTOR_TEMPERATURE=0 SKILLSPECTOR_SEED=42 skillspector scan ./my-skill/

skillspector baseline <path> now produces version 2 fingerprints that bind accepted findings to the scanner version, source content, and full finding evidence. Baseline files containing version 1 fingerprints are rejected on upgrade — run skillspector baseline <path>, review the generated version 2 entries, and commit the replacement; rules-only version 1 baselines remain supported with a warning.

$ skillspector baseline ./my-skill/ -o .skillspector-baseline.yaml

New BH1, BH2, and BH3 findings detect bundled lifecycle hook execution (hooks/hooks.json), directly proven remote transfer of sensitive event or file content, and broad or ignored project permission surfaces (.claude/settings.json, .claude/settings.local.json). Existing scans may start surfacing these findings, so review them before accepting into a baseline.

New risk_assessment.max_issue_severity field in JSON/SARIF output reports NONE when no active issue exists, giving downstream automation a single field to gate CI pipelines on.

$ SKILLSPECTOR_OUTPUT_LANGUAGE=French skillspector scan ./my-skill/ --format json --output report.json && jq '.risk_assessment.max_issue_severity' report.json

langgraph-cli[inmem] is no longer included in the base installation; LangGraph Studio users who install only the base package must now install skillspector[langgraph-dev] explicitly.

Adds opt-in automatic discovery of a top-level .skillspector-baseline.yaml file so CI can suppress known findings without an explicit flag; an explicitly supplied baseline via --baseline remains authoritative.

$ skillspector baseline ./my-skill/ -o .skillspector-baseline.yaml
# commit .skillspector-baseline.yaml to the repo root, then in CI:
skillspector scan ./my-skill/

Enforces INGEST_MAX_BYTES (100 MiB per-ingest cap) and INGEST_MAX_ZIP_MEMBERS (10,000 entries) for streamed URL downloads, zip archives, and Git repository clones, failing closed with IngestLimitExceededError on breach.

Adds Ollama support for local OpenAI-compatible inference, Azure OpenAI deployment routing, and a configurable provider for other OpenAI-compatible endpoints, all selected via SKILLSPECTOR_PROVIDER. skillspector scan --help now lists every supported hosted, local, compatible, and CLI-backed provider together with its authentication path.

Adds bounded local inspection of hidden files and ZIP-compatible nested artifacts (ZIP, DOCX, XLSX, PPTX) without extracting or executing members, raising HIGH-severity SC9 findings for concealed executables.

Scans can now safely traverse intermediate path components using O_PATH on Linux, allowing scans in restricted sandboxes where ancestor directories lack read permission, while preserving final-file and no-symlink protections.

HTTP MCP clients can no longer scan local filesystem paths or supply local YARA-rule directories; use a remote repository or URL for HTTP requests, or use stdio transport for local scans.

Adds MCP registry posture scanning via skillspector mcp, available by installing the mcp extra: skillspector[mcp].

New SKILLSPECTOR_OUTPUT_LANGUAGE environment variable sets the language of human-readable LLM-generated finding text across discovery analyzers, the meta-analyzer, and MCP tool-poisoning analysis.

$ SKILLSPECTOR_OUTPUT_LANGUAGE=French skillspector scan ./my-skill/ --format json --output report.json && jq '.risk_assessment.max_issue_severity' report.json

Adds a promptfoo logs command for viewing log files directly from the CLI; a -d/--description flag on redteam generate commands for labeling generated scans; --share and --no-share flags on Model Audit for controlling cloud sharing; $VAR syntax support in file paths for referencing environment variables in configuration; support for multiple --env-file flags to load several environment files in a single invocation; and an --extension CLI flag to load extensions via command line.

$ promptfoo logs

$ promptfoo redteam generate -d 'nightly-telecom-scan' --config redteam.yaml

$ promptfoo eval --env-file .env --env-file .env.secrets -c promptfooconfig.yaml

Adds a numTests config key to cap the number of test cases generated per red team strategy; metadata.sessionId as a surfaced variable column in eval result tables and exports; a word-count assertion type for validating response word counts; a __count variable for use in derived metrics to compute averages; and test-level prompts filter, providers filter, and per-test structured output configuration at the individual test case level.

strategies:
  - id: jailbreak
    numTests: 20
assertions:
  - type: word-count
    threshold: 200

Adds apiBaseUrl and websocketUrl override config and function call support to the xAI Voice provider.

thv skill sync restores a project's pinned skill set and verifies on-disk content in CI, and thv skill upgrade re-resolves pinned skills to newer content without silent drift; both pin installs in toolhive.lock.yaml (including Sigstore provenance fields) with typed exit codes and a pre-install confirmation gate. Sigstore signature verification was added for skills at install, sync, and upgrade time. Signing of pushes was enabled by default (the lock feature gate removed), and lock provenance now records certificate ref and runner, enforced during skill verification. The --clients flag on thv skill sync and a {"clients": [...]} body field on POST /api/v1beta/skills/sync let operators scope which skill-supporting clients are targeted; qoder was added as the 18th skill-supporting client, materializing skills into <project>/.qoder/skills/. thv skill push now signs keylessly by default and requires exactly one of --key, --identity-token, or --no-sign.

$ thv skill sync

$ thv skill upgrade

$ thv skill sync --check --clients claude-code

$ curl -X POST http://127.0.0.1:8080/api/v1beta/skills/sync \
  -H 'Content-Type: application/json' \
  -d '{"clients": ["claude-code", "cursor"]}'

thv ai-plugin ships a full CLI and REST API for end-to-end AI-tool plugin management, alongside a registry catalog for discovering and managing plugins. A plugins key was added to the lock file schema to track plugin entries, backed by a new PluginLockService and a managed install flag, with lock provenance recording certificate ref and runner. Plugin artifacts gained end-to-end Sigstore bundle verification; stored bundles and git commit payloads/signatures are rejected with HTTP 422 above 1 MiB. thv ai-plugin upgrade gained an --allow-signer-change flag so operators can explicitly confirm a signer rotation — without it, upgrades whose signature identity differs from the lock file, or that are unsigned, are blocked with exit code 4 and signer-change-blocked, gated behind the experimental TOOLHIVE_PLUGINS_LOCK_ENABLED environment variable.

$ thv ai-plugin upgrade --allow-signer-change <plugin-name>

Adds aggregation.defaultToolVisibility: deny to vMCP config so only workloads explicitly listed in aggregation.tools have their tools advertised, closing the fail-open gap in tool aggregation; on the Modern (2026-07-28) path, tools excluded via filter, excludeAll, or excludeAllTools are no longer directly callable — tools/call now returns -32602 at HTTP 400 instead of executing. Composite tools now support MCP tool annotations (readOnlyHint, destructiveHint, idempotentHint, openWorldHint), with a conservative fail-closed safety floor derived from the workflow's step tools when none are set explicitly.

name: "engineering-vmcp"
groupRef: "engineering-team"
aggregation:
  defaultToolVisibility: deny
  tools:
    - workload: github
    - workload: jira

A caBundleRef field was added to OIDC and OAuth2 upstream specs, letting an embedded auth server trust a private CA for discovery, token, user-info, and dynamic client registration calls to that upstream only; the operator-crds chart must be upgraded to 0.46.0 before or with the operator chart, or a stale CRD silently prunes the field. An insecureAllowHTTP: true field under spec.inline in MCPOIDCConfig lets operators opt in to plain-HTTP issuer and JWKS URLs for dev/test; production configs must use HTTPS, and inline configs with a plain-HTTP, malformed, or scheme-less issuer/jwksUrl now flip to Valid=False on reconcile, blocking dependent MCPServer, MCPRemoteProxy, and VirtualMCPServer resources.

caBundleRef: my-internal-ca-secret

apiVersion: toolhive.stacklok.dev/v1beta1
kind: MCPOIDCConfig
metadata:
  name: keycloak-auth
spec:
  type: inline
  inline:
    issuer: http://keycloak:8080/realms/toolhive
    jwksUrl: http://keycloak:8080/realms/toolhive/protocol/openid-connect/certs
    insecureAllowHTTP: true

The embedded authorization server's token endpoint now supports RFC 8693 token exchange, with delegated token audience bounded by the subject token; subject tokens from trusted external OIDC issuers (Keycloak, Entra, Okta) can be validated for exchange, and audit logs capture the RFC 8693 act claim and full delegation chain. trusted_issuers was added to the embedded auth server config so agents can exchange subject tokens from external OIDC providers for ToolHive-scoped delegated tokens under a fail-closed RFC 8693 consent policy. actor_token support was added for delegated-identity scenarios where an acting party is distinct from the subject. RFC 8693 delegate (token-exchange) clients configured in vmcp-config.yaml are now reachable and usable for downstream token exchange workflows.

backends:
  github:
    type: token_exchange
    tokenExchange:
      tokenUrl: "https://keycloak.example.com/realms/myrealm/protocol/openid-connect/token"
      clientId: "vmcp-github-exchange"
      clientSecretEnv: "GITHUB_EXCHANGE_SECRET"
      audience: "github-api"
      scopes: ["repo", "read:org"]

status.referencingWorkloads, status.referenceCount, and the References printer column are removed from all six config CRDs (MCPOIDCConfig, MCPAuthzConfig, MCPExternalAuthConfig, MCPToolConfig, MCPWebhookConfig, MCPTelemetryConfig); replace any automation reading them with workload field queries via -o json | jq.

The StorageVersionMigrator controller is now enabled by default (operator.features.storageVersionMigrator: true); namespace-scoped Helm installs (operator.rbac.scope=namespace) now fail helm upgrade at render time unless operator.features.storageVersionMigrator: false is explicitly set.

operator:
  rbac:
    scope: namespace
  features:
    storageVersionMigrator: false

Supports the MCP 2026-07-28 stateless ("Modern") spec revision end-to-end across transport proxies, transparent proxy, and Virtual MCP, bridging era-mismatched client×backend combinations; Modern client-facing dispatch is gated per capability instead of a global kill-switch, with listen-stream support and pagination. Adds opt-in strict MCP-Protocol-Version header validation for the streamable proxy; the readiness probe now sends the current MCP protocol version instead of a hardcoded 2024-11-05; guarantees tools/list pagination completeness for aggregated sets exceeding 1,000 tools; tool definitions carrying invalid x-mcp-header annotations (SEP-2243) are now rejected; W3C trace context propagates through outbound MCP _meta (SEP-414); backend list_changed notifications are consumed and propagated to clients for tools, resources, and prompts; multi-line Modern SSE events can now be parsed for better transport compatibility.

Package names in npx://, uvx://, and go:// references are now validated against [A-Za-z0-9@/:._+=~[\]-] at build time, blocking shell metacharacter injection into generated Dockerfiles; names outside this pattern now fail at build time with an "invalid package name" error instead of being interpolated into the Dockerfile.

thv serve management API now enforces Content-Type: application/json on state-changing requests over TCP, and adds Origin validation with a loopback-only allowlist on those same listeners; callers omitting the header receive 415 Unsupported Media Type.

$ curl -X POST http://127.0.0.1:8080/api/v1beta/workloads \
  -H 'Content-Type: application/json' \
  -d '{"name":"fetch","image":"ghcr.io/example/fetch:latest"}'

runtime_config.build_with on npx:///go:// images is now a 400 Bad Request, and runtime_config.runtime_env is now actually applied via POST /api/v1beta/workloads (previously silently discarded).

Adds TOOLHIVE_API_TIMEOUT environment variable to override the CLI API client timeout for thv skill and thv ai-plugin commands (default is 10 minutes).

$ TOOLHIVE_API_TIMEOUT=30s thv skill list