https://aitoolchain.io/tools/giskard New releases and features in Giskard, tracked by The AI Toolchain. en Wed, 26 Aug 2026 08:39:15 GMT https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.3 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.3 Wed, 26 Aug 2026 08:39:15 GMT giskard-checks v1.0.3 lets `set_default_generator` accept model name strings in addition to generator objects. • Extends `set_default_generator` to accept plain model name strings (e.g. `'openai/gpt-4o'`) in addition to generator objects, reducing boilerplate when configuring the default LLM judge or generator. https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.3 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.3 Wed, 26 Aug 2026 08:39:15 GMT giskard-checks v1.0.3 lets `set_default_generator` accept model name strings in addition to generator objects. • Extends `set_default_generator` to accept plain model name strings (e.g. `'openai/gpt-4o'`) in addition to generator objects, reducing boilerplate when configuring the default LLM judge or generator. https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0 Wed, 26 Aug 2026 01:15:51 GMT Giskard v3 rewrites as a modular monorepo with new LLM judge checks, OWASP scanning, Suite batching, regex/composition operators, and JUnit XML export. • Rewrites Giskard as a v3 monorepo structured around three installable packages: `giskard-checks`, `giskard-agents`, and `giskard-core`, with `giskard-scan` available via `pip install 'giskard[scan]'`. • Adds opt-out telemetry via `DO_NOT_TRACK=1` or `GISKARD_TELEMETRY_DISABLED=1` environment variables, collected through optional PostHog analytics in `giskard-core`. • Adds `RegexMatching` check to `giskard-checks` for asserting outputs match a regular expression, with ReDoS mitigation via regex timeout. • Adds `AllOf`, `AnyOf`, and `Not` check composition operators to `giskard-checks` for combining existing checks with boolean logic. • Adds `AnswerRelevance` LLM judge check to `giskard-checks`. • Adds `Toxicity` LLM judge check to `giskard-checks`. • Adds `annotations` support to `Scenario`/trace objects for attaching metadata to eval steps. • Introduces `Suite` for running batches of scenarios with dynamic binding; Suite.append() is chainable. • Allows scenarios to run multiple times within a suite. • Adds print_report() to result models for human-readable check output. • Adds scenario and check error details to suite reports. • Adds JUnit XML export for `SuiteResult`, enabling CI integration. • Adds JSON validity check to `giskard-checks`. • Adds `vulnerability_scan` function in `giskard-scan` for automated red-teaming across OWASP LLM Top-10 threat categories including prompt injection, harmful content, stereotypes, and misinformation. • Adds minimal OWASP LLM suite generator covering LLM01 indirect injection. • Introduces `giskard-llm` as a lean, provider-agnostic LLM routing layer replacing litellm, supporting provider extras (`openai`, `anthropic`, etc.) via `pip install 'giskard[openai]'`. • Introduces personas and extended context support for `UserSimulator` in generators. • Adds `metadata` parameter to the generator completion pipeline. • Adds generator retry and timeout policies to `giskard-agents`. • Adds step-level type discriminator and tool input coercion with output serialization to `giskard-agents`. • Adds generator-as-protocol-adapter pattern to `giskard-agents` for wrapping arbitrary LLM backends. • Supports pydantic-compatible input types in checks. • Introduces a judges directory for LLM and prompt-reliant checks in `giskard-checks`. Breaking changes: • Giskard v3 is a full rewrite; v2 APIs (`giskard.Model`, `giskard.Dataset`, `giskard.testing`, Giskard Hub) are not available in v3. Install `pip install 'giskard[llm]>2,<3'` to keep v2. • Requires Python 3.12+; Python versions below 3.12 are no longer supported. • `Scenario.from_sequence` is removed; use the step-based `Scenario` API instead. • Templating in conformity rules is no longer supported; configurations relying on template syntax in conformity rules will break. • Jinja parsing in `Workflow.chat` is now opt-in; workflows that relied on Jinja template rendering by default will no longer render templates unless explicitly enabled. • The `Interact` injection is now name-based; code using positional injection patterns will break. • The scenario() factory is removed; use the mutable `Scenario` constructor directly. https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0 Wed, 26 Aug 2026 01:15:51 GMT Giskard v3 rewrites as a modular monorepo with new LLM judge checks, OWASP scanning, Suite batching, regex/composition operators, and JUnit XML export. • Rewrites Giskard as a v3 monorepo structured around three installable packages: `giskard-checks`, `giskard-agents`, and `giskard-core`, with `giskard-scan` available via `pip install 'giskard[scan]'`. • Adds opt-out telemetry via `DO_NOT_TRACK=1` or `GISKARD_TELEMETRY_DISABLED=1` environment variables, collected through optional PostHog analytics in `giskard-core`. • Adds `RegexMatching` check to `giskard-checks` for asserting outputs match a regular expression, with ReDoS mitigation via regex timeout. • Adds `AllOf`, `AnyOf`, and `Not` check composition operators to `giskard-checks` for combining existing checks with boolean logic. • Adds `AnswerRelevance` LLM judge check to `giskard-checks`. • Adds `Toxicity` LLM judge check to `giskard-checks`. • Adds `annotations` support to `Scenario`/trace objects for attaching metadata to eval steps. • Introduces `Suite` for running batches of scenarios with dynamic binding; Suite.append() is chainable. • Allows scenarios to run multiple times within a suite. • Adds print_report() to result models for human-readable check output. • Adds scenario and check error details to suite reports. • Adds JUnit XML export for `SuiteResult`, enabling CI integration. • Adds JSON validity check to `giskard-checks`. • Adds `vulnerability_scan` function in `giskard-scan` for automated red-teaming across OWASP LLM Top-10 threat categories including prompt injection, harmful content, stereotypes, and misinformation. • Adds minimal OWASP LLM suite generator covering LLM01 indirect injection. • Introduces `giskard-llm` as a lean, provider-agnostic LLM routing layer replacing litellm, supporting provider extras (`openai`, `anthropic`, etc.) via `pip install 'giskard[openai]'`. • Introduces personas and extended context support for `UserSimulator` in generators. • Adds `metadata` parameter to the generator completion pipeline. • Adds generator retry and timeout policies to `giskard-agents`. • Adds step-level type discriminator and tool input coercion with output serialization to `giskard-agents`. • Adds generator-as-protocol-adapter pattern to `giskard-agents` for wrapping arbitrary LLM backends. • Supports pydantic-compatible input types in checks. • Introduces a judges directory for LLM and prompt-reliant checks in `giskard-checks`. Breaking changes: • Giskard v3 is a full rewrite; v2 APIs (`giskard.Model`, `giskard.Dataset`, `giskard.testing`, Giskard Hub) are not available in v3. Install `pip install 'giskard[llm]>2,<3'` to keep v2. • Requires Python 3.12+; Python versions below 3.12 are no longer supported. • `Scenario.from_sequence` is removed; use the step-based `Scenario` API instead. • Templating in conformity rules is no longer supported; configurations relying on template syntax in conformity rules will break. • Jinja parsing in `Workflow.chat` is now opt-in; workflows that relied on Jinja template rendering by default will no longer render templates unless explicitly enabled. • The `Interact` injection is now name-based; code using positional injection patterns will break. • The scenario() factory is removed; use the mutable `Scenario` constructor directly. https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1rc1 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1rc1 Tue, 18 Aug 2026 03:16:50 GMT Giskard v1.0.1rc1 adds PEP 561 type stubs, tighter public Literal/status types, and expanded prompt-injection scenarios for LLM agents. • Ships PEP 561 `py.typed` marker files for `giskard-core`, `giskard-llm`, `giskard-checks`, and `giskard-scan`, enabling full static-type-checking support in downstream projects. • Exports and tightens public `Literal`/status types from `giskard.types`, giving typed Python codebases a stable surface to import against. • Expands the built-in prompt-injection dataset with additional scenarios and templates, broadening adversarial coverage in `vulnerability_scan`. Breaking changes: • Unknown fields anywhere in the persisted-scenario tree are now rejected (strict validation); saved scenarios containing unrecognised fields will fail to load after upgrade. • The field holding the value under test on every check is renamed to `target_key`; any code that referenced the previous field name will break. https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1rc1 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1rc1 Tue, 18 Aug 2026 03:16:50 GMT Giskard v1.0.1rc1 adds PEP 561 type stubs, tighter public Literal/status types, and expanded prompt-injection scenarios for LLM agents. • Ships PEP 561 `py.typed` marker files for `giskard-core`, `giskard-llm`, `giskard-checks`, and `giskard-scan`, enabling full static-type-checking support in downstream projects. • Exports and tightens public `Literal`/status types from `giskard.types`, giving typed Python codebases a stable surface to import against. • Expands the built-in prompt-injection dataset with additional scenarios and templates, broadening adversarial coverage in `vulnerability_scan`. Breaking changes: • Unknown fields anywhere in the persisted-scenario tree are now rejected (strict validation); saved scenarios containing unrecognised fields will fail to load after upgrade. • The field holding the value under test on every check is renamed to `target_key`; any code that referenced the previous field name will break. https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0b3 https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0b3 Fri, 14 Aug 2026 00:45:43 GMT Giskard v3.0.0b3 ships a modular async-first eval and red-teaming library for agentic systems with `vulnerability_scan`, `quality_scan`, and LLM-as-judge checks. • Adds `vulnerability_scan` function in `giskard.scan` for automated red-teaming of agents across OWASP LLM Top-10 threat categories including prompt injection, harmful content, stereotypes, and misinformation — accepts `target`, `description`, and `languages` parameters. • Adds `quality_scan` with `KnowledgeBase` support in `giskard-scan` for RAG knowledge-base quality evaluation, replacing v2 RAGET. • Adds `generate_suite` function accepting custom `ScenarioGenerator` instances, with a `vulnerability_suite_generator_registry` for registering domain-specific generators. • Introduces `Scenario` API in `giskard.checks` for composing multi-turn eval interactions via .interact() and .check() chaining, with a .run() async entrypoint and .print_report() output. • Ships built-in LLM-as-judge checks — `Groundedness`, `Conformity`, and `LLMJudge` — in `giskard.checks`, with default model `openai/gpt-4o-mini`. • Adds a `Suite` abstraction in `giskard-checks` for running multiple `Scenario` instances together. • New `giskard[scan]`, `giskard[openai]`, and `giskard[anthropic]` install extras wire in provider SDKs and scanner dependencies independently. • Supports opting out of telemetry via `DO_NOT_TRACK=1` or `GISKARD_TELEMETRY_DISABLED=1` environment variables set before import. Breaking changes: • Giskard v2 is no longer actively maintained; the v2 automatic tabular scan (`giskard.Model` + `giskard.Dataset`), `giskard.testing` ML test suite, and Giskard Hub are not present in v3 — install `pip install 'giskard[llm]>2,<3'` to retain v2 behavior. • Requires Python 3.12+; earlier Python versions are no longer supported. https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0b3 https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0b3 Fri, 14 Aug 2026 00:45:43 GMT Giskard v3.0.0b3 ships a modular async-first eval and red-teaming library for agentic systems with `vulnerability_scan`, `quality_scan`, and LLM-as-judge checks. • Adds `vulnerability_scan` function in `giskard.scan` for automated red-teaming of agents across OWASP LLM Top-10 threat categories including prompt injection, harmful content, stereotypes, and misinformation — accepts `target`, `description`, and `languages` parameters. • Adds `quality_scan` with `KnowledgeBase` support in `giskard-scan` for RAG knowledge-base quality evaluation, replacing v2 RAGET. • Adds `generate_suite` function accepting custom `ScenarioGenerator` instances, with a `vulnerability_suite_generator_registry` for registering domain-specific generators. • Introduces `Scenario` API in `giskard.checks` for composing multi-turn eval interactions via .interact() and .check() chaining, with a .run() async entrypoint and .print_report() output. • Ships built-in LLM-as-judge checks — `Groundedness`, `Conformity`, and `LLMJudge` — in `giskard.checks`, with default model `openai/gpt-4o-mini`. • Adds a `Suite` abstraction in `giskard-checks` for running multiple `Scenario` instances together. • New `giskard[scan]`, `giskard[openai]`, and `giskard[anthropic]` install extras wire in provider SDKs and scanner dependencies independently. • Supports opting out of telemetry via `DO_NOT_TRACK=1` or `GISKARD_TELEMETRY_DISABLED=1` environment variables set before import. Breaking changes: • Giskard v2 is no longer actively maintained; the v2 automatic tabular scan (`giskard.Model` + `giskard.Dataset`), `giskard.testing` ML test suite, and Giskard Hub are not present in v3 — install `pip install 'giskard[llm]>2,<3'` to retain v2 behavior. • Requires Python 3.12+; earlier Python versions are no longer supported. https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b6 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b6 Thu, 06 Aug 2026 03:53:52 GMT Giskard v1.0.1b6 adds third-party scan integration, exposes default registry generators on the public API, and enforces non-blank judge reasons. • Exposes default registry generators on the public API via `vulnerability_suite_generator_registry`, allowing third-party generators to be registered and consumed by `vulnerability_scan` and `generate_suite`. • Integrates third-party scan support, enabling external `ScenarioGenerator` instances to plug into the scan pipeline. • Enforces non-blank judge reasons in LLM-as-judge checks, ensuring `Groundedness`, `Conformity`, and `LLMJudge` results always carry an explanatory rationale. https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b6 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b6 Thu, 06 Aug 2026 03:53:52 GMT Giskard v1.0.1b6 adds third-party scan integration, exposes default registry generators on the public API, and enforces non-blank judge reasons. • Exposes default registry generators on the public API via `vulnerability_suite_generator_registry`, allowing third-party generators to be registered and consumed by `vulnerability_scan` and `generate_suite`. • Integrates third-party scan support, enabling external `ScenarioGenerator` instances to plug into the scan pipeline. • Enforces non-blank judge reasons in LLM-as-judge checks, ensuring `Groundedness`, `Conformity`, and `LLMJudge` results always carry an explanatory rationale.