Draft
Conversation
- Add Sphinx and related dependencies to pyproject.toml - Add serve-docs Makefile target for local documentation serving - Remove .python-version files from libs directories - Add docs/ directory with Sphinx configuration - Update pre-commit config to exclude example API keys from secret detection
Refactor documentation to prioritize the fluent API pattern scenario().interact().check() over InteractionSpec and TestCase APIs. Changes: - Update all tutorial files (rag-evaluation, testing-agents, chatbot-testing) to use fluent API instead of InteractionSpec/TestCase - Update getting started guides (quickstart, single-turn, multi-turn) to prioritize fluent API - De-emphasize InteractionSpec/TestCase in core-concepts, move to Advanced Usage - Update giskard-checks README to use fluent API in Quickstart section - Move InteractionSpec/TestCase examples to Advanced Usage section in README - Update Python docstrings (TestCase, Scenario, InteractionSpec) to recommend fluent API for most use cases - Wrap all examples in async functions with asyncio.run() for proper execution - Rename test_* functions to run_*_example() to avoid pytest confusion This reduces the learning curve for beginners while keeping advanced APIs available in reference documentation for power users. Refs: ENG-1294
…fy-api-surface-in-tutorials-fluent-api-focus docs(checks): simplify API surface in tutorials with fluent API focus
Remove 47 unused image files from docs/source/_static/images/: - 2 images from sdk/ directory - 3 images from oss/ directory - 42 images from hub/ directory (including scan/ subdirectory) None of these images were referenced in any documentation source files.
Remove all async wrapper functions (run_*_example, main) and asyncio.run() calls from documentation examples across all guides and tutorials. Add a disclaimer in quickstart.rst explaining how to run async code examples using asyncio.run() or within an async context (e.g., Jupyter notebooks or pytest-asyncio). This simplifies the documentation examples by showing the async code directly without unnecessary wrapper functions, while still providing guidance on how to execute the code.
…mplementation - Replace StringMatchingCheck with StringMatching class name - Update API parameters: content -> keyword, key -> text_key/keyword_key - Update JSONPath syntax: interactions[-1] -> trace.last - Add examples for normalization_form and case_sensitive parameters - Update all documentation files including tutorials and API references
* feat: rich display for test case results * chore: refactoring * feat: improved formatting * chore: cleanup * Update src/giskard/checks/core/result.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update src/giskard/checks/core/result.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update src/giskard/checks/core/result.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * fix: code formatting * Add rich representation for trace and interaction --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Pierre Le Jeune <pierre@giskard.ai>
…st-stringmatching-behavior-to-match-hub feature(checks): Improve string matching checks
- Run all tests on main branch - On PRs, run tests for changed projects and their dependents - giskard-core changes trigger tests for all projects - giskard-agents changes trigger tests for giskard-agents and giskard-checks - giskard-checks changes only trigger its own tests
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.20.0 to 3.20.3. - [Release notes](https://github.com/tox-dev/py-filelock/releases) - [Changelog](https://github.com/tox-dev/filelock/blob/main/docs/changelog.rst) - [Commits](tox-dev/filelock@3.20.0...3.20.3) --- updated-dependencies: - dependency-name: filelock dependency-version: 3.20.3 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
--- updated-dependencies: - dependency-name: aiohttp dependency-version: 3.13.3 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
- Remove docs directory and all documentation source files - Remove documentation dependencies from pyproject.toml - Remove serve-docs target from Makefile - Update uv sync command to remove --all-groups flag
…-update-tests-run-condition ci: update test run conditions based on project dependencies
* Add similarity check * Clean up * Use gemini embedding as default * build: add numpy as explicit dependency to giskard-agents and giskard-checks numpy is used directly in embeddings and semantic similarity checks but was only available as a transitive dependency. Adding it as an explicit dependency ensures it's always available. * refactor(agents): make embedding batch parameters configurable per-call - Remove max_batch_size and max_total_chars from EmbeddingParams - Add support for per-call batch size and total chars parameters - Add environment variable support for default batch limits (GISKARD_AGENTS_DEFAULT_MAX_BATCH_SIZE, GISKARD_AGENTS_DEFAULT_MAX_TOTAL_CHARS) - Update embed() and _embed() methods to accept optional params and batch parameters - Update all tests to use new API * docs(checks): add documentation and docstrings for SemanticSimilarity - Add SemanticSimilarity section to builtin-checks.rst following the same format as other checks (EqualityCheck, StringMatchingCheck, etc.) - Add SemanticSimilarity to quick reference imports in index.rst - Add comprehensive class-level docstring to SemanticSimilarity with Attributes and Examples sections - Add docstrings to get_embeddings method and enhance cosine_similarity function docstring --------- Co-authored-by: Kevin Messiaen <kevin.messiaen@icloud.com> Co-authored-by: Kevin Messiaen <114553769+kevinmessiaen@users.noreply.github.com>
* feature(check): improved equality test * test(equality): added test for equality check * refactor(checks): simplify extraction API and add JSONPath utilities - Replace Extractor ABC pattern with simple resolve() and provided_or_resolve() functions - Remove extraction_check.py in favor of direct JSONPath resolution - Add NotProvided sentinel and utility functions to giskard.core.utils - Update builtin checks (equality, groundedness, string_matching) to use new extraction API - Update tests to reflect simplified extraction interface This change makes JSONPath extraction more accessible as a builtin feature for metadata checks while reducing code complexity. * refactor(checks): extract normalization utilities into shared module - Move normalization logic from string_matching to utils.normalization - Add normalize_string and normalize_data functions for reuse - Update string_matching to use new normalization utilities - Add comprehensive tests for normalization utilities - Add Unicode normalization tests for equality checks * refactor(checks): replace Equality with ComparisonCheck system Replace the Equality check with a more general ComparisonCheck base class that supports multiple comparison operators (equals, greater than, less than, greater than or equal, less than or equal). BREAKING CHANGE: The Equality check has been replaced with Equals and other comparison checks. Import paths have changed from Equality to Equals, and new comparison operators are now available. * feat(checks): add NotEquals comparison check Add NotEquals check class that validates if extracted values do not equal an expected value. The check uses Python's __ne__ method for comparison and follows the same pattern as other comparison checks (Equals, LesserThan, GreaterThan, etc.). - Implement NotEquals class in comparison.py - Register check as 'not_equals' - Add comprehensive test coverage for various types and edge cases - Export NotEquals in builtin and main __init__.py modules * chore: improve check kind naming Co-authored-by: Pierre Le Jeune <pierre@giskard.ai> * feat(checks): support extracting expected values from traces in comparison checks - Add expected_value_key parameter to ComparisonCheck for dynamic extraction - Make expected_value optional when expected_value_key is provided - Add provided_or_resolve utility function for flexible value resolution - Update NotProvided to be a Pydantic BaseModel for better validation - Update tests to reflect new behavior when expected value extraction fails * chore: improve variable clarity --------- Co-authored-by: Pierre Le Jeune <pierre@giskard.ai>
Refresh fluent scenario examples, JSONPath usage, and docstrings to match current API behavior and trace access patterns.
chore(ci): update CI workflow - separate environment for functional tests requiring external API secrets - updated Makefile and workflow - updated test markers in packages to enable functional testing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.