Skip to content

dataconnect(test): be more lax in the ChiSquareTest significance result assertion to reduce false positives in unit tests#7937

Merged
dconeybe merged 1 commit intomainfrom
dconeybe/dataconnect/ChiSquareTestMoreRelaxedSignificanceResult
Mar 19, 2026
Merged

dataconnect(test): be more lax in the ChiSquareTest significance result assertion to reduce false positives in unit tests#7937
dconeybe merged 1 commit intomainfrom
dconeybe/dataconnect/ChiSquareTestMoreRelaxedSignificanceResult

Conversation

@dconeybe
Copy link
Contributor

@dconeybe dconeybe commented Mar 19, 2026

This PR updates the ChiSquareTest significance result assertions in the firebase-dataconnect unit tests to be more lax. By changing the rejection threshold from 0.0001 to 0.00001, this change aims to reduce the number of false positives encountered during test execution.

The flaky test failure that motivated this PR was https://github.com/firebase/firebase-android-sdk/actions/runs/23309337787/job/67792345315?pr=7936 which had the following counts:

image
AssertionFailedError: observations.size=19, observations=[(1, 5146L), (2, 5306L), (3, 5329L), (4, 5216L), (5, 5109L), (6, 5324L), (7, 5113L), (8, 5414L), (9, 5426L), (10, 5299L), (11, 5256L), (12, 5269L), (13, 5319L), (14, 5273L), (15, 5130L), (16, 5305L), (17, 5250L), (18, 5497L), (19, 5019L)]
expectedObservedCount=5263.1578947368425, SignificanceResult(statistic=49.3887, pValue=0.00009333452)

From looking at the chart, the counts definitely have a reasonable standard deviation. Therefore, lowering the rejection threshold makes sense so that a distribution like this in the future does not result in a test failure.

Highlights

  • Test Stability: Reduced the strictness of the ChiSquareTest significance assertions to decrease the frequency of false positives in property-based unit tests.
Changelog
  • AbstractWithEvenNumDigitsDistributionUnitTest.kt
    • Updated the rejection threshold for ChiSquareTest.withDefaults().test() from 0.0001 to 0.00001.

…lt assertion to reduce false positives in unit tests
@gemini-code-assist
Copy link
Contributor

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

@github-actions
Copy link
Contributor

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

@dconeybe dconeybe merged commit 86fba0d into main Mar 19, 2026
43 checks passed
@dconeybe dconeybe deleted the dconeybe/dataconnect/ChiSquareTestMoreRelaxedSignificanceResult branch March 19, 2026 21:22
@github-actions github-actions bot mentioned this pull request Mar 20, 2026
rlazo pushed a commit that referenced this pull request Mar 20, 2026
…lt assertion to reduce false positives in unit tests (#7937)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants