Add Context.IsReplaying E2E tests by andystaples · Pull Request #3356 · Azure/azure-functions-durable-extension

andystaples · 2026-02-23T19:54:54Z

Summary

What changed?

Adds E2E tests that verify the behavior of orchestration context.IsReplaying across all relevant languages/backends
Also makes an improvement across the E2E tests to use polling-based log detection

Why is this change needed?

Bug found in Python, see linked work item

Issues / work items

Resolves Add E2E tests covering context.IsReplaying() across all languages/backends #3332
Related #

Project checklist

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

No
Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
Yes, an AI agent generated most of this PR

If AI was used:

Tool(s): VS Copilot
AI-assisted areas/files: All
What you changed after AI output: Fixed test skip attributes and small syntactic errors

AI verification (required if AI was used):

I understand the code and can explain it
I verified referenced APIs/types exist and are correct
I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
I reviewed concurrency/async behavior
I checked for unintended breaking or behavior changes

Testing

Automated tests

Result: Passed / Failed (link logs if failed)

Manual validation (only if runtime/behavior changed)

Environment (OS, .NET version, components):
Steps + observed results:
1.
2.
3.
Evidence (optional):

Notes for reviewers

N/A

test/e2e/Apps/BasicJava/src/main/java/com/function/IsReplayingChecks.java

Copilot

Pull request overview

Adds new E2E coverage to validate context.IsReplaying/equivalent behavior across non-.NET languages/backends, by introducing a new test suite plus language-specific orchestrators/activities that emit replay-state snapshots and log markers.

Changes:

Adds a new IsReplayingTests test class validating replay flags, snapshot progression, counters, conditional logging, and fan-out/fan-in behavior.
Adds IsReplaying* orchestrations + an echo activity to the Python, PowerShell, Node, and Java E2E apps to provide consistent test inputs/outputs.
Updates test.runsettings defaults (currently committed, but the file indicates it should not be).

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test/e2e/Tests/test.runsettings	Changes local default E2E language/app selection (should remain unmodified in repo).
test/e2e/Tests/Tests/IsReplayingTests.cs	New E2E test suite asserting replay-state semantics via orchestration outputs and host logs.
test/e2e/Apps/BasicPython/is_replaying_checks.py	Adds Python durable orchestrations/activities that record `context.is_replaying` at key checkpoints.
test/e2e/Apps/BasicPython/function_app.py	Registers the new Python blueprint so the functions are discoverable by the host.
test/e2e/Apps/BasicPowerShell/IsReplayingMultiActivity/run.ps1	Adds PowerShell multi-activity orchestration that records replay snapshots.
test/e2e/Apps/BasicPowerShell/IsReplayingMultiActivity/function.json	Binds the PowerShell multi-activity orchestrator to an orchestration trigger.
test/e2e/Apps/BasicPowerShell/IsReplayingFanOutFanIn/run.ps1	Adds PowerShell fan-out/fan-in orchestration to validate replay flags around parallel tasks.
test/e2e/Apps/BasicPowerShell/IsReplayingFanOutFanIn/function.json	Binds the PowerShell fan-out/fan-in orchestrator to an orchestration trigger.
test/e2e/Apps/BasicPowerShell/IsReplayingEcho/run.ps1	Adds PowerShell echo activity used by the replay-check orchestrations.
test/e2e/Apps/BasicPowerShell/IsReplayingEcho/function.json	Binds the PowerShell echo activity to an activity trigger.
test/e2e/Apps/BasicPowerShell/IsReplayingCounter/run.ps1	Adds PowerShell orchestration that counts replay vs non-replay checkpoints.
test/e2e/Apps/BasicPowerShell/IsReplayingCounter/function.json	Binds the PowerShell counter orchestrator to an orchestration trigger.
test/e2e/Apps/BasicPowerShell/IsReplayingConditionalLog/run.ps1	Adds PowerShell orchestration that logs different markers on replay vs live paths.
test/e2e/Apps/BasicPowerShell/IsReplayingConditionalLog/function.json	Binds the PowerShell conditional-log orchestrator to an orchestration trigger.
test/e2e/Apps/BasicPowerShell/IsReplayingBasic/run.ps1	Adds PowerShell “basic” orchestration that captures replay flags before/after a single activity.
test/e2e/Apps/BasicPowerShell/IsReplayingBasic/function.json	Binds the PowerShell basic orchestrator to an orchestration trigger.
test/e2e/Apps/BasicNode/src/functions/IsReplayingChecks.ts	Adds Node durable orchestrations/activities mirroring the replay-check scenarios.
test/e2e/Apps/BasicJava/src/main/java/com/function/IsReplayingChecks.java	Adds Java durable orchestrations/activities mirroring the replay-check scenarios (includes workaround for a known serialization issue).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/e2e/Tests/test.runsettings

test/e2e/Tests/Tests/IsReplayingTests.cs

test/e2e/Apps/BasicJava/src/main/java/com/function/IsReplayingChecks.java

…Checks.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/e2e/Tests/Tests/IsReplayingTests.cs

Copilot

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/e2e/Apps/BasicPython/is_replaying_checks.py

test/e2e/Tests/Tests/IsReplayingTests.cs

Copilot

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/e2e/Tests/Tests/IsReplayingTests.cs

Copilot

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/e2e/Tests/Tests/IsReplayingTests.cs

Copilot

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-25T19:13:12Z

test/e2e/Tests/Helpers/TestLoggerProvider.cs

@@ -2,9 +2,6 @@
 // Licensed under the MIT License. See License.txt in the project root for license information.

 using System.Collections.Concurrent;


The using Xunit.Sdk; directive was removed, but DiagnosticMessage on line 75 is from the Xunit.Sdk namespace. With implicit usings enabled, Xunit.Sdk is not automatically included, which will cause a compilation error. The using directive should be restored.

Suggested change

using System.Collections.Concurrent;

using System.Collections.Concurrent;

using Xunit.Sdk;

Copilot · 2026-02-25T19:13:12Z

test/e2e/Tests/Helpers/TestLoggerProvider.cs

@@ -2,9 +2,6 @@
 // Licensed under the MIT License. See License.txt in the project root for license information.

 using System.Collections.Concurrent;


The using Microsoft.Extensions.Logging; directive was removed, but types from this namespace (ILoggerProvider, ILogger, LogLevel, EventId) are used in this file (lines 8, 67, 72). Other files in the same project (e.g., FunctionAppFixture.cs, FixtureHelpers.cs) explicitly import this namespace, indicating it's not part of implicit usings. This will cause a compilation error. The using directive should be restored.

Suggested change

using System.Collections.Concurrent;

using System.Collections.Concurrent;

using Microsoft.Extensions.Logging;

Copilot · 2026-02-25T19:13:13Z

test/e2e/Tests/Helpers/TestLoggerProvider.cs

@@ -2,9 +2,6 @@
 // Licensed under the MIT License. See License.txt in the project root for license information.

 using System.Collections.Concurrent;


The using Xunit.Abstractions; directive was removed, but types from this namespace (ITestOutputHelper, IMessageSink) are used in this file (lines 10-11, 14, 46). Other files in the same project (e.g., FunctionAppFixture.cs, FunctionAppProcess.cs) explicitly import this namespace, indicating it's not part of implicit usings. This will cause a compilation error. The using directive should be restored.

Suggested change

using System.Collections.Concurrent;

using System.Collections.Concurrent;

using Xunit.Abstractions;

Initial implementation

c22fbb2

Copilot AI review requested due to automatic review settings February 23, 2026 19:54

andystaples changed the title ~~Initial implementation~~ Add Context.IsReplaying E2E tests Feb 23, 2026

Copilot started reviewing on behalf of andystaples February 23, 2026 19:55 View session

github-code-quality bot found potential problems Feb 23, 2026

View reviewed changes

test/e2e/Apps/BasicJava/src/main/java/com/function/IsReplayingChecks.java Dismissed Show dismissed Hide dismissed

Copilot AI reviewed Feb 23, 2026

View reviewed changes

test/e2e/Tests/test.runsettings Outdated Show resolved Hide resolved

test/e2e/Tests/Tests/IsReplayingTests.cs Outdated Show resolved Hide resolved

test/e2e/Apps/BasicJava/src/main/java/com/function/IsReplayingChecks.java Outdated Show resolved Hide resolved

Update test/e2e/Apps/BasicJava/src/main/java/com/function/IsReplaying…

3f799c6

…Checks.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 23, 2026 20:26

PR feedback

c88cc90

Copilot started reviewing on behalf of andystaples February 23, 2026 20:27 View session

Copilot AI reviewed Feb 23, 2026

View reviewed changes

test/e2e/Tests/Tests/IsReplayingTests.cs Outdated Show resolved Hide resolved

andystaples added 2 commits February 23, 2026 14:33

Fix Node tests

51ead96

Skip Node problematic test

9e2ba34

Copilot AI review requested due to automatic review settings February 23, 2026 22:09

Copilot started reviewing on behalf of andystaples February 23, 2026 22:10 View session

Copilot AI reviewed Feb 23, 2026

View reviewed changes

test/e2e/Apps/BasicPython/is_replaying_checks.py Show resolved Hide resolved

test/e2e/Apps/BasicPython/is_replaying_checks.py Outdated Show resolved Hide resolved

test/e2e/Tests/Tests/IsReplayingTests.cs Outdated Show resolved Hide resolved

andystaples added 2 commits February 23, 2026 15:22

Skip failing test for PS+DTS

b7406c9

PR feedback

511e82a

Copilot AI review requested due to automatic review settings February 23, 2026 22:32

Copilot started reviewing on behalf of andystaples February 23, 2026 22:33 View session

Copilot AI reviewed Feb 23, 2026

View reviewed changes

Merge branch 'dev' into andystaples/add-is-replaying-tests

53e70f8

andystaples mentioned this pull request Feb 24, 2026

Allow orchestration history to determine isPlayed Azure/azure-functions-durable-powershell#107

Merged

bachuv reviewed Feb 24, 2026

View reviewed changes

test/e2e/Tests/Tests/IsReplayingTests.cs Show resolved Hide resolved

Add test comments

e31f4a2

Copilot AI review requested due to automatic review settings February 25, 2026 18:18

Copilot started reviewing on behalf of andystaples February 25, 2026 18:20 View session

Copilot AI reviewed Feb 25, 2026

View reviewed changes

test/e2e/Tests/Tests/IsReplayingTests.cs Outdated Show resolved Hide resolved

andystaples added 2 commits February 25, 2026 11:29

More comments, expected output, strengthen log check

fbe24c9

Update tests to use log polling for consistency

7e5c9c5

Copilot AI review requested due to automatic review settings February 25, 2026 19:05

Copilot started reviewing on behalf of andystaples February 25, 2026 19:07 View session

Copilot AI reviewed Feb 25, 2026

View reviewed changes

Merge branch 'dev' into andystaples/add-is-replaying-tests

32dbb19

		@@ -2,9 +2,6 @@
		// Licensed under the MIT License. See License.txt in the project root for license information.

		using System.Collections.Concurrent;

	using System.Collections.Concurrent;
	using System.Collections.Concurrent;
	using Xunit.Sdk;

	using System.Collections.Concurrent;
	using System.Collections.Concurrent;
	using Microsoft.Extensions.Logging;

	using System.Collections.Concurrent;
	using System.Collections.Concurrent;
	using Xunit.Abstractions;

Conversation

andystaples commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed?

Why is this change needed?

Issues / work items

Project checklist

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

Testing

Automated tests

Manual validation (only if runtime/behavior changed)

Notes for reviewers

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

andystaples commented Feb 23, 2026 •

edited

Loading