Skip to content

EndEpisode cause OnEpisodeBegin called twice #6284

@GF-Huang

Description

@GF-Huang

Describe the bug
When I call EndEpisode in OnActionReceived, OnEpisodeBegin be called twice.

First time callstack:

Void ArmAgent:OnEpisodeBegin ()+0x1 at C:\Users\username\DEV\Code\UnityProjects\AI-Playground\Assets\Scripts\ArmAgent.cs:[28:9-28:31]	C#
Void Unity.MLAgents.Agent:_AgentReset ()+0x1d at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[1350:17-1350:34]	C#
Void Unity.MLAgents.Agent:EndEpisodeAndReset (DoneReason)+0xa at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[834:13-834:27]	C#
Void Unity.MLAgents.Agent:EndEpisode ()+0x3 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[808:13-808:55]	C#
Void ArmAgent:OnActionReceived (ActionBuffers)+0xe1 at C:\Users\username\DEV\Code\UnityProjects\AI-Playground\Assets\Scripts\ArmAgent.cs:[70:13-70:26]	C#
Void Unity.MLAgents.Actuators.VectorActuator:OnActionReceived (ActionBuffers)+0x1f at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Actuators\VectorActuator.cs:[76:13-76:64]	C#
Void Unity.MLAgents.Actuators.ActuatorManager:ExecuteActions ()+0xdc at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Actuators\ActuatorManager.cs:[295:17-295:98]	C#
Void Unity.MLAgents.Agent:AgentStep ()+0x27 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[1395:17-1395:52]	C#
Void Unity.MLAgents.Academy:EnvironmentStep ()+0x101 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[602:21-602:40]	C#
Void Unity.MLAgents.AcademyFixedUpdateStepper:FixedUpdate ()+0x33 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[43:17-43:52]	C#

Second time callstack:

Void ArmAgent:OnEpisodeBegin ()+0x1 at C:\Users\username\DEV\Code\UnityProjects\AI-Playground\Assets\Scripts\ArmAgent.cs:[28:9-28:31]	C#
Void Unity.MLAgents.Agent:_AgentReset ()+0x1d at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[1350:17-1350:34]	C#
Void Unity.MLAgents.Academy:ForcedFullReset ()+0x14 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[561:13-561:39]	C#
Void Unity.MLAgents.Academy:OnResetCommand ()+0x2 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[523:13-523:31]	C#
Void Unity.MLAgents.RpcCommunicator:SendCommandEvent (CommandProto)+0x72 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Communicator\RpcCommunicator.cs:[306:21-306:52]	C#
Void Unity.MLAgents.RpcCommunicator:UpdateEnvironmentWithInput (UnityRLInputProto)+0x19 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Communicator\RpcCommunicator.cs:[226:13-226:47]	C#
Void Unity.MLAgents.RpcCommunicator:SendBatchedMessageHelper ()+0xdf at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Communicator\RpcCommunicator.cs:[416:13-416:49]	C#
Void Unity.MLAgents.RpcCommunicator:DecideBatch ()+0x19 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Communicator\RpcCommunicator.cs:[328:13-328:40]	C#
ActionBuffers Unity.MLAgents.Policies.RemotePolicy:DecideAction ()+0xd at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Policies\RemotePolicy.cs:[67:13-67:43]	C#
Void Unity.MLAgents.Agent:DecideAction ()+0x41 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[1411:13-1411:74]	C#
Void Unity.MLAgents.Academy:EnvironmentStep ()+0xb8 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[591:21-591:44]	C#
Void Unity.MLAgents.AcademyFixedUpdateStepper:FixedUpdate ()+0x33 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[43:17-43:52]	C#

To Reproduce
Code:

public override void OnEpisodeBegin() {
    base.OnEpisodeBegin();
	// reset environment
	...
	Debug.Log("New episode started.");
}
public override void OnActionReceived(ActionBuffers actions) {
    base.OnActionReceived(actions);
	...
	SetReward(reward);
	...
	if (...) {
		Debug.Log("Task finished.");
		EndEpisode();
	}
}

Console logs / stack traces
Please wrap in triple backticks (```) to make it easier to read.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • Unity Version: Unity 6000.3.6f1
  • OS + version: Windows 11 25H2
  • ML-Agents version: 4.0.1
  • Torch version: (you can run pip3 show torch to get this)
  • Environment: Custom

NOTE: We are unable to help reproduce bugs with custom environments. Please attempt to reproduce your issue with one of the example environments, or provide a minimal patch to one of the environments needed to reproduce the issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIssue describes a potential bug in ml-agents.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions