-
Notifications
You must be signed in to change notification settings - Fork 4.4k
Open
Labels
bugIssue describes a potential bug in ml-agents.Issue describes a potential bug in ml-agents.
Description
Describe the bug
When I call EndEpisode in OnActionReceived, OnEpisodeBegin be called twice.
First time callstack:
Void ArmAgent:OnEpisodeBegin ()+0x1 at C:\Users\username\DEV\Code\UnityProjects\AI-Playground\Assets\Scripts\ArmAgent.cs:[28:9-28:31] C#
Void Unity.MLAgents.Agent:_AgentReset ()+0x1d at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[1350:17-1350:34] C#
Void Unity.MLAgents.Agent:EndEpisodeAndReset (DoneReason)+0xa at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[834:13-834:27] C#
Void Unity.MLAgents.Agent:EndEpisode ()+0x3 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[808:13-808:55] C#
Void ArmAgent:OnActionReceived (ActionBuffers)+0xe1 at C:\Users\username\DEV\Code\UnityProjects\AI-Playground\Assets\Scripts\ArmAgent.cs:[70:13-70:26] C#
Void Unity.MLAgents.Actuators.VectorActuator:OnActionReceived (ActionBuffers)+0x1f at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Actuators\VectorActuator.cs:[76:13-76:64] C#
Void Unity.MLAgents.Actuators.ActuatorManager:ExecuteActions ()+0xdc at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Actuators\ActuatorManager.cs:[295:17-295:98] C#
Void Unity.MLAgents.Agent:AgentStep ()+0x27 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[1395:17-1395:52] C#
Void Unity.MLAgents.Academy:EnvironmentStep ()+0x101 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[602:21-602:40] C#
Void Unity.MLAgents.AcademyFixedUpdateStepper:FixedUpdate ()+0x33 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[43:17-43:52] C#
Second time callstack:
Void ArmAgent:OnEpisodeBegin ()+0x1 at C:\Users\username\DEV\Code\UnityProjects\AI-Playground\Assets\Scripts\ArmAgent.cs:[28:9-28:31] C#
Void Unity.MLAgents.Agent:_AgentReset ()+0x1d at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[1350:17-1350:34] C#
Void Unity.MLAgents.Academy:ForcedFullReset ()+0x14 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[561:13-561:39] C#
Void Unity.MLAgents.Academy:OnResetCommand ()+0x2 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[523:13-523:31] C#
Void Unity.MLAgents.RpcCommunicator:SendCommandEvent (CommandProto)+0x72 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Communicator\RpcCommunicator.cs:[306:21-306:52] C#
Void Unity.MLAgents.RpcCommunicator:UpdateEnvironmentWithInput (UnityRLInputProto)+0x19 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Communicator\RpcCommunicator.cs:[226:13-226:47] C#
Void Unity.MLAgents.RpcCommunicator:SendBatchedMessageHelper ()+0xdf at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Communicator\RpcCommunicator.cs:[416:13-416:49] C#
Void Unity.MLAgents.RpcCommunicator:DecideBatch ()+0x19 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Communicator\RpcCommunicator.cs:[328:13-328:40] C#
ActionBuffers Unity.MLAgents.Policies.RemotePolicy:DecideAction ()+0xd at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Policies\RemotePolicy.cs:[67:13-67:43] C#
Void Unity.MLAgents.Agent:DecideAction ()+0x41 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Agent.cs:[1411:13-1411:74] C#
Void Unity.MLAgents.Academy:EnvironmentStep ()+0xb8 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[591:21-591:44] C#
Void Unity.MLAgents.AcademyFixedUpdateStepper:FixedUpdate ()+0x33 at .\Library\PackageCache\com.unity.ml-agents@1aec33be79a2\Runtime\Academy.cs:[43:17-43:52] C#
To Reproduce
Code:
public override void OnEpisodeBegin() {
base.OnEpisodeBegin();
// reset environment
...
Debug.Log("New episode started.");
}
public override void OnActionReceived(ActionBuffers actions) {
base.OnActionReceived(actions);
...
SetReward(reward);
...
if (...) {
Debug.Log("Task finished.");
EndEpisode();
}
}Console logs / stack traces
Please wrap in triple backticks (```) to make it easier to read.
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):
- Unity Version:
Unity 6000.3.6f1 - OS + version:
Windows 11 25H2 - ML-Agents version:
4.0.1 - Torch version: (you can run
pip3 show torchto get this) - Environment:
Custom
NOTE: We are unable to help reproduce bugs with custom environments. Please attempt to reproduce your issue with one of the example environments, or provide a minimal patch to one of the environments needed to reproduce the issue.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugIssue describes a potential bug in ml-agents.Issue describes a potential bug in ml-agents.