Improvement of the competition for the IROS edition

[omichel]

This issue lists the improvements that we are going to implement before starting a new edition of the wrestling competition.

• Add team members in the participant.json file, e.g., "team":["John Smith", "Robert Young", "Paul Parker"]
• Add an optional organization in the participant.jsonfile, e.g., "organization": "The University of Edinburgh"
• Allow more time at the end of the qualification for the participant to update their controllers.
• Run the final games as best of 5 games (instead of best of 3).
• Use a better CPU core partitionning:
  • Webots could be assigned CPU 0 (physical) via docker run --cpuset-cpus=0,4 ...
  • The controller of RED robot could run on CPU 1 (physical) + share CPU 3 (physical) via docker run --cpuset-cpus=1,5,3 ...
  • The controller of BLUE robot could run on CPU 2 (physical) + share CPU 3 (physical) via docker run --cpuset-cpus=2,6,7 ...
• Limit GPU RAM usage (mentioned in the README.md).
• Reduce a lot the height of the stage to give a chance to robots who fell off to recover by climbing up one small step (done in Updated rules #96).
• Reduce the strength of the motors to be more realistic. After checking, the current value seems to be correctly calibrated.
• Lower the basic time step to 10 ms (instead of 20) to allow for a 100Hz refresh rate of the sensors, including the IMU.
• Improve the KO countdown algorithm: when a robot gets below the height threshold, start the countdown for this robot. However, if the other robot gets below the first one, then stop the countdown for the first robot and start it for the second one, etc.
• Leave a little bit more time to the controller dockers to start-up before the game actually starts.
• Allow a participant to start a game versus another participant of their choice.
• Give awards to the second and third place.

[JustLuckyRobotics]

i forgot one thing! the imu sucks. 50hz really makes for lousy orientation measurement, it works but its laggy and drifts even with attempting to tune either madgwicks or Mahony filters. the nao robot imu itself is 100hz which is the minimum really for good orientation measurement. i gather webots can run at 100hz but might be too demanding for two async robot controllers in this scenario. Is there a compromise that could somehow be made for the imu measurement, i wish i had a better suggestion than to simply give the controllers the robots "true" chest orientation as a quaternion. if it was real, we would have the higher update rate and have better orientation data.

[omichel]

The basic time step of the simulation is set to 20 ms which limits the frequency of sensors to 50 Hz indeed. Lowering this value to 10 ms would allow a frequency of 100 Hz, but that would also slow down significantly the simulation speed and I am not sure Webots would be able to keep up real time in these conditions. I will make make some investigation to evaluate if that is doable.

Alternatively, we could add a InertialUnit with some noise. However, the inertial unit device provides ground truth information and doesn't drift at all, which is not very realistic. So, I would like to avoid this option as much as possible.

[omichel]

I conducted some tests on the competition server and it turns out that with a 10 ms time step, Webots cannot run real-time any more, but approximately 0.5x real time in particular when the robots are colliding with each other. A possibility would be to make Webots run 0.5x real-time on the server and document it. The robot controllers can always use the getTime() method to know the current simulation time. That would also leave twice as much real time for the controllers to compute. The drawback of this approach is that games running on the server would last twice as much time, making the waiting queue last longer than before.

We now have to consider what is the most important: having a 100Hz IMU (and other sensors) as well as extra controller compute time or having a fast evaluation queue? I would tend to consider the benefits are higher than the drawbacks. What do you think?

Also it would be possible to add more self-hosted runner machines to decrease the queue time. But we have to ensure these runners machine have the same hardware (CPU, GPU, RAM, etc.) so that the resulting games are not dependent on the machine on which the run. However, currently we don't have two exact same machines that we can register as self hosted runners.

[JustLuckyRobotics]

If Webots running at 0.5x real time is viable with the only drawback of the evaluation queue being slower, one potential middle ground would be to limit the number of fights during the qualifiers to either a max number of fights (5 for example) or a limited number of time (20-30 minutes) this way the reduced time would only require people to review what happened and resubmit. it would also capture and contain the times when the evaluation queue bricks.

I don't see resubmitting as that much of an issue as it requires participant interaction to climb which is a good thing, many times when submitting and having to wait longer than 20-30 to review the work we would zone out. Having to wait for multiple hours completely derailed development.

[JustLuckyRobotics]

Oh! and with the new addition of selective fights, maybe it would be cool to include "number of fights" "Wins" "loss's", would be fun stats us nerds love. Would also help with things like the "side" wins other mentioned, things such as "most fought controller", "most aggressive controller" etc etc.

[omichel]

Nice idea. We will have to introduce badges for all those achievements.

[kibergrad]

I think something is wrong with the improved KO countdown algorithm in terms of its implementation, or may be I have understood the rule wrong?
Isn't it that in the unique situation, if the first bot is right above (or even some part of it may be a hand or leg...), then the countdown will start only for the one right below it?
If this is so, the match between the Oliviers' and ijneks' controller shows that even if they are way apart from each other, it will trigger the unique situation and the KO countdown as mentioned above.

It would be great if you could please clarify the rule a bit more.

[omichel]

It is difficult to determine if a robot is right above the other. So, the rule is actually simpler than what you imagined, based on the altitude of the head of the robot. It can be summarized as follow:

Definitions:

1. "Up" means that the Z coordinate of the head of a robot is above the height threshold.
2. "Down" means that the Z coordinate of the head of a robot is below the height threshold.
3. "Below" means that the Z coordinate of the head of a robot is below the one of the other robot.
4. "Above" means that the Z coordinate of the head of a robot is above the one of the other robot.

Rules:

1. The two robots are up: if one robot gets down, then its countdown starts.
2. Only one robot is up: if it falls below the second robot, then the countdown of the second robot is stopped and the countdown of the first robot starts.
3. The two robots are down, the first robot being above the second one: if first robot gets below the second one, the countdown of the first robot is resumed while the one of the second robot is stopped.
4. Whenever a robot gets up, its countdown is reset.