Questions w.r.t. GNNPolicy implementation #5
Description
Hi @HannesStark @TobiasSchmidtDE , thank you for this repository -- I've been playing around with it for a bit now, and find it much easier to work with than the original NerveNet. I am planning to implement some updates for my own research, but I've noticed some issues with the current codebase, and was wondering if you had any pointers for me to look into.
- I see that most configs use
GnnPolicy(v0), but there exists a more up-to-date policyGnnPolicy_V2. Do they have separate config files that are not pushed to the repo, or require additional hyperparameter tuning? I did perform some hyperparameter tuning on Ant (based onGNN_AntBulletEnv-v0.yaml), but cannot get the Ant to move even at 5M iterations with the GNN policies. The MLPs work well, however, pointing to either bad hyperparameters or a potential bug in the GNNPolicy. Did your eventual hyperparameters Any pointers w.r.t. these would be a huge help. - Agents other than Ant need their .xml files to be edited slightly to be parsed correctly. Specifically, the parts with
ignorein their names need to be commented out for to work:
<body name="torso">
<!--joint armature="0" axis="1 0 0" damping="0" limited="false" name="ignore1" pos="0 0 0" stiffness="0" type="slide"/-->
<!--joint armature="0" axis="0 0 1" damping="0" limited="false" name="ignore2" pos="0 0 0" ref="1.25" stiffness="0" type="slide"/-->
<!--joint armature="0" axis="0 1 0" damping="0" limited="false" name="ignore3" pos="0 0 0" stiffness="0" type="hinge"/-->
I'm currently testing if GNN training will work for a simpler agent e.g. Hopper. Are there any other issues I should be aware of? Many thanks!