You train a PPO agent on a web-navigation task with a small reward at every step and 200-step episodes, using GAE with and .
The value network fits well on held-out states, yet updates are noisy and returns swing widely between seeds.
You train a PPO agent on a web-navigation task with a small reward at every step and 200-step episodes, using GAE with and .
The value network fits well on held-out states, yet updates are noisy and returns swing widely between seeds.