AI 지식창고 AI 지식창고

PPO

• Proximal Policy Optimization with Generalized Advantage Estimation (PPO2)

  CS & ML Basic 2023. 2. 14. 17:04

  PPO2 (Proximal Policy Optimization with Generalized Advantage Estimation) is an extension of PPO that combines the PPO algorithm with Generalized Advantage Estimation (GAE), which is a method for estimating the advantage function. The main difference between PPO and PPO2 is the way they estimate the advantage function. In PPO, the advantage function is estimated using a single-step estimate, whi..

  Read More
• Comparison of TRPO and PPO in Reinforcement Learning

  CS & ML Basic 2023. 2. 14. 16:33

  TRPO In TRPO, the policy update is performed by solving the following constrained optimization problem: maximize L(θ, θ_old) subject to KL(π_θ_old || π_θ)

  Read More
• Proximal Policy Optimization (PPO) Algorithm

  CS & ML Basic 2023. 2. 14. 16:02

  The objective of the Proximal Policy Optimization (PPO) algorithm is to train a policy function that can control an agent's behavior in a given environment, such that it maximizes the expected cumulative reward over time. More formally, we can define the objective of PPO as follows: $$J(\theta )=\mathbb{E}\pi \theta \left[\sum _{t=0}^{\mathrm{\infty }}{\gamma }^t{r}_t\right]$$ where $J(\theta )$ is the ..

  Read More