Proximal Policy Optimization

E162136

Proximal Policy Optimization is a popular reinforcement learning algorithm that improves policy gradient methods by using clipped objective functions to achieve stable and efficient training.

All labels observed (5)

How this entity was disambiguated

Statements (49)

Predicate Object
instanceOf policy gradient method
reinforcement learning algorithm
abbreviation PPO
aimsTo constrain policy updates
improve sample efficiency
improve training stability
algorithmFamily actor-critic
arXivId 1707.06347
citationVenue arXiv preprint
commonlyUsedFor game playing
robotics control
simulated control tasks
comparedTo Trust Region Policy Optimization
linked to: TRPO
designGoal avoid second-order optimization
simplify trust region methods
developedAt OpenAI
evaluationBenchmarks Atari 2600 games
linked to: Atari 2600

MuJoCo continuous control tasks
OpenAI Gym
influenced many modern deep RL baselines
introducedBy Alec Radford
Filip Wolski
John Schulman
Oleg Klimov NERFINISHED
Prafulla Dhariwal
introducedInPaper Proximal Policy Optimization Algorithms
introducedInYear 2017
keyHyperparameter GAE lambda
clip range
discount factor
entropy coefficient
learning rate
minibatch size
number of epochs
objectiveContains clipping term
entropy bonus (optional)
objectiveType surrogate objective
oftenImplementedIn PyTorch
TensorFlow
optimizationType on-policy
policyRepresentation neural network
relatedTo Trust Region Policy Optimization
linked to: TRPO
supports continuous action spaces
discrete action spaces
updateRule multiple epochs of minibatch updates per batch of data
uses clipped surrogate objective
policy gradient
stochastic gradient ascent
valueFunction critic network

How these facts were elicited

Referenced by (13)

Full triples — surface form annotated when it differs from this entity's canonical label.

John Schulman notableWork Proximal Policy Optimization
John Schulman authorOf “Proximal Policy Optimization Algorithms”
linked to: Proximal Policy Optimization
PPO fullName Proximal Policy Optimization
PPO abbreviationFor Proximal Policy Optimization
PPO introducedInPaper Proximal Policy Optimization Algorithms
linked to: Proximal Policy Optimization
PPO hasVariant PPO-Clip
linked to: Proximal Policy Optimization
PPO2 basedOn Proximal Policy Optimization
PPO2 abbreviationOf Proximal Policy Optimization 2
linked to: Proximal Policy Optimization
Proximal Policy Optimization introducedInPaper Proximal Policy Optimization Algorithms
linked to: Proximal Policy Optimization
Generalized Advantage Estimation compatibleWith Proximal Policy Optimization
Philipp Moritz notableWork Proximal Policy Optimization
Philipp Moritz coAuthorOf Proximal Policy Optimization Algorithms
linked to: Proximal Policy Optimization
Philipp Moritz hasGivenTalkOn Proximal Policy Optimization