Proximal Policy Optimization

E162136

Proximal Policy Optimization is a popular reinforcement learning algorithm that improves policy gradient methods by using clipped objective functions to achieve stable and efficient training.

All labels observed (7)

How this entity was disambiguated

Statements (49)

Predicate Object
instanceOf policy gradient method ⓘ
reinforcement learning algorithm ⓘ
abbreviation PPO ⓘ
aimsTo constrain policy updates ⓘ
improve sample efficiency ⓘ
improve training stability ⓘ
algorithmFamily actor-critic ⓘ
arXivId 1707.06347 ⓘ
citationVenue arXiv preprint ⓘ
commonlyUsedFor game playing ⓘ
robotics control ⓘ
simulated control tasks ⓘ
comparedTo Trust Region Policy Optimization ⓘ
linked to: TRPO
designGoal avoid second-order optimization ⓘ
simplify trust region methods ⓘ
developedAt OpenAI ⓘ
evaluationBenchmarks Atari 2600 games ⓘ
linked to: Atari 2600

MuJoCo continuous control tasks ⓘ
OpenAI Gym ⓘ
influenced many modern deep RL baselines ⓘ
introducedBy Alec Radford ⓘ
Filip Wolski ⓘ
John Schulman ⓘ
Oleg Klimov NERFINISHED ⓘ
Prafulla Dhariwal ⓘ
introducedInPaper Proximal Policy Optimization Algorithms ⓘ
introducedInYear 2017 ⓘ
keyHyperparameter GAE lambda ⓘ
clip range ⓘ
discount factor ⓘ
entropy coefficient ⓘ
learning rate ⓘ
minibatch size ⓘ
number of epochs ⓘ
objectiveContains clipping term ⓘ
entropy bonus (optional) ⓘ
objectiveType surrogate objective ⓘ
oftenImplementedIn PyTorch ⓘ
TensorFlow ⓘ
optimizationType on-policy ⓘ
policyRepresentation neural network ⓘ
relatedTo Trust Region Policy Optimization ⓘ
linked to: TRPO
supports continuous action spaces ⓘ
discrete action spaces ⓘ
updateRule multiple epochs of minibatch updates per batch of data ⓘ
uses clipped surrogate objective ⓘ
policy gradient ⓘ
stochastic gradient ascent ⓘ
valueFunction critic network ⓘ

How these facts were elicited

Referenced by (20)

Full triples — surface form annotated when it differs from this entity's canonical label.

John Schulman → notableWork → Proximal Policy Optimization ⓘ
John Schulman → authorOf → “Proximal Policy Optimization Algorithms” ⓘ
linked to: Proximal Policy Optimization
PPO → fullName → Proximal Policy Optimization ⓘ
PPO → abbreviationFor → Proximal Policy Optimization ⓘ
PPO → introducedInPaper → Proximal Policy Optimization Algorithms ⓘ
linked to: Proximal Policy Optimization
PPO → hasVariant → PPO-Clip ⓘ
linked to: Proximal Policy Optimization
PPO2 → basedOn → Proximal Policy Optimization ⓘ
PPO2 → abbreviationOf → Proximal Policy Optimization 2 ⓘ
linked to: Proximal Policy Optimization
Proximal Policy Optimization → introducedInPaper → Proximal Policy Optimization Algorithms ⓘ
linked to: Proximal Policy Optimization
Generalized Advantage Estimation → compatibleWith → Proximal Policy Optimization ⓘ
Philipp Moritz → notableWork → Proximal Policy Optimization ⓘ
Philipp Moritz → coAuthorOf → Proximal Policy Optimization Algorithms ⓘ
linked to: Proximal Policy Optimization
Philipp Moritz → hasGivenTalkOn → Proximal Policy Optimization ⓘ
Filip Wolski → notableWork → Proximal Policy Optimization ⓘ
Filip Wolski → contributedTo → Proximal Policy Optimization (PPO) ⓘ
linked to: Proximal Policy Optimization
Filip Wolski → authorOf → “Proximal Policy Optimization Algorithms” ⓘ
linked to: Proximal Policy Optimization
Prafulla Dhariwal → notableWork → Proximal Policy Optimization Algorithms ⓘ
linked to: Proximal Policy Optimization
Prafulla Dhariwal → coAuthorOf → "Proximal Policy Optimization Algorithms" ⓘ
linked to: Proximal Policy Optimization
Natural Policy Gradient → relatedTo → Proximal Policy Optimization ⓘ
Oleg Klimov → knownFor → Proximal Policy Optimization ⓘ