“Stochastic Gradient Descent Tricks”
E367291
“Stochastic Gradient Descent Tricks” is a well-known paper by Léon Bottou that surveys practical techniques and heuristics for effectively applying stochastic gradient descent in machine learning.
All labels observed (1)
| Label | Occurrences |
|---|---|
| “Stochastic Gradient Descent Tricks” canonical | 1 |
How this entity was disambiguated
This entity first appeared as the object of triple T3542937 — resolving that mention is where its identity was fixed. The disambiguator weighed these candidate entities and picked the highlighted one (or “None”, minting a new entity). This is how homonymy is resolved: the same surface form can point to different entities.
NED1
Entity disambiguation (via context triple)
gpt-5-mini-2025-08-07
Target entity: “Stochastic Gradient Descent Tricks” Context triple: [Léon Bottou, hasPublication, “Stochastic Gradient Descent Tricks”]
-
A.
Adam: A Method for Stochastic Optimization
"Adam: A Method for Stochastic Optimization" is a highly influential machine learning paper that introduces the Adam optimizer, a widely used adaptive gradient-based optimization algorithm for training deep neural networks.
-
B.
“A fast learning algorithm for deep belief nets”
“A fast learning algorithm for deep belief nets” is a seminal 2006 paper by Geoffrey Hinton that introduced an efficient unsupervised pretraining method for deep neural networks using stacked restricted Boltzmann machines.
-
C.
Large-Scale Distributed Deep Networks
Large-Scale Distributed Deep Networks is a seminal research work that introduced methods for training deep neural networks efficiently across large-scale distributed computing infrastructure, enabling breakthroughs in modern large-scale AI systems.
-
D.
Automatic Adam
Automatic Adam is the nickname of Adam Vinatieri, a legendary NFL placekicker renowned for his clutch, game-winning field goals in high-pressure situations.
-
E.
“Learning representations by back-propagating errors”
“Learning representations by back-propagating errors” is a landmark 1986 research paper that popularized the backpropagation algorithm for training multi-layer neural networks, helping to launch the modern field of deep learning.
- F. None of above. chosen
- G. Unsure - the case is ambiguous/there is not enough information to decide.
NED2
Entity disambiguation (via description)
gpt-5-mini-2025-08-07
Target entity: “Stochastic Gradient Descent Tricks” Target entity description: “Stochastic Gradient Descent Tricks” is a well-known paper by Léon Bottou that surveys practical techniques and heuristics for effectively applying stochastic gradient descent in machine learning.
-
A.
Adam: A Method for Stochastic Optimization
"Adam: A Method for Stochastic Optimization" is a highly influential machine learning paper that introduces the Adam optimizer, a widely used adaptive gradient-based optimization algorithm for training deep neural networks.
-
B.
“A fast learning algorithm for deep belief nets”
“A fast learning algorithm for deep belief nets” is a seminal 2006 paper by Geoffrey Hinton that introduced an efficient unsupervised pretraining method for deep neural networks using stacked restricted Boltzmann machines.
-
C.
Large-Scale Distributed Deep Networks
Large-Scale Distributed Deep Networks is a seminal research work that introduced methods for training deep neural networks efficiently across large-scale distributed computing infrastructure, enabling breakthroughs in modern large-scale AI systems.
-
D.
Automatic Adam
Automatic Adam is the nickname of Adam Vinatieri, a legendary NFL placekicker renowned for his clutch, game-winning field goals in high-pressure situations.
-
E.
“Learning representations by back-propagating errors”
“Learning representations by back-propagating errors” is a landmark 1986 research paper that popularized the backpropagation algorithm for training multi-layer neural networks, helping to launch the modern field of deep learning.
- F. None of above. chosen
Statements (47)
| Predicate | Object |
|---|---|
| instanceOf |
machine learning paper
ⓘ
scientific paper ⓘ |
| author | Léon Bottou ⓘ |
| citedFor |
empirical insights into SGD behavior
ⓘ
practical guidance on SGD hyperparameters ⓘ |
| contributesTo |
best practices for training large-scale models
ⓘ
understanding of SGD convergence in practice ⓘ |
| describes |
effect of noise in gradient estimates
ⓘ
impact of learning rate on convergence ⓘ implementation-level optimization tricks ⓘ practical stopping criteria ⓘ stochastic approximation viewpoint of SGD ⓘ trade-offs between batch and stochastic methods ⓘ |
| field |
machine learning
ⓘ
optimization ⓘ |
| focusesOn |
computational efficiency of SGD
ⓘ
convergence behavior of SGD ⓘ convergence diagnostics ⓘ data shuffling strategies ⓘ implementation details of SGD ⓘ large-scale learning ⓘ learning rate schedules ⓘ mini-batch strategies ⓘ numerical stability in SGD ⓘ online learning setting ⓘ practical aspects of stochastic gradient descent ⓘ practical heuristics for SGD ⓘ regularization techniques ⓘ scaling to large datasets ⓘ step size selection ⓘ variance reduction in SGD ⓘ |
| hasAuthorAffiliation | NEC Laboratories America (for Léon Bottou at the time of writing) ⓘ |
| intendedFor |
practitioners of machine learning
ⓘ
researchers in optimization ⓘ |
| language | English ⓘ |
| mainTopic |
optimization algorithms
ⓘ
practical training heuristics ⓘ stochastic gradient descent ⓘ |
| proposes |
heuristics for choosing learning rates
ⓘ
heuristics for data ordering ⓘ heuristics for handling non-convex objectives ⓘ heuristics for parameter initialization ⓘ |
| relatedTo |
convex optimization
ⓘ
neural network training ⓘ non-convex optimization ⓘ online learning algorithms ⓘ support vector machines ⓘ |
How these facts were elicited
The pipeline generated the facts above by prompting gpt-5.1 with this entity's name + description and the instruction below.
Instruction
You are a knowledge base construction expert. Given a subject entity and a description of it, return factual statements that you know for the subject as a JSON list of dictionaries(triples), where keys must be "subject", "predicate" and "object". The number of facts may be very high, between 25 to 50 or more, for very popular subjects. For less popular subjects, the number of facts can be very low, like 5 or 10. # Requirements - If you don't know the subject at all, return an empty list. - If the subject is not a named entity, return an empty list. - Include at least one triple where predicate is "instanceOf". - Do not get too wordy. - Separate several objects into multiple triples with one object.
Input
Subject: “Stochastic Gradient Descent Tricks” Description of subject: “Stochastic Gradient Descent Tricks” is a well-known paper by Léon Bottou that surveys practical techniques and heuristics for effectively applying stochastic gradient descent in machine learning.
Referenced by (1)
Full triples — surface form annotated when it differs from this entity's canonical label.