Shannon entropy
E1168
Shannon entropy is a fundamental measure in information theory that quantifies the average uncertainty or information content in a random variable or message source.
All labels observed (5)
| Label | Occurrences |
|---|---|
| Shannon entropy canonical | 13 |
| Shannon information | 2 |
| Shannon additivity axiom | 1 |
| Shannon limit | 1 |
| shannon | 1 |
How this entity was disambiguated
This entity first appeared as the object of triple T26944 — resolving that mention is where its identity was fixed. The disambiguator weighed these candidate entities and picked the highlighted one (or “None”, minting a new entity). This is how homonymy is resolved: the same surface form can point to different entities.
NED1
Entity disambiguation (via context triple)
gpt-5-mini-2025-08-07
Target entity: Shannon entropy Context triple: [Claude Shannon, knownFor, Shannon entropy]
-
A.
Claude Shannon
Claude Shannon was an American mathematician and electrical engineer known as the "father of information theory" for founding the mathematical framework underlying digital communication and data compression.
-
B.
SHASS
SHASS is the abbreviated name commonly used for the School of Humanities, Arts, and Social Sciences at academic institutions.
-
C.
Chomsky hierarchy
The Chomsky hierarchy is a classification of formal grammars into four types that correspond to increasing levels of generative power and computational complexity in formal language theory.
-
D.
Geeks Bearing Gifts
Geeks Bearing Gifts is a book by computing pioneer Ted Nelson that reflects on the history, philosophy, and future of digital media and information technology.
-
E.
Enquire (software)
Enquire (software) was an early hypertext system created by Tim Berners-Lee that served as a conceptual precursor to the World Wide Web.
- F. None of above. chosen
- G. Unsure - the case is ambiguous/there is not enough information to decide.
NED2
Entity disambiguation (via description)
gpt-5-mini-2025-08-07
Target entity: Shannon entropy Target entity description: Shannon entropy is a fundamental measure in information theory that quantifies the average uncertainty or information content in a random variable or message source.
-
A.
Claude Shannon
Claude Shannon was an American mathematician and electrical engineer known as the "father of information theory" for founding the mathematical framework underlying digital communication and data compression.
-
B.
SHASS
SHASS is the abbreviated name commonly used for the School of Humanities, Arts, and Social Sciences at academic institutions.
-
C.
Chomsky hierarchy
The Chomsky hierarchy is a classification of formal grammars into four types that correspond to increasing levels of generative power and computational complexity in formal language theory.
-
D.
Geeks Bearing Gifts
Geeks Bearing Gifts is a book by computing pioneer Ted Nelson that reflects on the history, philosophy, and future of digital media and information technology.
-
E.
Enquire (software)
Enquire (software) was an early hypertext system created by Tim Berners-Lee that served as a conceptual precursor to the World Wide Web.
- F. None of above. chosen
Statements (50)
| Predicate | Object |
|---|---|
| instanceOf |
entropy measure
ⓘ
information theory concept ⓘ random variable functional ⓘ uncertainty measure ⓘ |
| appliesTo |
discrete probability distributions
ⓘ
discrete random variables ⓘ |
| captures | expected codeword length lower bound in lossless compression ⓘ |
| dependsOn | probability distribution of a random variable ⓘ |
| field | information theory ⓘ |
| generalizes | Hartley entropy ⓘ |
| hasFormula | H(X) = -\sum_x p(x) \log p(x) ⓘ |
| introducedBy | Claude Shannon ⓘ |
| introducedInWork | A Mathematical Theory of Communication ⓘ |
| introducedInYear | 1948 ⓘ |
| invariantUnder | relabeling of outcomes ⓘ |
| isAdditiveFor | independent random variables ⓘ |
| isConcaveIn | probability distribution ⓘ |
| isMaximumWhen | distribution is uniform ⓘ |
| isMinimumWhen | distribution is degenerate ⓘ |
| isNonNegative | true ⓘ |
| isSpecialCaseOf |
Rényi entropy
ⓘ
Tsallis entropy ⓘ |
| logarithmBaseDetermines | unit of information ⓘ |
| measuredIn |
bits
ⓘ
hartleys ⓘ nats ⓘ |
| minimumValue | 0 ⓘ |
| namedAfter | Claude Shannon ⓘ |
| quantifies |
average information content of a random variable
ⓘ
average uncertainty of a random variable ⓘ |
| relatedConcept |
Kullback–Leibler divergence
ⓘ
conditional entropy ⓘ differential entropy ⓘ mutual information ⓘ relative entropy ⓘ |
| satisfies |
Shannon–Khinchin axioms
ⓘ
chain rule for entropy ⓘ |
| symbol | H ⓘ |
| usedIn |
bioinformatics
ⓘ
channel coding theory ⓘ cryptography ⓘ data compression theory ⓘ ecology diversity indices ⓘ machine learning ⓘ neuroscience ⓘ signal processing ⓘ statistical mechanics ⓘ thermodynamics analogies ⓘ |
| usedToDefine |
Shannon capacity of a channel
ⓘ
linked to:
A Mathematical Theory of Communication
entropy rate of a stochastic process ⓘ |
How these facts were elicited
The pipeline generated the facts above by prompting gpt-5.1 with this entity's name + description and the instruction below.
Instruction
You are a knowledge base construction expert. Given a subject entity and a description of it, return factual statements that you know for the subject as a JSON list of dictionaries(triples), where keys must be "subject", "predicate" and "object". The number of facts may be very high, between 25 to 50 or more, for very popular subjects. For less popular subjects, the number of facts can be very low, like 5 or 10. # Requirements - If you don't know the subject at all, return an empty list. - If the subject is not a named entity, return an empty list. - Include at least one triple where predicate is "instanceOf". - Do not get too wordy. - Separate several objects into multiple triples with one object.
Input
Subject: Shannon entropy Description of subject: Shannon entropy is a fundamental measure in information theory that quantifies the average uncertainty or information content in a random variable or message source.
Referenced by (18)
Full triples — surface form annotated when it differs from this entity's canonical label.
subject linked to:
Claude
linked to: Shannon entropy
linked to: Shannon entropy
linked to: Shannon entropy
subject linked to:
Boltzmann–Gibbs entropy in statistical mechanics
linked to: Shannon entropy
linked to: Shannon entropy