Queries, comments, suggestions? Please contact us.



Area Digital technology – Fundamental concepts / Information theory

IEV ref171-07-15

Symbol
H(X)

en
entropy, <in information theory>
average information content
DEPRECATED: negentropy
mean value of the information content of the events in a finite set of mutually exclusive and jointly exhaustive events

H= i=1 n p( x i )I( x i ) = i=1 n p( x i )log( 1 p( x i ) ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGibGaeyypa0ZaaabCaeaacaWGWbGaaiikaiaadIhadaWgaaWc baGaamyAaaqabaGccaGGPaGaeyyXICTaamysaiaacIcacaWG4bWaaS baaSqaaiaadMgaaeqaaOGaaiykaaWcbaGaamyAaiabg2da9iaaigda aeaacaWGUbaaniabggHiLdGccqGH9aqpdaaeWbqaaiaadchacaGGOa GaamiEamaaBaaaleaacaWGPbaabeaakiaacMcacqGHflY1ciGGSbGa ai4BaiaacEgacaGGOaWaaSaaaeaajugqbiaaigdaaOqaaiaadchaca GGOaGaamiEamaaBaaaleaacaWGPbaabeaakiaacMcaaaGaaiykaaWc baGaamyAaiabg2da9iaaigdaaeaacaWGUbaaniabggHiLdaaaa@6788@

where X={ x 1 ,, x n } MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGybGaeyypa0ZaaiWaaeaacaWG4bWaaSbaaSqaaiaaigdaaeqa aOGaaiilaiaac6cacaGGUaGaaiOlaiaadIhadaWgaaWcbaGaamOBaa qabaaakiaawUhacaGL9baaaaa@47F5@ is the set of events x i ( i=1,,n ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWG4bWaaSbaaSqaaiaadMgaaeqaaOGaaGPaVpaabmaabaGaamyA aiabg2da9KqzafGaaGymaiaacYcacaaMc8UaeSOjGSKaaiilaiaayk W7kiaad6gaaiaawIcacaGLPaaaaaa@4C2F@ , I( x i ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGjbGaaiikaiaadIhadaWgaaWcbaGaamyAaaqabaGccaGGPaaa aa@414F@ are their information contents and p( x i ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGWbGaaiikaiaadIhadaWgaaWcbaGaamyAaaqabaGccaGGPaaa aa@4176@ the probabilities of their occurrences, subject to i=1 n p( x i )=1 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aadaaeWbqaaiaadchacaGGOaGaamiEamaaBaaaleaacaWGPbaabeaa kiaacMcacqGH9aqpjugqbiaaigdaaSqaaiaadMgacqGH9aqpcaaIXa aabaGaamOBaaqdcqGHris5aaaa@49CA@

EXAMPLE Let { a,b,c } MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aaomaacmaakeaajugibiaadggacaGGSaGaamOyaiaacYcacaWGJbaa kiaawUhacaGL9baaaaa@43FE@ be a set of three events and let p(a)=0,5 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGWbGaaiikaiaadggacaGGPaqcLbqacqGH9aqpcaaIWaGaaiil aiaaiwdaaaa@43D9@ , p(b)=0,25 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGWbGaaiikaiaadkgacaGGPaqcLbqacqGH9aqpcaaIWaGaaiil aiaaikdacaaI1aaaaa@4496@ and p(c)=0,25 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGWbGaaiikaiaadogacaGGPaqcLbqacqGH9aqpcaaIWaGaaiil aiaaikdacaaI1aaaaa@4497@ be the probabilities of their occurrences. The entropy of this set is H(X)=p(a)I(a)+p(b)I(b)+p(c)I(c)=1,5ShMathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aajugibiaadIeacaGGOaGaamiwaiaacMcacqGH9aqpcaWGWbGaaiik aiaadggacaGGPaGaeyyXICTaamysaiaacIcacaWGHbGaaiykaiabgU caRiaadchacaGGOaGaamOyaiaacMcacqGHflY1caWGjbGaaiikaiaa dkgacaGGPaGaey4kaSIaamiCaiaacIcacaWGJbGaaiykaiabgwSixl aadMeacaGGOaGaam4yaiaacMcacqGH9aqpjugabiaaigdacaGGSaGa aGynaiaaysW7caGGtbGaaiiAaaaa@63F9@ .


[SOURCE: IEC 80000-13:2008, 13-25, modified – Addition of information useful for the context of the IEV, and adaptation to the IEV rules]


fr
entropie, <en théorie de l’information> f
DÉCONSEILLÉ: néguentropie, f
espérance mathématique de la quantité d'information des événements d'un ensemble exhaustif d'événements s'excluant mutuellement

H= i=1 n p( x i )I( x i ) = i=1 n p( x i )log( 1 p( x i ) ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGibGaeyypa0ZaaabCaeaacaWGWbGaaiikaiaadIhadaWgaaWc baGaamyAaaqabaGccaGGPaGaeyyXICTaamysaiaacIcacaWG4bWaaS baaSqaaiaadMgaaeqaaOGaaiykaaWcbaGaamyAaiabg2da9iaaigda aeaacaWGUbaaniabggHiLdGccqGH9aqpdaaeWbqaaiaadchacaGGOa GaamiEamaaBaaaleaacaWGPbaabeaakiaacMcacqGHflY1ciGGSbGa ai4BaiaacEgacaGGOaWaaSaaaeaajugqbiaaigdaaOqaaiaadchaca GGOaGaamiEamaaBaaaleaacaWGPbaabeaakiaacMcaaaGaaiykaaWc baGaamyAaiabg2da9iaaigdaaeaacaWGUbaaniabggHiLdaaaa@6788@

X={ x 1 ,, x n } MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGybGaeyypa0ZaaiWaaeaacaWG4bWaaSbaaSqaaiaaigdaaeqa aOGaaiilaiaac6cacaGGUaGaaiOlaiaadIhadaWgaaWcbaGaamOBaa qabaaakiaawUhacaGL9baaaaa@47F5@ est l'ensemble des événements x i ( i=1,,n ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWG4bWaaSbaaSqaaiaadMgaaeqaaOGaaGPaVpaabmaabaGaamyA aiabg2da9KqzafGaaGymaiaacYcacaaMc8UaeSOjGSKaaiilaiaayk W7kiaad6gaaiaawIcacaGLPaaaaaa@4C2F@ , I( x i ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGjbGaaiikaiaadIhadaWgaaWcbaGaamyAaaqabaGccaGGPaaa aa@414F@ sont leurs quantités d'information et p( x i ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGWbGaaiikaiaadIhadaWgaaWcbaGaamyAaaqabaGccaGGPaaa aa@4176@ leurs probabilités de réalisation, avec i=1 n p( x i )=1 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aadaaeWbqaaiaadchacaGGOaGaamiEamaaBaaaleaacaWGPbaabeaa kiaacMcacqGH9aqpjugqbiaaigdaaSqaaiaadMgacqGH9aqpcaaIXa aabaGaamOBaaqdcqGHris5aaaa@49CA@

EXEMPLE Soit { a,b,c } MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aaomaacmaakeaajugibiaadggacaGGSaGaamOyaiaacYcacaWGJbaa kiaawUhacaGL9baaaaa@43FE@ un jeu de trois événements dont les probabilités de réalisation sont p(a)=0,5 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGWbGaaiikaiaadggacaGGPaqcLbqacqGH9aqpcaaIWaGaaiil aiaaiwdaaaa@43D9@ , p(b)=0,25 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGWbGaaiikaiaadkgacaGGPaqcLbqacqGH9aqpcaaIWaGaaiil aiaaikdacaaI1aaaaa@4496@ et p(c)=0,25 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aacaWGWbGaaiikaiaadogacaGGPaqcLbqacqGH9aqpcaaIWaGaaiil aiaaikdacaaI1aaaaa@4497@ . L'entropie de ce jeu est H(X)=p(a)I(a)+p(b)I(b)+p(c)I(c)=1,5ShMathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbbjxAHX garuavP1wzZbItLDhis9wBH5garmWu51MyVXgaruWqVvNCPvMCG4uz 3bqee0evGueE0jxyaibaiGc9aspC0FXdbbc9asFfpec8Eeeu0lXdbb a9frFj0xb9Lqpepeea0xd9s8qiYRWxGi6xij=hbba9q8aq0=yq=He9 q8qiLsFr0=vr0=vr0db8meaabaGacmaadiWaaiWabaabaiaafaaake aajugibiaadIeacaGGOaGaamiwaiaacMcacqGH9aqpcaWGWbGaaiik aiaadggacaGGPaGaeyyXICTaamysaiaacIcacaWGHbGaaiykaiabgU caRiaadchacaGGOaGaamOyaiaacMcacqGHflY1caWGjbGaaiikaiaa dkgacaGGPaGaey4kaSIaamiCaiaacIcacaWGJbGaaiykaiabgwSixl aadMeacaGGOaGaam4yaiaacMcacqGH9aqpjugabiaaigdacaGGSaGa aGynaiaaysW7caGGtbGaaiiAaaaa@63F9@ .


[SOURCE: IEC 80000-13:2008, 13-25, modifié – Ajout d’informations utiles pour le contexte de l’IEV, et adaptation aux règles de l’IEV]


ar
الانتروبيا, <في نظرية المعلومات>
القيمة المتوسطة للمعلومات
متوسط محتوى المعلومات

de
Entropie, <in der Informationstheorie> f

fi
entropia, <informaatioteoriassa>
keskimääräinen informaatiomäärä

ja
エントロピー, <情報理論>

pl
entropia , <w teorii informacji> f
średnia zawartość informacji, f

pt
entropia, <em teoria da informação>

zh
熵, <信息论中>

Publication date: 2019-03-29
Copyright © IEC 2020. All Rights Reserved.