tostr_johncarlosbaez / is on Nostr

Tweets forwarded from https://twitter.com/johncarlosbaez by [tostr](https://github.com/slaninas/tostr) bot.

Last Notes

2022-07-12 15:01:57

We've been doing examples with just two alternatives: heads up or down. But you can do Shannon entropy for any number of mutually exclusive alternatives. It measures how ignorant you are of what will happen. That is: how much you learn on average when it does! (6/n, n = 6) ([source](https://twitter.com/johncarlosbaez/status/1546871778751836166))

2022-07-12 14:56:52

Here's the pattern: the Shannon entropy is biggest when the coin is fair. As it becomes more and more likely for one side to land up than the other, the entropy drops. You're more sure about what will happen... so you learn less, on average, from seeing what happens! (5/n) ([source](https://twitter.com/johncarlosbaez/status/1546870850376282113))

2022-07-12 14:51:47

Suppose you know a coin lands heads up ¼ of the time and tails up ¾ of the time. The expected amount of information you get from a coin flip is -¼ log(¼) - ¾ log(¾) What is this in bits? (4/n) ([source](https://twitter.com/johncarlosbaez/status/1546869680966471680))

2022-07-12 14:46:41

Suppose you know a coin lands heads up ⅓ the time and tails up ⅔ the time. The expected amount of information you get from a coin flip is -⅓ log(⅓) - ⅔ log(⅔) What is this in bits? You can use this calculator that does logs in base 2: https://t.co/MoBsMvLWYn (3/n) ([source](https://twitter.com/johncarlosbaez/status/1546868455332188160))

2022-07-12 14:41:35

Suppose you know a coin lands heads up ½ the time and tails up ½ the time. The expected amount of information you get from a coin flip is -½ log(½) - ½ log(½) Taking the log base 2 gives you the Shannon entropy in bits. What do you get? (2/n) ([source](https://twitter.com/johncarlosbaez/status/1546867039209279490))

2022-07-12 14:36:30

You flip a coin. You know the probability that it lands heads up. How much information do you get, on average, when you discover which side lands up? It's not hard to work this out. It's a simple example of 'Shannon entropy'. (1/n) https://t.co/R5sqx3SNtF ([source](https://twitter.com/johncarlosbaez/status/1546864911338221568))