Mr. Costantino said the design was not at fault and that the towering mast, which stood 237 feet tall, had not created āany kind of problem.ā
āThe ship was an unsinkable ship,ā he said. āI say it, I repeat it.ā
- Designer of sunken ship
I read one of the papers. About the specific question you have: given a string of bits s, theyāre making the choice to associate the empirical distribution to s, as if s was generated by an iid Bernoulli process. So if s has 10 zero bits and 30 one bits, its associated empirical distribution is Ber(3/4). This is the distribution which theyāre calculating the entropy of. I have no idea on what basis they are making this choice.
The rest of the paper didnāt make sense to me - they are somehow assigning a number N of āinformation statesā which can change over time as the memory cells fail. I honestly have no idea what itās supposed to mean and kinda suspect the whole thing is rubbish.
Edit: after reading the authorās quotes from the associated hype article Iām 100% sure itās rubbish. Itās also really funny that they didnāt manage to catch the COVID-19 research hype train so theyāve pivoted to the simulation hypothesis.