Yesterday I handed in the essay which I have mentioned a few times now in this blog. As it was now finished, I decided to send it over to my younger brother to see what he thought…
Topher is a software engineer working for Rare, and has a far more in-depth knowledge of computer science than I do, so we often have interesting conversations on topics such as these. This time was no exception, and it turns out that Topher actually has a huge interest in fixed vs floating-point mathematics, which he readily shared his thoughts on with me. I will below transcribe an excellent comparison of the two which he sent me as part of WhatsApp conversation we had on the subject:
“The simplest way to imagine it is this – think of your fixed-point number as a metre stick, where the smallest number is 0 and the largest is 100 (ignore the sign, and ignore how many bits this is). Looking at the metre stick you can count all the centimetres and all the way down to the millimetres, but no smaller. There is a finite, fixed, minimum amount of distance between the two values on the stick – one millimetre.
Now, say I want to store a value of 106cm. I can’t, its too big. This is where the fixed point shift comes in (conceptually). We’ll ‘shift the fixed point twice’, and double the exponent at the same time. In a real fixed-point number this would mean the maximum value doubles twice – but we’re working in base 10 on this stick, not base 2 – so it multiplies by 10 twice. Now our metre stick can suddenly show 100m! However, if we look down, the smallest value we can now see is 1cm – we can no longer see the millimetre divisions – if we had a value that used to be 33.34cm it is now 33cm.
This is the key idea behind fixed point. Adding extra bits lets us get more precision, and shifting where our ‘fixed point’ is increases the maximum value we can store, at the cost of losing precision. BUT we have clear, guaranteed precision, which is important because of the relationship to floating point numbers.
Now, consider our metre stick again. It is one metre long, and when you look at it, you very clearly see two markings – 0m at the left end, and 1m at the right end. In between this, you can see markings for 10cm, 20cm, 30cm etc. spaced evenly along it. But you notice something odd. You can’t find say, 64cm. It’s either 60cm or 70cm – there are no markings in between. Seemingly, all of the 10cm demarcations are empty, until you look to the very left. You can see 1cm, 2cm, 3cm etc. inbetween 0cm and 10cm, but nothing between most of them – no 3.8cm to be found. But you can see millimetre markings between 0cm and 1cm and, looking really closely, you can actually see 10 tiny markings between 0cm and 1mm. The space between your markings are apparently not fixed or finite – they instead get exponentially larger, every 10 times (it’s important again that we’re in base 10 here, not base 2). You think that this is a stupid-ass metre stick!
Now, you’ve got a piece of furniture that you know is somewhere between 60cm and 70cm, but you cannot for the life of you figure out anything more. Without going into the nitty gritty details, this is where the ‘shift’ you talked about earlier comes into play. We increase our exponent by a value of 1 and… boom! Our metre stick is now actually 10m long! And even better, we can now see every centimetre between 0 and 100! We don’t have anything longer than a metre in our flat that needs measuring that precisely, so we’re all good. We can now measure the coffee table and see that it’s exactly 68cm.
BUT! There are no longer any markings between 0 and 1mm. A millimetre is now the smallest value we can measure, where before it was 1/10th of a millimetre. This has a very important implication – if the range of values being captured becomes large enough, then your higher values will become increasingly more imprecise.”
At this point the conversation digressed into the particulars of how this number theory relates to digital audio in particular, and becomes much harder to transcribe in a readable way, but it left me with a lot of food for thought and opened up some new interesting research avenues. Not bad for a guy who used to use photos of himself drinking milk to present himself to the world!
