LerpworksAcademy
On this page

Math and physics › Transitions, randomness, numbers

Numbers in a computer: integers, floats and precision

Inside a computer, every number is a row of on/off switches, and the row has a fixed length. That explains the odd things numbers do in games. A score turns negative after its maximum. 0.1 + 0.2 is not 0.3. A clock falls forty minutes behind in a day. Models jitter far from the center of the world, and black squares appear on the screen. This page shows how whole numbers and fractions are stored, and which rules follow from that.

Why this matters

A game does arithmetic all the time: positions, time, gold, the color of every pixel. Most of the time numbers behave the way they do in a notebook. But every number in a computer has a limit and a step size, and when the game reaches them, the bug looks like magic. An object jitters, a timer lies, an “equals” check never fires, the screen goes black. Once you know how numbers are stored, you recognize these bugs at a glance.

This is the last lesson of the course “Math in plain words.” From the earlier lessons you need only normalizing a vector.

Further in the article

  1. Binary: everything is built from yes and no
  2. Integers
  3. Fractions: scientific notation
  4. Why 0.1 + 0.2 does not equal 0.3
  5. Errors add up
  6. Money and counts belong in integers
  7. Few bits: half precision and packing
  8. Zero, infinity and “not a number”
  9. In engines and Blender
  10. Where it breaks
  11. Connections: the same idea in other fields
  12. Key points
  13. Test yourself