It may come as a surprise to many that the way computers handle numbers is not very accurate. Indeed, it can be said that error is built into the very foundation of digital computers, and while the end user often does not see the result of these errors, they can be very problematic for programmers, scientists, engineers, and calculation intense industries such as money management and military operations.

At the recent Supercomputing Frontiers 2015 conference in Singapore, computer scientist John Gustafson outlined the problems with floating points in his keynote and later in an interview. Given the complexity — and severity — of the problem, it’s worth taking a second in-depth look at the issue.

The Problem

Developer Richard Harris, who wrote a series of articles on the dangers of floating point, said in one post, “The dragon of numerical error is not often roused from his slumber, but if incautiously approached he will occasionally inflict catastrophic damage upon the unwary programmer’s calculations. So much so that some programmers, having chanced upon him in the forests of IEEE 754 floating point arithmetic, advise their fellows against travelling in that fair land.”

Because computers – which are machines of precision and exactness – are often made to deal with unprecise and inexact numbers (such as pi, and irrationals), methods must be devised to compensate for computational error, and to make the end result as close to the correct answer as possible. One solution has been devised that is still in use today: floating point. Floating point is a method similar to scientific notation, which uses a decimal point, sign bit, and a number of exact digits to represent a number.

Since The IEEE Standard for Floating-Point Arithmetic was published in 1985, this standard has come to dominate the mathematical methods used by hardware and software engineers for the basic operations computers perform whenever running an application. Ideally, a one-size-fits-all standard such as this one would minimize error and promote uniformity of results across a broad spectrum of hardware.

Unfortunately, this has not been the practical result. Different processors and software packages, designed to handle floating point operations, often result in slightly different answers, due to rounding errors, and differing orders of operation.

One way that programmers often compensate is to use as many digits as possible to represent a number. In modern computers, this means that 32 – 64 bits of data are almost always used to represent a single floating point number. While modern computers are also very fast at calculations, this many bits must be stored and retrieved from memory, causing significant latency in calculations.

Furthermore, due to compounding error, traditional properties of algebra – such as the commutative and associative property – do not necessarily apply to floating point operations. In other words, (a + b) + c =/= a + (b + c), nor does c * (a + b) = c*a + c*b.

In the case of floating point, using these differing approaches often yields dissimilar results.

The post The Evils of Floating Point, and the Joys of Unum appeared first on VR World.

To many people, the floating point–universal number debate is something extraneous: an academic issue that involves computer scientists, engineers, and hardware manufacturers.

But as John Gustafson said during his keynote at the Supercomputing Frontiers 2015 conference on Tuesday, the inaccuracies of floating point estimates have real world implications. They can be deadly both in the real sense  — with missile defense batteries mis-calculating intercept times — or as Gustafson explained they can also lose battles in a virtual sense.

During intense battles in multiplayer games, floating point estimates would give different answers for different players. The calculation of if a players’ shot would be a lethal headshot — or a frustrating miss — would have slightly different answers on different platforms. In order to get reliable, reproducible results in the event of discrepancy the software would need to switch back to integers.

In order to have a better understanding of the benefits of unums, and the challenges of implementing them into hardware, the VR World team spoke with Gustafson on the sidelines of the Supercomputing Frontiers 2015 conference in Singapore to learn more.

The VR World team interviews Dr. Gustafson

VR World: You mentioned in your keynote that the implementation of Unum is challenging — in the words of one unnamed Intel executive ‘you can’t boil the ocean’. Why is this?

John Gustafson: What he’s saying is that you can’t change the world. All you have is IEEE floats. That’s the standard. ‘You can’t add a new number type, that’s not going to happen’ is what he said.

VRW: How would you categorize the feedback you’ve gotten from CPU vendors about implementing unums?

JG: People at AMD also didn’t get it. That was a kind of different opposition. They just didn’t see that I could save them so much power, electricity and bandwidth. Maybe it just looked too ambitious to them.

I’m not worried about what the hardware people think. I know they are going to hate it. They’ll have to build it, re-design circuits and all of that. I’m  more interested in everyone else.

VRW: What’s the cost of keeping the existing floating point system, versus implementing Unums? What’s the cost of transitioning hardware to support this, versus the cost of errors in everyday life?

JG: Remember: everything you can do with floats you can do with Unums. They are a subset. It’s a choice between one or the other; if it were I think it would never get off the ground. But if you can do everything you can do now if you have Unums, and you can also do other things, you can then incrementally work your way into them.

The other thing is right now we have to deal with at least two, or three, different precisions. Half precision is now out there. Nvidia has got the half precision out there in hardware as a native type, and single precision as well as double precision are everywhere. Quad precision is not supported by anyone’s hardware… I keep watching to see if it’s going to pop up.

But we already have to manage two, or three, different sizes.

I say replace it with one. And the hardware will let that slide continuously from all different sizes. It will simplify things so it may be cheaper and smaller on chip to do it that way then to have a bunch of single precision units and double precision units. That’s the way they do it now. They have to build separate hardware. Which is very wasteful.

The post Error-Free Computing: Unums Save Both Real and Virtual Battles appeared first on VR World.

As the processing power of the world’s fastest high-performance computers gets faster and faster, we eventually need to think about an era after the processing speed arms race, argued John Gustafson at the Supercomputing Frontiers 2015 conference in Singapore on Tuesday.

Gustafson said that the big challenge for the future of HPC is not necessarily faster processors, but more accurate processors. Using the metaphor of HPC doesn’t need a faster horse, but rather start thinking about a world “post-horse” era, Gustafson proposed moving beyond floating point rounding numbers — which he described as having become sloppy — to something called the universal number or “Unum”.

Unum, as Gustafson first proposed in his book The End of Error, is a new way to represent numbers that’s more accurate than the floating point estimate found in the IEEE 754 standard which Gustafson hopes it would ultimately replace. The IEEE 754 standard is based on the 101×10^2 floating point (which adds up to 64-bits) first introduced by L. Torres y Quevedo in Madrid in 1914.

Unums, which are 29-bits, contain metadata that allows for a longer and more in-depth answer rather than the rounding that floating point integers contain — including the overflow and underflow that goes along with it. Unums also obey algebraic laws and are safe to parallelize. With the mathematically complex problems that comes with the parallelism and sheer power found in modern HPC clusters, complex physics equations are reduced to mere “guesswork”.

The use of rounding can lead to disastrous results. Gustafson gave the example of how during the first Gulf War the 24-bit integer clock used in the Patriot missile batteries miscalculated the approach of a Scud missile by 0.34 seconds — killing 28 and injuring 100. The reason why the missile launched late is because of integer crowding. This came from the inaccuracy of the computer’s system clock due to it multiplying the time from milliseconds to seconds by multiplying 1/10. The 1/10 value was chopped after 24 decimal points. As the system had been on for 100 hours, the continued decimal chopping made the system continually less accurate. When dealing with missiles that travel hundreds of meters per second, this inaccuracy is unacceptable.

The other advantage of Unums is that due to their shorter float size, they take less external memory bandwidth to process. For a data center the largest single line item is its power bill. If power can be saved for the lowest level, for things like RAM calls, this would add up substantially in a massive data center. The US Department of Energy wants vendors to be able to produce an exascale system by 2019-2020 that uses less than 20 MW and to do this power savings has to happen everywhere.

Gustafson said that the next steps to get Unums to go “mainstream” is to convert the Mathematica C library into Unums. After that a strictly Unum compatible FPGA will need to be created. These are the first steps to the long road to a fully Unum compatible CPU.

For more on Unums, Gustafson’s book The End of Errors is worth a read.

The post Supercomputing Frontiers 2015: The 101×10^2 Problem. Solution: Unums appeared first on VR World.