Tuesday, March 22, 2011

JavaScript Numeric Format - Decimal Precision

Today's discussion about JavaScript number format based on decimal precision. Round a number to a certain number of places you can use JavaScript built-in methods toFixed and toPrecision. This two methods toFixed and toPrecision are part of the Number object. Any browser that supports ECMAScript version 3 should support toFixed and toPrecision.

How to use?
Its very simple and easy to use. Let's see some example which is help us to understand better.

Example : toFixed()

Use toFixed to set precision after the decimal point. It doesn't matter how large the number is before the decimal point. For normal decimal formatting, this is your best option.

// Example: toFixed(2) when the number has no decimal places
// It will add trailing zeros
var num = 10;
var result = num.toFixed(2); // result will equal 10.00

// Example: toFixed(3) when the number has decimal places
// It will round to the thousandths place
num = 930.9805;
result = num.toFixed(3); // result will equal 930.981

Example: toPrecision ()
Use toPrecision when you're setting the overall precision. Here, it matters how large the number is before and after the decimal point. This is more useful for mathematical purposes than for formatting.

// Example: toPrecision(4) when the number has 7 digits (3 before, 4 after)
// It will round to the tenths place
num = 500.2349;
result = num.toPrecision(4); // result will equal 500.2

// Example: toPrecision(4) when the number has 8 digits (4 before, 4 after)
// It will round to the ones place
num = 5000.2349;
result = num.toPrecision(4); // result will equal 5000

// Example: toPrecision(2) when the number has 5 digits (3 before, 2 after)
// It will round to the tens place expressed as an exponential
num = 555.55;
result = num.toPrecision(2); // result will equal 5.6e+2

Floating-point errors

toFixed and toPrecision are subject to floating-point errors.

Here is a test where the starting number is 162.295. The following should show the JavaScript results:

162.30 // toFixed(2)
162.30 // toPrecision(5) 162.29 // toFixed(2)
162.29 // toPrecision(5)

Do they show up correctly as 162.30 in your browser? Most JavaScript implementations will display it as 162.29

Here is basically what happens when rounding 162.295 to two decimal places

num = 162.295
num *= 100 // 16229.499999999998
num = Math.round(num) // 16229
num /= 100 // 162.29

As you can tell, it's in the second step that the number changes from its actual value.

No comments:

Method 'StartWorkflowOnListItem' in type 'Microsoft.SharePoint.WorkflowServices.FabricWorkflowInstanceProvider'

Exception: Method 'StartWorkflowOnListItem' in type 'Microsoft.SharePoint.WorkflowServices.FabricWorkflowInstanceProvider'...