Decimal being rounded wrong?

Last post 01-28-2008 10:16 AM by inspiringdesign. 5 replies.

Sort Posts:

  • Decimal being rounded wrong?

    01-02-2008, 1:52 AM

    Hello -

    I've got a Microsoft SQL Server 2005 Database running with ASP.NET 1.1/VB .NET interface, I'm having an issue with my parameters.

    I am using a sp to retrieve values, then doing math in vb .net, then using a sp to input the modified values back.

    I am aware of a bug with decimals if nulls are passed in SQL 2000 - that's not the case here.  Anyway, here's some code:

     

    Dim NM As New Decimal

    Dim pNM As New SqlParameter("@ANBP_NM", SqlDbType.Decimal, 18, 2)

    pNM.Direction = ParameterDirection.Output

    oCM.Parameters.Add(pNM)

    oCM.CommandText = "ANBP"

    oCM.CommandType = CommandType.StoredProcedure

    oCM.Connection = objCN

    oCM.Connection.Open()

    oCM.ExecuteNonQuery()

    oCM.Connection.Close()

    NM = pNM.Value

    Dim b As New Decimal

    b = NM

    b += 0.01

     

    My connection string and all that are correct - I use the same for everything and everything works fine - including the sending of the processed data (b += 0.01) - because in my database, the value is showing as "NM".01 - the problem is, it's rounding NM to the nearest whole number, up or down, and completely negating the decimal - then ADDING .01 as the decimal.  It's replacing the decimal.  I've tried this with =, +=, subtraction variants (they're possible, I've just abandoned them as they're the long way), my sp's are all declared properly with decimal(18,2) OUTPUT (where appropriate), I've even thrown in some Math. functions - no luck.

     

    Any ideas???

  • Re: Decimal being rounded wrong?

    01-03-2008, 11:44 AM
    • Contributor
      2,657 point Contributor
    • DkUltra
    • Member since 03-19-2007, 2:06 PM
    • South Dakota / Nebraska
    • Posts 518

    Pull the decimal back as a string and convert to decimal in the codebehind and then add

    Hope this helps 

    DK 

     

  • Re: Decimal being rounded wrong?

    01-25-2008, 1:01 AM

    No - that doesn't make a difference, I've tested further and the stored procedure is pulling back as though the number didn't have any trailing decimal (despite EVERYWHERE BEING DECLARED 18,2) - it sounds very familiar to this KB article

     http://support.microsoft.com/kb/892406

    But - my hosting company uses SQL Server 2005 - and they're pretty much refusing to do anything on their end, despite the fact that it is most definitely on their end.  Any help - anyone?

  • Re: Decimal being rounded wrong?

    01-25-2008, 1:27 AM
    • All-Star
      62,414 point All-Star
    • TATWORTH
    • Member since 02-04-2003, 8:34 AM
    • England
    • Posts 12,177
    • TrustedFriends-MVPs

    I have found that I had to explicitly round before saving a value:

        #region " Decimal Functions "
        /// <summary>
        /// Adjust decimal value to a (sanitized) number of decimal places
        /// </summary>
        /// <param name="decIN">Decimal value</param>
        /// <param name="iPlaces">Number of decimal places</param>
        /// <returns>Adjusted value</returns>
        public static decimal DecimalAdjust(decimal decIN, int iPlaces)
        {
          if (iPlaces < 0) iPlaces = 0;
          if (iPlaces > 28) iPlaces = 28;
          return decimal.Round(decIN, iPlaces);
        }
        #endregion

    Don't forget to click "Mark as Answer" on the post that helped you.
    This credits that member, earns you a point and marks your thread as Resolved so we will all know you have been helped.
  • Re: Decimal being rounded wrong?

    01-25-2008, 1:36 AM
    • All-Star
      62,414 point All-Star
    • TATWORTH
    • Member since 02-04-2003, 8:34 AM
    • England
    • Posts 12,177
    • TrustedFriends-MVPs

     A further thought - set up a table with the following column types:

    • Identity integer
    • String to take a formatted number of what is inserted into decimal
    • Decimal 
    Insert values from 0 to 1.00 incrementing by 0.01.  Display the result in a datagrid. This will give clearer indication of what is going on. In the select, display the decimal column both directly and via a CONVERT(VARCHAR(10), Decimalcol) AS XDEC.
    Don't forget to click "Mark as Answer" on the post that helped you.
    This credits that member, earns you a point and marks your thread as Resolved so we will all know you have been helped.
  • Re: Decimal being rounded wrong?

    01-28-2008, 10:16 AM

    What I ultimately had to do was create a second field in the table that is a varchar mimic of the decimal field - and yes, I need both because I sort by the decimal (the varchar will not mathematically sort right) but I compute with the varchar by converting string to double and back.  There's an MS KB article that says it's a server issue that requires updates - but my hosting company refuses to help me in this, so I have to have extra fields and write extra code everywhere.  I will be shopping for a new company.

Page 1 of 1 (6 items)