State-of-the-Art Measurement Analysis Software, Training and Consulting Services
Celebrating over 28 YRS of Excellence and Innovation 1987-2015



Products  Software UncertaintyAnalyzer 3.0 Software Comparison


Uncertainty Analysis Software Comparison


As a developer and marketer of uncertainty analysis tools, it is important for Integrated Sciences Group to periodically assess the capabilities of similar software applications.  A summary of our software assessment is provided below.  Detailed results of our software assessment were presented at the 2004 Measurement Sciences Conference in a paper titled "A Comprehensive Comparison of Uncertainty Analysis Tools."  An updated copy of this paper in Adobe Acrobat pdf format can be downloaded via the hypertext link below. 


 Software Comparison Paper


Software Comparison Summary

The focus of this comparison was limited to applications or tools that apply the methods contained in ISO/TAG4/WG3 (the GUM) and ANSI/NCSL Z540-2-1997 (the U.S. Version of the GUM). Software using Monte Carlo simulation, or other alternative analysis methods, were not included in this assessment.  The software applications and tools evaluated herein were also limited to those readily available via Internet download (e.g., demos or freeware) or currently available for purchase.  In all, two freeware applications and five commercial products were evaluated.  Basic software information is summarized in the Software Summary Table.


In order to assess the features and capabilities of individual software applications, we must consider how they address the basic steps in conducting an uncertainty analysis.  The general uncertainty analysis steps are outlined below along with hypertext links to corresponding software comparison tables.


1. Define the Measurement Process

The first step in any uncertainty analysis is to identify the physical quantity whose value is estimated via measurement. This quantity may be a directly measured value, such as the weight of a 1 gm mass or the output of a voltage reference.  Alternatively, the quantity may be indirectly determined through the measurement of other variables, as in the case of estimating the volume of a cylinder by measuring its length and diameter.  The former type of measurement are called "direct measurement," while the latter are call "multivariate measurements."


For multivariate measurements, it is important to develop an equation that defines the mathematical relationship between the quantity of interest and the measured variables.  At this stage of the analysis, it is also useful to briefly describe the test setup, environmental conditions, technical information about the instruments, reference standards, or other equipment used and the procedure for obtaining the measurement(s).  This information will help identify the measurement process errors.


A overview of the main software features and capabilities, including the entry and display of analysis information is provided in the Main Software Features Table.


2. Identify the Error Sources and Distributions

Measurement process errors are the basic elements of uncertainty analysis.  Once these fundamental error sources have been identified, we can begin to develop uncertainty estimates.  The errors most often encountered in making measurements include, but are not limited to the following:

  • Bias in the measuring device and/or quantity being measured.

  • The error associated with repeat measurements.

  • The error resulting from the finite resolution of the measuring device and/or quantity being measured.

  • The error introduced by variations in environmental conditions or by correcting for environmental conditions.

  • The error introduced by digitizing an analog signal.

  • The error introduced by the person making the measurements.

Another important aspect of the uncertainty analysis process is the fact that measurement errors can be characterized by probability distributions.  The distribution for a type of measurement error is a mathematical description that relates the frequency of occurrence of values with the values themselves.  A comparison of how the various software applications assist the user in identifying and describing measurement process errors can be found in the Errors and Distributions Table.


3. Estimate Uncertainties

Our lack of knowledge about the sign and magnitude of measurement error is called measurement uncertainty.  And, since measurement errors follow probability distributions, they can be described in such a way that their sign and magnitude have some definable probability of occurrence.  In this context, uncertainty is defined as the square root of the variance of the measurement error distribution.  The variance is the mean square dispersion of the error distribution.


There are two approaches to estimating variance and uncertainty.  Type A estimates involve data sampling and analysis.  Type B estimates use engineering knowledge or recollected experience of measurement processes. Given the marked difference in these approaches, it is best to separately evaluate the software capabilities for each type of uncertainty estimate.


A comparison of software features and capabilities for conducting Type A uncertainty estimates is given in the Type A Estimates Table.  A comparison of software features and capabilities for conducting Type B uncertainty estimates is given in the Type B Estimates Table.


4. Combine Uncertainties

The next step in the analysis procedure is to combine the uncertainty estimates for the measurement process errors.  To do this, we invoke the variance addition rule, which requires that we account for any correlations between measurement process errors or any cross-correlations between measurement components.  For multivariate analyses, it is also important that the uncertainties are weighted or multiplied by the appropriate sensitivity coefficients.  We must also estimate the effective degrees of freedom for the combined uncertainty.  A comparison of software features and capabilities for combining uncertainties is given in the Combining Uncertainties Table.


5. Report the Analysis Results

The last step in the uncertainty analysis procedure is to report the results. When reporting the results of an uncertainty analysis, Section 7 of the GUM recommends that the following information be included:

  1. The estimated value of the quantity of interest (measurand) and its combined uncertainty and degrees of freedom.

  2. The functional relationship between the quantity of interest and the measured components, along with the sensitivity coefficients.

  3. The value of each measurement component and its combined uncertainty and degrees of freedom.

  4. A list of the measurement process uncertainties and associated degrees of freedom for each component, along with a description of how they were estimated.

  5. A list of applicable correlations coefficients, including any cross-correlations between component uncertainties.

It is also a good practice to provide a brief description of the measurement process, including the procedures and instrumentation used, and additional data, tables and plots that help clarify the analysis results.  A comparison of the software reporting features and capabilities is given in the Reporting the Analysis Results Table.



  Home  |  Company  |  Products  |  Services  |  Training  |  Updates  |  Freeware  |  Articles  |  Help  |  Links 


Integrated Sciences Group
14608 Casitas Canyon Road Bakersfield, CA 93306
Phone 1-661-872-1683 FAX 1-661-872-3669


Privacy Policy     Return Policy     Shipping Info


Page Updated February 12, 2015