MINIMIZING ERROR IN CALIBRATING SPECTROMETERS – Part 2. Method Development – Choosing Preprocessing, Model Complexity, and Algorithm Selection

Choosing the best method will optimize how future spectra will be processed. The calibration procedure for spectroscopic models begins with selecting appropriate preprocessing methods. Different analyzer technologies (e.g., NIR vs. Raman), spectrometer manufacturers, and chemometric algorithms can influence the optimal choices. To evaluate preprocessing methods, Root Mean Squared Error of Cross Validation (RMSECV) is commonly used, offering a reliable measure of combined bias and precision error.

In one example, various combinations of preprocessing techniques (e.g., derivatives, scatter correction, normalization) were tested. The results showed that some combinations significantly outperformed others, with the best approaches selected based on minimizing RMSECV with the fewest model factors.

Partial Least Squares (PLS) regression is the standard algorithm due to its widespread integration and effectiveness in handling errors from both spectroscopy and lab sources. However, PLS assumes linearity, which doesn’t always apply. In such cases, alternative methods like Locally Weighted PLS (LWR-PLS) can provide better performance by addressing non-linearity through localized modeling.

Finally, method development is a one-time effort. Once a spectral data processing method is chosen, it generally remains fixed throughout the instrument’s use.

Here is the link to the full presentation on Minimizing Error in Calibrating Spectrometers from ATC 2025 Conference. Also available in Youtube video from 00:00 to 22:00.

MINIMIZING ERROR IN CALIBRATING SPECTROMETERS – Part 1. Accuracy vs Precision: Misconceptions of error that influences decisions

The terms “accuracy” and “precision” are often confusing. Accuracy refers to how close a result is to a known value, while precision indicates the consistency of repeated results. Ideal measurements are both accurate and precise. However, it’s possible to have high precision with poor accuracy, or low precision that still averages out to an accurate result over many trials.

Application to Octane Measurement:
In octane rating analysis, precision is a challenge, especially with the reference octane engine, which is not always precise despite being used to define accuracy. Essentially, the octane engine represents the “Not precise, maybe accurate” portion of picture. Repeated measurements of the same gasoline sample show a normal distribution of values with a standard deviation of ~0.25 octane units, meaning 95% of results fall within ±0.5 units.

The calibration process, then, is charged with mapping a very precise spectrum to a far less precise reference value and calibration must account for this. One can run multiple engine tests on the same sample and use the average value to reduce error—error decreases with the square root of the number of runs. In dealing with a system where the reference is imprecise, there is more that we can do.

Here is the link to the full presentation on Minimizing Error in Calibrating Spectrometers from ATC 2025 Conference. Also available in Youtube video from 00:00 to 22:00.

Unlock Data. Discover Systems. Save Big

IMAGE '25 MeetingVenue: George R. Brown Convention Center, Houston, TX

Date: August 24, 2025

Short Course:  SC-01: Chemometric Tools to Establish Petroleum Systems, Predict Physical Properties, and De-Convolute Mixed Production

Special Software Discount for SC-01 Attendees

Join Infometrix at IMAGE 2025 and discover how multivariate data analysis is transforming petroleum geoscience.

Dr. Kenneth Peters from LSB NExT Training and Brian Rohrback from Infometrix will be leading the short course Sunday, August 24th at the George R. Brown Convention Center.

This one-day course demonstrates how chemometric techniques, using real-world datasets, can enhance interpretation of geochemical, petrophysical, and production data – all powered by Infometrix Pirouette® software.

Exclusive Offer:
Register for SC-01 and receive a discount on Infometrix software license, optimized for the workflows presented in the course.

Expand your skillset. Enhance your toolkit. 
Learn from experienced instructors and leave with the power of advanced analytics at your fingertips.

Register today and claim your software discount.


Infometrix: Turning complex data into confident decisions.

 

SciX 2025 Conference – On Machine Learning, PLS, and Local Weighting

Venue: Northern Kentucky Convention Center, Covington, KY

Date: Oct 5 – 10, 2025

Brian Rohrback will be presenting in the session on Artificial Intelligence and Machine Learning in Process Analytical Technology (PAT). See abstract below.

On Machine Learning, PLS, and Local Weighting

AUTHOR: Brian Rohrback
ABSTRACT:
A dozen years ago, Infometrix embraced the target of completely automating the installation and maintenance of any optical spectrometer and for any application. Ultimately, the goal is to identify a generic machine learning approach that can be taken to mimic the results that an experienced chemometrician would achieve if charged with producing an optimized model.  The idea has been presented in numerous publications and this specific work was triggered initially by Workman et al. (1995). Clearly, the chemometrics tasks can be broken down, assigning best practices procedures for each. One part of this process is choosing the algorithmic approach. Partial Least Squares (PLS) is the workhorse and is incorporated into nearly every spectroscopy system.   In cases of non-linearity, a locally weighted application of PLS will avoid the failures of non-linear methods.  Local models can also simplify model maintenance as conditions (spectrometer, ingredients, unit operations) change.

Automation of Local Regression Model Building for Spectroscopic Data – Journal of Chemometrics

ATC 2025 Conference, Visit Booth 420

ATC 2025 Conference, Infometrix Booth 420Venue: Galveston Convention Center, Texas

Date: April 28 – May 2, 2025

Booth: #420

The Analyzer Technology Conference for 2025 is less than a month away.  Join Infometrix in booth 420 for presentation on Ai-Metrix and the automation of chemometric calibrations. Meet with industry peers for informal discussion on new and innovative analyzer techniques, developments, and applications for process and laboratory measurements. Most recently, the fundamentals of quality control employing optical spectroscopy were presented and the paper is available for download.

ATC 2024 Rohrback Paper

Chemometric plus Automation equals Machine Learning and Best Practices.