Machine Learning and First Principle Models for Process Optimization

Machine learning and first principle models are two widely discussed approaches for process optimization nowadays. First principle models, which are also referred as dynamic models, have long been one of the most powerful tools to optimize manufacturing processes. In recent years, machine learning, a subfield of artificial intelligence, has also become increasingly popular for process optimization. In this blog post, we would like to discuss the pros and cons of each approach, and cases where it might be more appropriate to use one over the other, or a combination of both. 

It is without doubt that first principle models are great solutions when the underlying principles are well understood and the system being modeled is relatively simple. On the other hand machine learning for manufacturing process optimization is very useful when there is sufficient data and when the underlying principles are not easy to understand or too complex to be modeled with first principles. And for many cases in between, the two approaches also complement each other well: 

1. First principle models typically rely on deep expertise from process scientists or engineers and can be quite time-consuming to develop and maintain. In contrast, machine learning models can learn from data without much understanding of the underlying principles that is driving the process. And the pipeline for machine learning models can be the same for various processes. Hence, it is not only easier to build models for one process with machine learning, but also with much less effort to scale from that process to other new processes. 

2. Machine learning models are sensitive to data quality while first principle models can be more robust by design. Even though there are ways to mitigate the risk of bad data, the concept of “Garbage in Garbage out” is generally true for machine learning models. First principle models, on the contrary, can minimize the influence of data with deeply incorporated human knowledge about the process. 

3. First principle models offer more transparency and interpretability while machine learning needs an additional layer for model interpretation. First principle models provide the exact mathematical equations but need to make more assumptions and may be too difficult to build accurately for complex processes. And since machine learning models approximate the process dynamics and its explanation, it can effectively scale to arbitrarily complex processes.  

4. Machine learning models automatically improve with more and more data going into the data pipeline. There are also techniques like transfer learning and meta learning that can boost the performance for one process given the models trained on other processes. The learned patterns and relationships can also be used to adjust, fine tune and improve the first principle models.  

Therefore, both approaches can benefit significantly from each other and the key to better process optimization is working out an efficient way to maximize the benefits realized by using both.  

Fortunately, we can have both approaches in our toolkit for the best benefits. In fact, combining the two can achieve impressive results. Our research showed that machine learning can increase batch yield by 7.2% with only 5 data points from the recipes developed with first principles, and by 17% with a dataset containing just 10 batch runs. Machine learning can also achieve a better economical objective by training on data collected from a MPC controller developed with first principles, as shown here. And first principle models have been used to advance the research of the machine learning based process control algorithms, as explained in this paper.  

Moreover, the speed of innovation and breakthroughs in AI/Machine Learning is eye-opening. Just the past year alone has seen many astonishing advancements like Stable Diffusion and ChatGPT. Most of them mark the rise of foundation models which are trained with various data sources and can be highly adaptable, easily scalable, more robust, and more cost efficient for many new applications. We believe that it will shape the future of manufacturing and help us achieve autonomy much more quickly.