- Video Views: 292
- Published On: 2022-04-21 12:00:07
- Video Published/Author: Chemometrics & Machine Learning in Copenhagen
- Video Duration: 00:58:55
- Source: Watch on YouTube
By Harald Martens
How to make sense of Quantitative Big Data? Modern multi-channel measurements of real-world complexity-, for instance from a hyperspectral or thermal video camera, give informative, but overwhelming streams of raw data. To convert such data into meaningful and reliable information, some sort of mathematical modelling is necessary. But which type of mathematical modelling should we use? Exact deduction based on detailed, purely theory-driven modelling alone (using classical differential equations etc), will fail if our understanding of the causal mechanisms is incomplete. Approximate induction based on purely data-driven modelling alone (using black box ANN machine learning etc) will fail if we don’t have the right training data.
In our Big Data Cybernetics group at NTNU, and in the NTNU spin-off company Idletechs AS, we develop an alternative approach. “Interpretable machine learning with an eye for the physics”1 is what I like to call it. Based on Big Data measurements (industrial & space cameras etc) and our limited prior understanding, this hybrid multivariate approach combines four levels of subspace modelling: 1) To quantify variation sources that we think we already understand: Theory-driven multivariate pre-processing, directly or via multivariate metamodels. 2) To discover unexpected, but clear covariation components: data-driven multivariate “chemometrics”. 3) To “sweep the floor” for strange, unmodelled phenomena: Traditional machine learning by ANN etc. 4) To correct for alias errors: Rotate the obtained state variables from stages 1,2,3) into more likely causal structures, e.g. by Varimax, ICA etc., preferably in the setting of cybernetic control theory