This paper presents a new modelling methodology for compensation of the thermal errors on a gantry-type 5-axis CNC machine tool. The method uses a “Grey Neural Network Model with Convolution Integral” (GNNMCI(1, N)), which makes full use of the similarities and complementarity between Grey system models and artificial neural networks (ANNs) to overcome the disadvantage of applying either model in isolation. A Particle Swarm Optimisation (PSO) algorithm is also employed to optimise the proposed Grey neural network. The size of the data pairs is crucial when the generation of data is a costly affair, since the machine downtime necessary to acquire the data is often considered prohibitive. Under such circumstances, optimisation of the number of data pairs used for training is of prime concern for calibrating a physical model or training a black-box model. A Grey Accumulated Generating Operation (AGO), which is a basis of the Grey system theory, is used to transform the original data to a monotonic series of data, which has less randomness than the original series of data. The choice of inputs to the thermal model is a non-trivial decision which is ultimately a compromise between the ability to obtain data that sufficiently correlates with the thermal distortion and the cost of implementation of the necessary feedback sensors. In this study, temperature measurement at key locations was supplemented by direct distortion measurement at accessible locations. This form of data fusion simplifies the modelling process, enhances the accuracy of the system and reduces the overall number of inputs to the model, since otherwise a much larger number of thermal sensors would be required to cover the entire structure. The Z-axis heating test, C-axis heating test, and the combined (helical) movement are considered in this work. The compensation values, calculated by the GNNMCI(1, N) model were sent to the controller for live error compensation. Test results show that a 85% reduction in thermal errors was achieved after compensation.