Machine Learning-Driven Drug Discovery: Prediction of Structure-Cytotoxicity Correlation Leads to Identification of Potential Anti-Leukemia Compounds

In vitro cytotoxicity screening is a crucial step of anticancer drug discovery. The application of deep learning methodology is gaining increasing attentions in processing drug screening data and studying anticancer mechanisms of chemical compounds. In this work, we explored the utilization of convolutional neural network in modeling the anticancer efficacy of small molecules. In particular, we presented a VGG19 model trained on 2D structural formulae to predict the growth-inhibitory effects of compounds against leukemia cell line CCRF-CEM, without any use of chemical descriptors. The model achieved a normalized RMSE of 15.76% on predicting growth inhibition and a Pearson Correlation Coefficient of 0.72 between predicted and experimental data, demonstrating a strong predictive power in this task. Furthermore, we implemented the Layer-wise Relevance Propagation technique to interpret the network and visualize the chemical groups predicted by the model that contribute to toxicity with human-readable representations.Clinical relevance - This work predicts the cytotoxicity of chemical compounds against human leukemic lymphoblast CCRF-CEM cell lines on a continuous scale, which only requires 2D images of the structural formulae of the compounds as inputs. Knowledge in the structure-toxicity relationship of small molecules will potentially increase the hit rate of primary drug screening assays.