Deep learning-based visual analytics play an important role in the automation of industrial processes, performing exceptionally well compared to traditional machine learning approaches. However, there are several challenges faced by deep learning methods when employed in industrial applications, such as noisy data, lack of labeled data, computational complexity, and adversarial attacks. In this context, this paper presents a case study on deep learning-assisted coastal crane automation. The case study presents real-time container corner casting recognition for efficient loading and unloading of the container. The proposed crane casting recognition consists of a lightweight dehazing method for pre-processing noisy videos to remove haze, fog, and smoke, and end-to-end corner casting recognition by applying a recurrent neural network along with long short-term memory (LSTM) units. The proposed method is real-time, and is verified in the field with an average rate of accuracy of 96%.