An investigation is presented of the use of finite element models in the time domain to represent a load moving on a railway track on a flexible ground. A systematic study is carried out to compare different sizes and shapes of finite element mesh, different boundary conditions intended for suppressing reflections from the truncated model boundaries, and different models of soil damping. The purpose is to develop guidance to assist in selecting appropriate finite element models for moving load problems. To prevent reflections from the boundaries of the finite domain two approaches are compared. A 40 m radius hemispherical finite element mesh has been used first with infinite elements around the perimeter. This approach gives good results for a point harmonic load at the centre of the domain but some problems are highlighted when it is used for moving load calculations. An alternative approach has therefore been investigated based on a cuboid mesh. The base was fixed to prevent rigid-body motions of the model and, rather than use infinite elements at the sides, these were also fixed. It is shown that, provided that a suitable damping model is used, the spurious reflections from the sides of the model can be suppressed if the model is wide enough. On the other hand, if infinite elements are used, the calculations are found to be considerably more costly with little added benefit. Different models of soil damping are also compared. It is shown that a mass-proportional damping model gives a decay with distance that is independent of frequency, making it particularly suitable for this application. The length of model required to achieve steady state has been investigated. For a homogeneous half-space it is found that the required length increases considerably in the vicinity of the critical speed, up to 130 m in the present example, whereas for the layered ground a more modest length is sufficient for all speeds.