The concept of using sensors on a passing vehicle, rather than on the bridge, is known as 'Drive-by' damage detection. The newly developed traffic speed deflectometer (TSD) is a device used for pavement deflection measurements and is investigated here in numerical simulations as a means of bridge damage detection. A TSD vehicle model containing two displacement sensors is simulated crossing a simply supported finite element beam containing damage simulated as a loss in stiffness of one of the elements. An adapted Cross Entropy optimisation algorithm procedure is proposed to determine the apparent profile, where the displacements recorded by the sensors are used as the inputs. The time-shifted difference in the apparent profile is used as the damage indicator. Results show that this can be reliably used as a damage indicator in the presence of noise and changes in the transverse position of the vehicle on the bridge.