Apart from the family which migrated to Yorkshire, the main pre-19th grouping of Cayleys seems to have been in East Sussex. In medieval times, the de Cailly family owned extensive lands north of the Sussex part of the South Downs, so it is possible that these Cayleys are descended from the same family that gave rise to the Cayleys of Brompton, Yorkshire. There is, though, no proof of this.
The Cayleys seems to have settled at Heathfield, a village in the Weald: there are a number of records of baptisms, marriages and burials at the parish church of All Saints in the 18th century. In the previous century, two, a Thomas Cayley and a William Cayley, moved to Lamberhurst, just across the boundary with Kent – Thomas’s will mentions that his older brother Edward Cayley was a yeoman of Heathfield.
By the mid to late 1700s one part of the family was based at nearby Maresfield, some ten miles west of Heathfield. There John Cayley, husband of Sarah Anne Bradbury, was probably born in about 1768. You can see information about him and his descendants on the page Cayley-Bradbury line.