Apart from the family which migrated to Yorkshire, the main pre-19th grouping of Cayleys seems to have been in East Sussex. In medieval times, the de Cailly family owned extensive lands north of the Sussex part of the South Downs, so it is possible that these Cayleys are descended from the same family that gave rise to the Cayleys of Brompton, Yorkshire. There is, though, no proof of this.
The Cayleys seems to have settled at Heathfield, a village in the Weald: there are a number of records of baptisms, marriages and burials at the parish church of All Saints in the 18th century. In the previous century, two, a Thomas Cayley and a William Cayley, moved to Lamberhurst, just across the boundary with Kent – Thomas’s will mentions that his older brother Edward Cayley was a yeoman of Heathfield.
By 1803 one part of the family was based at nearby Maresfield, some ten miles west of Heathfield. There, in that year, Abraham Cayley was born. He earned his living as an agricultural labourer, turnpike keeper and gamekeeper in Sussex before moving to Bath in Somerset, where he was a gardener at the Pulteney Hotel, which still operates. He and his wife Eliza Tester had ten children.
- Three of the daughters moved to London. There Harriet Cayley married John Henry Hickman in 1864; Louisa Cayley married Isaac Fell in 1868; and Martha Cayley married James Day in 1873.
- One son, Jacob Cayley, moved to Bristol, Gloucestershire, where he became a stationary engine driver – that is, he operated a steam engine in a factory. He married Frances Jane Tutton in about 1869. They had six children:
- Isaac Tester Cayley (b. 1871), who emigrated to Canada in 1911 (there are descendants in both Canada and the United States);
- Rosina Minnie Cayley (b. 1873), who stayed single and worked first as a domestic servant and then as a dressmaker in Bristol, Gloucestershire, dying in Weston-Super-Mare, Somerset in 1940;
- Bessie Sarah Louisa Cayley (b. 1875), who worked as a bonnet-maker in Bristol, where she married Ernest Charles Frank Cole in 1900;
- Alice Eliza Cayley (b. 1878), a tailoress in Bristol who stayed single and died in hospital in Axbridge, Somerset in 1960;
- Alfred Jacob Cayley (b. 1881), who worked variously as a cycle maker and an insurance agent in Bristol before marrying Emily Mustoe in Derby in 1917 – he died in Hendon, Middlesex in 1940;
- and Emily Mabel Cayley (b. 1885), who in her teens worked as a stationery packer in Bristol, and married Harold Player in 1909, dying in Weston-Super-Mare in 1968.
- Another son, Benjamin Cayley, married Emma Burge in Bath in 1873, and moved back to Sussex to work as a gardener in Hove before emigrating to Omaha, Nebraska in the 1880s. His brother Alfred Cayley married Adelaide Markwick in Bath in 1877 and moved with him to both Hove and Omaha, working as a labourer. Both Benjamin and Alfred have descendants living in the United States.