Hugh de Caly, who died in 1286, may be the first member of the family known for certain to have held lands in Yorkshire – in his case, the modest amount of a quarter of a knight’s fee at Richmond. But he and his immediate descendants still had their interests concentrated in Norfolk. His grandson John de Cailly was Sheriff of Norfolk and Suffolk in the 1330’s, and kept Norwich Castle for Edward III. It was John de Cailly’s son John Cayley, who, while retaining major holdings in Norfolk, made North Yorkshire his main base, settling at Normanton, probably because of the suitability of the countryside for the profitable activity of rearing sheep: sheep’s wool was one of the mainstays of the English economy at the time. This was probably in the late 14th century, and John Cayley was one of the first members of the family whose surname we find spelt in the modern way.
From then until the second half of the 16th century dating becomes tricky, largely because the same first names – particularly John and William – keep recurring and it is not clear how many generations there were or who begat whom. On the Dissolution of the Monasteries, the family shared with one other the right to nominate the incumbent of the parish of Thormanby; and probably the family base moved there by the end of the 15th century. About this time Edmund Cayley had homes at Thormanby and in York. There were other family holdings at Malton and Sowerby. Sowerby, Thormanby and Malton continued to have Cayley connections for several centuries.
The move to Brompton-by-Sawdon, inland of Scarborough, occurred later. A descendant of Edmund, one of the many William Cayleys, maried Jean Gouldthorp, daughter of a wealthy York merchant Richard Gouldthorp, who was mayor of York in 1588. It was William’s elder son, Edward (died in 1642), who purchased the family estate at Brompton (possibly with money Jean Gouldthorp inherited from her father) and settled there, while his younger brother James stayed at Thormanby. Edward was the father of the first Cayley baronet, Sir William Cayley, and the senior branch of the Cayleys made Brompton their seat. Members of the family still live there today.