Harwood Brierley described Scarborough harbour somewhat romantically at the opening of the 20th century.
"Then come the tiers of the red roofed fishermen's dwellings ,the steep wynds and stair like alleys ,which reach down to the quays and piers ; heavy built breakwaters ,graced, perhaps, with some white towered lighthouse. The masts and yards of big trawlers and yawls, of small cobles and mules, are seen congregated in the harbour, like Autumnal reeds on the edge of a pond; while out in the bay itself are the red cutch-coloured lugsails of pleasure craft. and beyond, the steely blue of the North Sea, with merchant ships passing to and from other countries,with now and again, the trailing pennons of smoke from distant steamships, low upon the horizon".
"Let us go nearer the town and inspect those quarters wherein remains all that is left of ancient Scarborough. We may thread several continental seaport- like wynds on the rising ground beyond the quay.They circle with the harbour crescent, and serve both as stair and thoroughfare, with narrow shadowy passages and yards running out of them. Curious architectural bits are found, half hidden in small courts; windows seem to have sprung through roofs, balconies overhang succeeding roofs, landings lead into into street doors and second story entrances of the same house. I saw several most dismal shuttered- up tenements, evidently voted insanitary and condemned as dwelling houses for that reason.I should think there must have been fully one cat to every inhabited dwelling,thus showing that finny folk do undoubtedly attract the feline tribe".
"Near the harbour are several 14th or 15th century houses, with beetling upper stories. The captains and skippers of old have lived under their roof trees, and it is not surprising to find that some of them have been inns. At the top of Quay street, there is a smugglers palace, tenanted for several generations by the Kidd family. It is a three storied house, gabled with yellow walls and constructed throughout of ship's timber . with wainscots; on the ground floor is a bay windowed shop and kitchen behind it, with one side of the entrance to the staircase , a cupboard containing a recess of which there is no external evidence .To escape the press(gang), you could be raised by ropes to the roof or lowered into the cellar in which was a door to the next house and traditionally a passage to the other side of castle hill."
Much of Brierley's harbour scene would be removed in clearances between the wars.