The Canal Basin was originally at the western end of the Thames and Medway Canal built in the early 1800s and opened on 14 October 1824. Its lock gates gave entry to the River Thames. The Canal was seven miles long, saving a coastal journey of 46 miles. However, owing to tolls charged and tidal difficulties, it soon became something of a white elephant and was taken over by the South Eastern Railway in 1845. Part of the Canal was later filled in leaving the Basin isolated and now used as a mooring for pleasure craft.
At the western end of the Canal, there stood until 1942 a small cottage, the roof of which was an upturned hull of a boat, with a window in one side. This is held by some to have been the inspiration that led Dickens to invent the house of Peggoty in David Copperfield. Peggoty's house, however, is described as being constructed of a vessel in an upright position.
For many years there stood at a point near the north west corner of the present canal basin what was known as the Round Tree. This was regarded as marking the seaward limit of the Port of London, beyond which the duty on coal entering the port was not chargeable. This was a convenient, rather than an exact landmark, as the eastward end of the parish of Milton is some distance farther east. The tree was damaged by gun practice from the fort towards the end of the 18th century, and early in the 19th century was mischievously set on fire, sailors from a collier lying in the Canal Basin extinguishing the flames. Its end came when it was blown down during a violent storm, with strong winds from the south south west on the night of 5 August 1825.
After the destruction of the tree, an obelisk was set up in its place in 1826 and when excavations were made for its foundation bricks were found similar to those used in the Gravesend blockhouse, from which it seemed that this was the site of the Milton blockhouse. Excavation on the spot by a small group of the Gravesend Historical Society in February 1973 confirmed the existence of these foundations. From early maps earthworks are known to have existed here. The use of this stone as a landmark being no longer needed, it was taken down and lay for many years on the edge of the Canal Basin. In 1892, it was re erected at the entrance to the Gordon Memorial Gardens, but with no mention of its former function.