The island is of great spiritual significance to the Gunghandji Aboriginal people who live on the mainland. They believe that during the Dreamtime a person named Gulnyjarubay came to the area and named the island Gabara meaning 'the fist'. It was a way of saying that the arm and the wrist had been submerged and all that was left was 'the fist' standing up out of the water.
The Gunghandji Aboriginal people have used Fitzroy Island for thousands of years as place for hunting and fishing. Today they still have an active involvement with the island.
In 1770 Captain Cook became the first European to sight Fitzroy Island. He named the island after the Duke of Grafton who was the Prime Minister when the Endeavour left England on its epic voyage through the Pacific.
Throughout the nineteenth century the island was visited by itinerants. By the late 1800s a beche de mer (sea cucumber) industry was operating on the island. It was also around this time (1876) that the island became a quarantine station for the Chinese who arrived in north Queensland eager to try their luck on the goldfields around the Palmer River.
By the early 1900s there was an Anglican Aboriginal Mission at Yarrabah and by 1929, after a number of shipwrecks, the first lighthouse was built. It was little more than a navigation light and was established on Little Fitzroy only to be replaced by a more substantial lighthouse on the main island in 1943. The current lighthouse, a far cry from the solidity of the great 19th century edifices, was built in 1970 and continued to operate until 1991 when an automatic system was established.