The area was occupied by the Woiwurung people before white settlement. The first Europeans in the area were the party of explorers William Hovell and Hamilton Hume who crossed the Yea River near this point in 1824. Their favourable report of the grazing land they had seen led to the settlement of what is now Victoria.
The first settlers in the district were overlanders from NSW who arrived in 1837. Most of the good land along the Goulburn had been taken up by 1839. Yea would later develop on the Murrindindi run (the homestead is still standing to the south of town).
The future townsite was initially known to whites as the Muddy Creek settlement, as the Yea River was called Muddy Creek until 1878.
The town was surveyed and laid out in 1855 with town lots going on sale at Kilmore the following year. It was named after a Colonel Lacy Yea who was killed in the Crimean War.
Gold was discovered in the area in 1859 and a number of smaller mining settlements came into existence at this time, including Molesworth. Yea grew as a service centre to the diggers.
An Anglican church was erected in 1869 and Yea became a shire in 1873, at which time the population was about 250.
When the gold ran out the town survived on the back of farming and timbergetting. The railway arrived in 1883. Yea was promoted as something of a tourist centre in the 1890s with trout being released into King Parrot Creek to attract recreational anglers. A butter factory was built in 1891. There was a proposal in 1908 to submerge the town under the Trawool Water Scheme but it never went ahead.
By 1911 the town's population was 1126 and it is roughly the same today, despite two severe floods in 1934 and 1973 and a major conflagration in 1969.