Football started in Abergavenny in the 1890s with Abergavenny Rangers, who became Abergavenny Rangers Thursdays before dropping the Rangers name in 1908. The Thursday name originated from works in Welsh Towns having half day on Thursdays, enabling football and other social clubs to form.

Abergavenny Thursdays joined the Welsh League in 1950, winning Division 2 Eastern Section, losing just 1 game and scoring 148 goals in 34 games with 30 wins. They were crowned Welsh League champions in 1958-59 and 1959-60, and were runners up in 1962-63, 1963-64 & 1964-65. Relegations were suffered though in 1972-73 & 1973-74, but the club bounced back and won the Welsh League again in 1990-91 & 1991-92 to become founder members of the Konica League of Wales.

However, this was only short-lived, and the Thursdays were relegated back to the Welsh League after just one season. Unfortunately, the good times never returned to Pen-y-Pound, and they dropped through the Gwent County League until an agreement was reached with Govilon AFC in 2014.

Govilon had the first team but were unable to progress further up the Welsh Pyramid system at their village ground, while Thursdays had the stadium and youth set up. The agreement saw Thursdays dropping out of the league with Govilon becoming known as Abergavenny Town. Six of the current first team and most of the reserves came through Thursdays Juniors.

Most village communities around Abergavenny also hosted local football teams, and Govilon had began over 100 years ago. Playing mainly at District level, they had won the Gwent Central League gaining entry to the Gwent County League in 2007-08. They also won the Gwent Amateur and Gwent County Challenge Cup.

Abergavenny Town quickly built an excellent fan base by producing an outstanding standard of football winning the Gwent County League Division 1 after dropping only two league points all season to gain promotion to the Welsh League.

Pen-y-Pound became a formidable venue for the club which had been built in the 1930’s. In 1963 the pitch was used as a helicopter landing site for John Lennon. The Beatles were playing in the Town and Lennon had been in London making an appearance on the BBC television show Juke Box Jury. He landed at 9.50 pm.

Town continued to progress, winning promotion from Division 3 as runners up to Pontypridd Town in 2015-16. The club also won the FAW Amateur Trophy beating Sully Sports 1-0 in the final played at Dragon Park, Newport.

2016-17 saw Abergavenny finish 11th in Division 2 with long time servant and Manager Steve ‘Butch’ Morgan leaving the club after 20 years of involvement at Govilon and Abergavenny. The following season saw the club again finish in 11th spot, while 2018-19 saw the club miss out on the final day of the season losing despite taking the lead at Caerau (Ely), who sealed the final promotion spot.

Abergavenny finished 2019-20 in 13th position on a points per game average due to the pandemic. However, after achieving a Tier 3 license, Abergavenny Town take their place in the Ardal Southern South East Division for 2021-22.