The original Greek-Revival style courthouse was constructed in 1835 at the corner of Court and Main Street, the two oldest streets in Rutherfordton, aptly named due to their location. The building featured a cupola containing a bell.  It stood as a landmark in the town until it burned on December 24, 1907.  A new courthouse was built further north on Main street in 1908 in a lot previously occupied by the Isothermal Hotel which burned in 1895 but was reconstructed further north on Main Street.

The new structure was a large rectangular building with Corinthian porticos at the front and rear and a three-stage cupola that contained clocks that faced in all four directions.  The building was constructed of limestone and brick and the hallways and staircases were sheathed in marble from Tennessee. Construction began in March of 1908 and took seven months to complete.

Given the pressure to complete the new courthouse, after the original structure burned, it was built quickly to fill the needs of the county.  Through the next seventeen years there are several recorded complaints about the inadequacies of the courthouse.  These complaints included; a leaky roof, probably due to the number of angles and levels, small offices that were not conducive to the work that needed to be done such as record filing, and insufficient toilets. This led to the construction of the new courthouse that stands today in 1926.

An editorial in The Rutherfordton Sun state, “it is time that a new, modern Court House be built that is in keeping with the great progress of the county. We should build a court house that every citizen can be proud of and one that will stand for many years to come.”