The earliest records of a Jewish community in Kaharlyk are from 1847. By 1897, the Jewish population had reached 1,414. The community was severely affected by the pogroms during the civil war in 1919. The last known burial occurred at the site in 1991.
The cemetery can be found around 2 kilometres from the centre of Kaharlyk on Pervomayskaya Street, behind an oil depot. ESJF surveyors in spring 2015 found no signage or fencing at the site. Only a few gravestones were visible, some of which were toppled or broken. The cemetery contains no known mass graves. Some of the tombstones have Soviet-style portraits on them, while others are surrounded by individual metal fences. The cemetery was vandalised during World War II.