Complementary features of batteries and supercapacitors can be effectively used in a hybrid energy storage system (HESS). The utilization of the HESS in electric vehicles (EVs) offers many advantages, such as efficient regenerative braking, battery safety, and improved vehicle acceleration. In this paper, a new regenerative braking system (RBS) is proposed for EVs with HESS and driven by brushless DC (BLDC) motor. During regenerative braking, the BLDC acts as a generator. Hence, by using an appropriate switching algorithm, the dc-link voltage is boosted and the energy is transferred to the supercapacitor or the battery through the inverter. The harvested energy can be utilized to improve the vehicle acceleration and/or keep the battery pack from deep discharging while driving uphill. To provide a reliable and smooth brake, braking force distribution is realized through an artificial neural network. Simultaneously, the braking current is adjusted by a PI controller for constant torque braking. To evaluate the performance of the proposed RBS, different simulations and experiments are carried out. The results confirm high capability of the proposed RBS.