Accident prevention is the key facet of road safety. As the number of road accidents in Bangladesh is rising exponentially, precautionary steps are required to prevent this from happening. Many researchers are working on accident prediction models that are commonly used in road safety research. Artificial intelligence (AI) is particularly important in many real-world applications where data are not consistent and are affected by random changes. In this work, we propose an AI-based driver vigilance system for assisting drivers with accident prevention. The system detects drowsiness of the driver from dash camera using the Convolutional Neural Network algorithm; detects anomalies in the heartbeat using two-layer long short term memory algorithm and detects over-speed using GPS and front camera. The suggested model uses a Neuro-Fuzzy controller to integrate these inputs and generates alerts and controls brakes if necessary using drowsiness, heartbeat and speed variables.