Drivers' Real-Time Drowsiness Identification Using Facial Features and Automatic Vehicle Speed Control

The road crash is one of the significant problems that is of great concern in today's world. Road accidents are often caused by drivers' carelessness and negligence. The drowsy condition of the drivers, which occurs due to overwork, fatigue, and many other factors, is one of those causes. It is therefore most critical to establish systems that can detect the driver's drowsy state and provide the drivers with the appropriate warning system. In addition to the automatic speed control of the car, this system thus supports drivers in incidents by providing warnings in advance. This means that road collisions that are harmful to living lives are minimised. This is achieved by using the technique of image recognition, where driver drowsiness is observed, and using this method, simultaneous warning and speed monitoring of the vehicle is carried out.


Introduction
In recent years, an increase in demand for modern transportation has led to faster growth in car parks. The car is currently an integral mode of transport for citizens. A total of 97 million vehicles were sold worldwide in 2017, which was 0.3 percent more than in 2016. The global average estimate of the number of vehicles being used was over 1 billion in 2018. While the car has changed the lifestyle of individuals and increased the ease of performing everyday activities, many negative consequences, such as traffic accidents, are also associated with it. A National Highway Traffic Safety Administration study found that in 2016, there were a total of 7,277,000 traffic collisions in the United States, resulting in 37,461 fatalities and 3,144,000 injuries. In these incidents, fatigue driving caused traffic accidents of around 20 percent-30 percent. Fatigued driving is also a major and latent threat in traffic accidents. The fatigue-driving-detection method has become a hot research subject in recent years [1][2][3][4][5]. This paper focuses primarily on alerting the sleepy driver and in addition, automatic speed control of the vehicle is carried out by reducing the supply of fuel to the engine in the case of diesel-powered vehicles or directly reducing the speed of the DC engine in the case of electric vehicles.
The paper is structured as follows: The relevant works are reviewed in Section 2. Section 3 presents this paper's full hardware design along with the implementations of the algorithm. The findings obtained are discussed in Section 4. This paper ends with Section 5.

Related Works
For example, the identification of eye blinks is critical in systems that monitor the vigilance of a human operator, such as driver drowsiness, in systems that alert a computer user to look at the screen without blinking for a long time to avoid dry eye and computer vision syndromes, in human computer interfaces that promote communication for people with disabilities, or for anti-spoofing protection in face recognition syndromes Either active or passive are existing techniques. Active strategies are effective but use specific hardware, often costly and invasive, such as infrared cameras and illuminators, wearable devices, eye-watching glasses with special close-up cameras [6]. Although the passive systems depend solely on a standard remote camera. Many strategies for automatically detecting eye blinks in a video series have been suggested. Several techniques are based on an approximation of motion in the eye area. Usually, a Viola-Jones style detector detects the face and eyes. This method uses the Dricare technique which uses yawning, eye blinking, and eye closure length to detect drowsiness. From optical flow, by sparse monitoring, or by frame-to-frame intensity difference and adaptive thresholding, the motion in the eye region is estimated. Finally, a choice is made as to whether or not the eyes are hidden by eyelids. A different method is to infer from a single image the state of the eye opening, such as a heuristic horizontal or vertical image strength projection over the eye area, a parametric model appropriate for finding the eyelids, or active shape models through analogy with open and closed eye templates [7][8][9].
A major disadvantage of previous methods is that in the context of a relative face-camera pose (head orientation), image resolution, lighting, motion dynamics, etc., they typically implicitly place too strong criteria on the setup. In terms of their real-time efficiency, heuristic techniques using raw image intensity are especially likely to be very sensitive. Thus, a strong real-time facial landmark detector is taken that catches most of the characteristic points on a picture of a human face, including eye corners and eyelids. A regression problem is formulated by most state-of-theart landmark detectors, where mapping from an image into landmark positions or into another landmark parametrization is taught. These modern landmark detectors are trained on in-the-wild datasets" and are thus resilient to varying lighting, different facial expressions, and moderate rotations of the non-frontal head. The average error of a state-of-the-art detector's landmark localization is typically below five percent of the inter-ocular gap. Latest techniques are running super real-time, also dramatically [10]. The figure 2.1 explains the block diagram of the proposed system work. It consists of a camera the captures the drowsy state of the drivers. The processed frames from the video that is captured is given to the arduino UNO. The LEDs, vibrator, buzzer and DC motor are connected to arduino UNO to indicate the alert system and speed control.

Proposed System Working
The proposed system makes use of similar techniques used by the existing system, but it is also accompanied by the automatic speed control of the vehicle. The major cause for the road accidents at present is due to drowsiness state of the driver. Hence it is the most important requirement to make a drowsiness detection and alert system for the drivers in order to control the accidents. The block diagram is given in the figure 2.1. In the existent system, there is no automatic speed control of the vehicle, this requires the necessary action of the driver which includes either reducing the speed of the car, or applying an immediate brake whenever an obstacle is close by. In the existent system, the driver has to go through several parameters that is involved in the response time. The parameters include,

A. Visual Object Tracking
Visual object tracking is a crucial problem in computer vision. It has a wide range of applications in fields such as human-computer interaction, behaviour recognition, robotics, and surveillance. Visual object tracking estimates the target position in each frame of the image sequence, given the initial state of the target in the previous frame [11][12]. The tracking of the moving target can be realized using the pixel relationship between adjacent frames of the video sequence and displacement changes of the pixels. However, this algorithm can only detect the medium-sized target that shifts between two frames. With the recent advances of the correlation filter in computer vision, which can produce stable correlation filters to track the object.

B. Facial Landmarks Recognition
The purpose of facial key-points recognition is that getting the crucial information about locations of eyebrows, eyes, lips and nose in the face. With the development of deep learning, human facial key points can be detected.

C. Driver Drowsiness Detection
Driver drowsiness is done using Facial land mark detectors. The nodes of eyes are taken under consideration. The Eye aspect ratio (EAR) is computed for both the eyes. Based on the Eye aspect ratio value, drowsiness detection can be made. If the value is 0, the eyes are closed. This indicates the sleepy state of the drivers. If the value is constant, the eyes are open. This indicates the normal condition of the driver.

D. Automatic speed control
The main contribution of this system is automatic speed control of the vehicle as soon as the actuator starts working. This is essentially done by reducing the fuel supply to the engine. This can be done by constriction of the valve that is responsible for delivering the fuel supply to the engine. This can be done using the Electronic Control Unit (ECU) that is present inside every vehicle. ECU serves a medium for several functionings, out of which, fuel supply to engine is of major concern. This is therefore used in the automatic speed control of the vehicles as soon as a wake-up call is given.

Experimental Diagram
The above figure 3.1 explains the overall experimentation.
• The camera captures the driver's sleepy state.
• This is fed as an input to the controller.
• Once sleepy state of the driver is detected, Simultaneous vibration to wake up the driver, and the automatic control is made. • Thus the emergency detection before braking is done. If both IN1 and IN2 are 0, that is when drowsiness is detected, motor speed reduces to half, if either IN1 or IN2 is found to be 1, motor runs at the usual speed, that is no drowsiness is detected. For motor, arduino pin 3 is used, which can be used as both digital pin or pwm pin. In pwm, 0 to 255 ranges are available, in which 0 means 0V and 255 means 5V. At normal condition, 255 pulses are used, while during sleep detection, 150 pwm pulses are used. The motor usually can run at 60rpm for 12V, hence when drowsiness is detected, the motor speed is reduced to 30rpm for same 12V.Hence, this prototype explains that whenever dro wsiness is detected, the buzzer and LED indicators are activated, along with the reduction in the motor speed roughly to half of the motor speed.In real time, LED indicators can be mounted in speedometer or steering, and buzzer can be mounted on seat belt indicator. Vibrator can be placed either on seat or on steering. Once these are activated, the signal can be sent to ECU board for the speed reduction of the vehicle.

Algorithm Used
A simple but efficient algorithm to detect eye blinks by using a recent Facial landmark detector. A single scalar quantity that reflects a level of the eye opening is derived from the landmarks. Real-time facial landmark detectors capture most of the characteristic points on a human face image, including eye corners and eyelids Per-frame sequence of the eye-opening estimates, the eye blinks are found that is trained on examples of blinking and non-blinking patterns A decision is made whether the eyes are or are not covered by eyelids. A different approach is to infer the state of the eye opening from a single image This can be done using Correlation matching with open and closed eye templates.

Eye Aspect Ratio (EAR)
A method to exploit state-of-the-art facial landmark detectors to localize the eyes and eyelid contours. From the landmarks detected in the image, we derive the eye aspect ratio (EAR) that is used as an estimate of the eye-opening state. Since the perframe EAR may not necessarily recognize the eye blinks correctly, a classifier that takes a larger temporal window of a frame into account is trained.
The eye aspect ratio (EAR) between height and width of the eye can be computed for every video frame given in equation 4.1 EAR= ||p2-p6||-||p3-p5|| …3.1 2||p1-p4|| • EAR is mostly constant when eye is open • EAR is close to zero when eye is closed   In the above figure 4.2, drowsiness state of the person is detected, hence once a person's eyes are closed for 2 sec, the actuators are triggered. The system waits for around 2 sec, since eye blinks can occur at about 0.5 sec. The closure of eyelids is checked by the camera, after which an alert on the screen is displayed. Apart from this, LEDs, buzzer, vibrator are activated. Ultimately, the motor speed is reduced automatically by half, thus indicating the reduction in speed of the vehicle.

Conclusion
In today's scenario, the identification of drowsiness systems is of great necessity since more than half of road accidents are attributed to driver drowsiness. Thus, the driver's drowsiness can be automatically identified by the camera that can be mounted directly in front of the driver by using this device and tracks the sleepy state. As sources of wake-up call, the use of warning systems and vibration techniques is of great use but automatic speed control is often performed in this method, which does not require drivers decreasing the vehicle's speed. Automatic braking can also be unsafe for other cars, but for both the driver and the adjacent vehicles, automatic speed control is safer. This system is built into the electronic control unit, where the fuel supply to the engine can also be controlled automatically in the case of ordinary vehicles, and the ECU can automatically reduce the speed in the case of electric vehicles, as the dc engine is used. The extension of this device is the use of infrared light, along with the camera, to provide a light source during the night. This system will thus serve as the most effective system in order to detect the system's drowsiness and automatically regulate speed.