Variation of the Heartbeat and Activity as an Indicator of Drowsiness at the Wheel Using a Smartwatch

Sleepiness is one of the first causal factors of accidents. An estimated 10-30% of road deaths are related to fatigue driving. A large number of research studies have been conducted to reduce the risk of accidents while driving. Many of these studies are based on the detection of biological signals by drowsiness/sleepiness. The activity of the autonomic nervous system (ANS) presented alterations during different physical states such as stress or sleepiness. This activity is measured by ECG (electroencephalogram) and, in different studies, it can be measured with the variation of the heart beat (HRV-Heart Rate Variability) in order to analyze it and then detect drowsiness/sleepiness in drivers. The main advantage is that HRV can be performed using non invasive and uncomfortable means compared to EEG sensors. New Wearables technologies are capable of measuring the heart beat and, further, using other sensors like Accelerometer and Gyroscope, embedded on a simple clock allow us to monitor the physical activity of the user. Our main goal is to use the pulsations measurements in conjunction with the physical activity for the detection of driver drowsiness/sleepiness in advance in order to prevent accidents derived from fatigue.


I. INTRODUCTION
ISTORICALLY, the role of sleepiness and sleep disorders have been underestimated in comparison with other classic causes of accidents, such as alcohol or drugs, which may also be associated with drowsiness/sleepiness.
Previous studies estimate that between 10-30% of traffic accidents are directly related to sleepiness and fatigue while driving [1] [2] [3][4] [5]. The fatigue related accidents are directly dependent on the time of day (circadian factor) and the type of road, particularly on motorways and monotonous roads. Mortality associated with such accidents is very high, probably because of the high speed at impact both the type of road and the lack of reaction by the driver.
Drivers usually are unable to assess their symptoms to detect or predict events drowsiness within minutes. This indicates that drivers may need technological interventions for detecting drowsiness/sleepiness while driving. Given these facts, we decided to find a method for alerting the driver about drowsiness/sleepiness. We had to find a way to evaluate driver fatigue states compatible with driving.
Studies related to ANS activity, detection of different physical states and physical activity detection were reviewed and we found that the current technology allowed to reconcile driving with a detection and warning of drowsiness/sleepiness mechanism.
This paper is structured as follows. After discussing related works in section 2, we describe in section 3 the methodology for this case study, technical details about extraction for classifying human activities. Finally section 4 presents the final conclussions II. RELATED WORK Several studies have been conducted to identify effective ways to monitor sleepiness driving [6]. The eye-blinking indicator has been the favorite in many studies. Almost in all the cases, the most commonly used test is the EEG. However, some uncomfortable EEG electrodes were used on the head, which are inappropriate for detecting drowsiness/sleepiness while driving, which is our goal.
Another biological measure that has been significantly related to drowsiness/sleepiness is Heart Rate Variability. HRV is regulated by both the sympathetic and parasympathetic systems, and it can be used to estimate the autonomous nervous system (ANS) activity [1] [7]. The activity of the autonomous nervous system also can be measured in a non-invasive manner from the HRV, whose signal can be obtain from an ECG (electrocardiogram) [1].
Variation of the Heartbeat and Activity as an Indicator of Drowsiness at the Wheel Using a Smartwatch HRV has been studied in transitions from wakefulness to states of extreme relaxation, and there has been a decrease in Heart rate in the beginning of the dream [1] [19].
Once it is known that the number of pulses decreases during sleep [8], the degree in which the number of pulses decrease during the non-REM stages of sleep has been explained as a depression in the activity of the sympathetic central system. Spectral analysis techniques have been used to link the pulse changes with the activity of the central nervous system [8].
Furthermore, there have also been studies in which are shown, it can be detected different patterns of physical activity [9][10][11][12][13][14][15][17] [18] using the accelerometer and gyroscope sensors, one being the state of drowsiness/sleepiness. These studies were worth to know that it is possible to identify activities through sensors like the accelerometer and the gyroscope. Among them are various studies attempting to interpret body activity using the latest current technology (wearables) [9]. More specifically used smart watches, for example the Samsung Gear S, using accelerometer and gyroscope sensors [10], and others also include pedometer [11] [20][21] [23].
All these studies clarify the possibility of detecting sleep or detect patterns of physical activity. Our study goes a step further and has combined both formulae.

III. METHODOLOGY
Assuming you could use data obtained from Heart Rate Monitor (HRM) to detect changes in the nervous system and on the other hand, the data obtained from accelerometer, gyroscope and pedometer to identify physical activities, we decided to unify all the data in order to detect the possible state of drowsiness/sleepiness of a driver, and then alert the user advising him to stop driving.
For this experiment, we needed to find a device that contains all the necessary sensors and did not pose a threat to the user when driving. Technology currently offers, in a single device, a smartwatch: information provided by the heart rate and also accelerometer, gyroscope and pedometer sensors to detect as accurately as possible states of drowsiness/sleepiness during driving. In a smartwatch, the heart rate sensor is PPG type (photoplethysmography). These sensors use a lightbased technology to sense the rate of blood flow as controlled by the heart's pumping action. The other sensors are similar to other devices like tablets or smart phones.
For our choice, the essential features that should be fulfilled by the smart watch include those aforementioned sensors and large battery capacity, since the continued use of all the sensors causes a non insignificant power consumption. Other features, such as screen size, were not significant for the election. Several Smart watches, like Sony Smart Watch 3 SWR50, LG G Watch o Motorola Moto 360, were discarded because they do not have HRM or other sensors.
The smart watch chosen was the Samsung Gear S, containing all the above mentioned sensors, battery life of 4 days with occasional use, and 1-2 days with increased use, and allows us to program the required application to use all the information.
It should be noted that the study of drowsiness/sleepiness at the wheel requires a deep and thorough knowledge of the behavior of people. It is therefore necessary for the study to collect data from both sexes and different ages, ranging from 18 to 70 years, since the behavior with respect to sleep and, especially bodily activity, is different in each of them and reaches the state of drowsiness slightly differently. Another fact to note are the types of roads; the motorways require a monotonous driving as indicated in studies [2] y [5] it induces sleep and causes more accidents, and this is considered in this study. On the other hand, the fact of using a smart watch (Fig  1)[11] that is worn on the wrist and contains sensors Herat rate, accelerometer, gyroscope and pedometer, has many advantages as all sensors are integrated in one device, it is easy to carry, and also allows to detect the lack of physical activity while driving. But, it makes it so challenging the classification of physical activity since the smart watch worn on the wrist should control very well the movements made by hand to predict the driver's state [13] [16].

A. Data Acquisition/collection.
In this first phase, we gathered all the information that sensors reported to us from the largest possible number of people, with different ages and in different scenes in order to see the behavior and so we can analyze data and classify the states of people.
We invited 30 volunteers to collected data, with equal numbers of men and women, ages commented above. The Gear S smart watches were prepared to store the user data for the following case studies: - -Being tired, driving in a driving simulator (City Car Driving + Steering Wheel Logitech G27) for two hours, simulated driving situation on a highway. -Watching TV before sleeping.
-Going to bed and sleeping the whole night.
The first three scenarios are real driving. The other scenarios may not be exactly like a real driving situation, but the use of simulation is pretty close to reality and, they serve to not risk the integrity of individuals and to gather some subjective outcomes.
After completing driving, the driver filled out a form with the actions that had performed, the route taken and, if there was anything significant related to their physical state (either, tired, fatigued, some drowsiness, stressed, etc). Thus, we obtained daily data about the behavior of different people, as well as, samples that we knew what behavior was followed by the driver and under what conditions as well.To detect cases of sleep, the person took note of the time that he slept and the time he started to watch the television.
The objective of this first phase is the collection of samples that allow us to analyze the data for the unambiguous detection of the different states. The

B. Feature Extraction.
For each of the samples obtained, we analyzed on the one hand the results collected by the accelerometer, gyroscope and pedometer sensors; on the other hand, the results collected by the heart rate sensor [22], [23]. After analyzing the results of each driver, they analyzed as a whole, and common patterns were found in the results captured for the different states of the driver. There were small differences between each of them, but it could be possible to clearly identify the different actions.

C. Classification.
Using FFT tools, various types of filters and scatter plots points, we found out common patterns for each of the states of the driver and state of drowsiness/sleepiness. While driving behavior of each person is not the same, after analyzing the data, with very small differences, they all converge at the same point for each of the states identified [10] [12]. We can find more nervous drivers when parking -since they made faster movements-, yet, other drivers made movements with greater confidence, but in the end, both drivers make the same set of movements (82%) and this is what serves our interests. In the case of the detection of sleep, the same approach is followed. Although there are people who find it hard to get more sleep and other costs them less, ultimately all of their heart rate converge to decrease rates in order to stabilize at a threshold associated with the dream state. For those cases, where the person fell asleep, such as going to bed or get to watch TV after dinner, it observed as pulsations are reduced as was reconciling the dream, as other studies have reported [1] [19]. It is also noted that the values set with the accelerometer and gyroscope are few swings because the driver tends to correct the direction of the car compared to previous moments.
The data were separated and classified according to the following states: - D. Process. After analysis of the data with the various tools, such as Kalman filter, pass band filter, high pass filter…, we concluded that these data did not provide further information to identify actions, which were ruled to implement the system.
The software system developed for detecting driver drowsiness/sleepiness, and providing additional capabilities, it´s based on the following processing: To identify actions using data collected by the sensors accelerometer, gyroscope y pedometer: Fourier series: Fast Fourier Transform (FFT): Vector direction: For identification of drowsiness with heart rate sensor data: Average: The algorithm processes the data stored once it has completed the minimum time window for collecting the data set needed to predict the status of the driver. The window time is 30 seconds. At the beginning of the application, within 30 seconds, the algorithm is not run as the data set needed to decide the status of the driver is still not available. After the first 30 seconds, the algorithm, every second, runs checking the new data that have been registered (20 new records per second).
The final decision for detection of sleep is made based on the result (pic 4) by the algorithm which uses the heartrate sensor (50%) and the algorithm that uses accelerometer and gyroscope sensors (50%). Both must have had the same result, that is to say, having detected that the driver is entering a state of drowsiness/sleepiness, to alert the driver. Thus, false positives are avoided.
In addition, data collected from the accelerometer and gyroscope sensors are also used to detect other driver actions: park, leaving the car, etc. In the tests carried out, it should be noted, that in travel four hours, the battery has an average consumption of 50%, running throughout the trip application in background mode. 1. Online method: once the algorithm was coded, the resulting application is installed on the smart watch and re-run on the same users on which data were captured. The application runs in debug mode so that we could record all the events produced and reanalyze the data recorded to refine, if necessary, the algorithm. 2. Offline method: Having already recorded data from previous lines, what we did was to define a test environment using the same algorithm we have designed for the application, so you can run the same data stream on the algorithm in real time and then you can also validate the algorithm designed. Thus, also we can detect errors of the algorithm in a fully controlled environment that simulates the real environment.

IV. CONCLUSION
In this paper, we have presented a new application designed for Samsung Gear S smart watch recognizing and alerting drowsy state of a driver at the wheel of a vehicle. Using this application is to avoid car accidents caused by the driver falling asleep at the wheel.
Other application functionalities are (1) warning to the driver that he must rest every two hours, and (2) firing an alarm in states of nervousness that could mean danger while driving.
In this application, specific algorithm was designed using known mathematical tools such as FFT, filters, scatter of points and statistic power in order to detect that the driver is entering a state of drowsiness/sleepiness. It has a very acceptable power consumption rate and at no time blocks another applications or the system itself, thus providing optimum performance.
Finally. using this application on Samsung Gear S smartwatch is a very simple and easy way, from user point of view, to be protected against a circumstance of drowsiness/sleepiness while driving.