scholarly journals Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging

10.2196/10140 ◽  
2019 ◽  
Vol 6 (4) ◽  
pp. e10140 ◽  
Author(s):  
Youngjun Cho ◽  
Simon J Julier ◽  
Nadia Bianchi-Berthouze

Background A smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. A smartphone camera–based photoplethysmography (PPG) and a low-cost thermal camera can be used to create cheap, convenient, and mobile monitoring systems. However, to ensure reliable monitoring results, a person must remain still for several minutes while a measurement is being taken. This is cumbersome and makes its use in real-life situations impractical. Objective We proposed a system that combines PPG and thermography with the aim of improving cardiovascular signal quality and detecting stress responses quickly. Methods Using a smartphone camera with a low-cost thermal camera added on, we built a novel system that continuously and reliably measures 2 different types of cardiovascular events: (1) blood volume pulse and (2) vasoconstriction/dilation-induced temperature changes of the nose tip. 17 participants, involved in stress-inducing mental workload tasks, measured their physiological responses to stressors over a short time period (20 seconds) immediately after each task. Participants reported their perceived stress levels on a 10-cm visual analog scale. For the instant stress inference task, we built novel low-level feature sets representing cardiovascular variability. We then used the automatic feature learning capability of artificial neural networks to improve the mapping between the extracted features and the self-reported ratings. We compared our proposed method with existing hand-engineered features-based machine learning methods. Results First, we found that the measured PPG signals presented high quality cardiac cyclic information (mean pSQI: 0.755; SD 0.068). We also found that the measured thermal changes of the nose tip presented high-quality breathing cyclic information and filtering helped extract vasoconstriction/dilation-induced patterns with fewer respiratory effects (mean pSQI: from 0.714 to 0.157). Second, we found low correlations between the self-reported stress scores and the existing metrics of the cardiovascular signals (ie, heart rate variability and thermal directionality) from short measurements, suggesting they were not very dependent upon one another. Third, we tested the performance of the instant perceived stress inference method. The proposed method achieved significantly higher accuracies than existing precrafted features-based methods. In addition, the 17-fold leave-one-subject-out cross-validation results showed that combining both modalities produced higher accuracy than using PPG or thermal imaging only (PPG+Thermal: 78.33%; PPG: 68.53%; Thermal: 58.82%). The multimodal results are comparable to the state-of-the-art stress recognition methods that require long-term measurements. Finally, we explored effects of different data labeling strategies on the sensitivity of our inference methods. Our results showed the need for separation of and normalization between individual data. Conclusions The results demonstrate the feasibility of using smartphone-based imaging for instant stress detection. Given that this approach does not need long-term measurements requiring attention and reduced mobility, we believe it is more suitable for mobile mental health care solutions in the wild.

2018 ◽  
Author(s):  
Youngjun Cho ◽  
Simon J Julier ◽  
Nadia Bianchi-Berthouze

BACKGROUND A smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. A smartphone camera–based photoplethysmography (PPG) and a low-cost thermal camera can be used to create cheap, convenient, and mobile monitoring systems. However, to ensure reliable monitoring results, a person must remain still for several minutes while a measurement is being taken. This is cumbersome and makes its use in real-life situations impractical. OBJECTIVE We proposed a system that combines PPG and thermography with the aim of improving cardiovascular signal quality and detecting stress responses quickly. METHODS Using a smartphone camera with a low-cost thermal camera added on, we built a novel system that continuously and reliably measures 2 different types of cardiovascular events: (1) blood volume pulse and (2) vasoconstriction/dilation-induced temperature changes of the nose tip. 17 participants, involved in stress-inducing mental workload tasks, measured their physiological responses to stressors over a short time period (20 seconds) immediately after each task. Participants reported their perceived stress levels on a 10-cm visual analog scale. For the instant stress inference task, we built novel low-level feature sets representing cardiovascular variability. We then used the automatic feature learning capability of artificial neural networks to improve the mapping between the extracted features and the self-reported ratings. We compared our proposed method with existing hand-engineered features-based machine learning methods. RESULTS First, we found that the measured PPG signals presented high quality cardiac cyclic information (mean pSQI: 0.755; SD 0.068). We also found that the measured thermal changes of the nose tip presented high-quality breathing cyclic information and filtering helped extract vasoconstriction/dilation-induced patterns with fewer respiratory effects (mean pSQI: from 0.714 to 0.157). Second, we found low correlations between the self-reported stress scores and the existing metrics of the cardiovascular signals (ie, heart rate variability and thermal directionality) from short measurements, suggesting they were not very dependent upon one another. Third, we tested the performance of the instant perceived stress inference method. The proposed method achieved significantly higher accuracies than existing precrafted features-based methods. In addition, the 17-fold leave-one-subject-out cross-validation results showed that combining both modalities produced higher accuracy than using PPG or thermal imaging only (PPG+Thermal: 78.33%; PPG: 68.53%; Thermal: 58.82%). The multimodal results are comparable to the state-of-the-art stress recognition methods that require long-term measurements. Finally, we explored effects of different data labeling strategies on the sensitivity of our inference methods. Our results showed the need for separation of and normalization between individual data. CONCLUSIONS The results demonstrate the feasibility of using smartphone-based imaging for instant stress detection. Given that this approach does not need long-term measurements requiring attention and reduced mobility, we believe it is more suitable for mobile mental health care solutions in the wild.


2021 ◽  
Author(s):  
◽  
Matt Cryer

<p>Colloidal semiconductor nanocrystals (NCs) with bandgaps less than 1 eV allow the development of mid wave infrared (MIR) sensitive detectors that exploit the benefits of colloidal materials, primarily bandgap selection and solution deposition. Additionally, the electrical behaviour of these films can be examined for characteristics that can increase the functionality of NC based detectors.  The production of devices that are designed to be competitive as ultra-low-cost, room temperature MIR detectors, operating with photonic, rather than thermal detection is detailed. The evolution of the colloidal synthesis, spray deposition methods, substrate materials and post deposition treatments used here lead to highly robust and high performing devices. These devices demonstrate a “colour” sensitivity down to 300 nm in the MIR (≈10 % of scale), with superior responsivities for this class of device, up to 0.9 AW⁻¹, and competitive specific detectivity up to 8 × 10⁹ Jones at 200 Hz and 300 K. Furthermore, these devices utilise a cheap and robust substrate material that allows operation after deformation up to 45 ° without degradation over many cycles. These devices offer a template for ultra-low-cost MIR detectors with performance that rivals microbolometers but with better measurement speed and spectral sensitivity. As such these devices showcase the key advantages of using colloidal NCs in MIR applications.  Planar and fully air processed thin film devices that demonstrate photo-induced memristive behaviour and can be used as a transistors, photode-tectors or memory devices are investigated. Following long term (60 h) air exposure, unpackaged NC films develop reliable memristive characteristics in tandem with temperature, gate and photoresponse. On/off ratios of more than 50 are achieved and the devices show long term stability, producing repeatable metrics over days of measurement. The on/off behaviour is shown to be dependent on previous charge flow and carrier density, implying memristive rather than switching behaviour. These observations are described within a long term trap filling model. This work represents an advance in the integration of NC films into electronic devices, which may lead to the development of multi-functional electronic components.  Building on the previous work the steps taken to move from a planar device, that works well in controlled conditions, to a multi-pixel sensor that can demonstrate MIR video imaging at room temperature in a noisy environment are shown. This is achieved with a 15 pixel detector that consists only of a polymer substrate and solution patterned NC pixels. This device can detect a 373 K object with the device at 298 K in a noisy environment. This performance is enabled by photogain at 5 V bias that reaches a maximum External Quantum Efficiency (EQE) of 1940 ± 290 % for a pixel with a 3.3 µm bandgap. Through the use of four separate bandgaps it is shown that “multicolour” thermal imaging systems can deliver another layer of information, on top of intensity, to the user. The behaviour of the system is examined under use and it is shown that the photoconductive device behaves as expected with regards to bias, and that trap enabled gain is sensitive to total incident flux, more than the spectral energy distribution of the target. Finally, it is shown that solution patterned QD fabrication methods can deliver electrical reproducibility between pixels that is sufficient to allow an imaging plane of multiple pixels.  The somewhat neglected tin chalcogenide semiconductor nanocrystals are investigated and inverse MIR detection at room temperature is demonstrated with planar, solution and airprocessed PbSnTe and SnTe QD devices. The detection mechanism is shown to be mediated by an interaction between MIR radiation and the vibrational stretches of adsorbed hydroxyl species at the oxdised NC surface. Devices are shown to possess mAW⁻¹ responsivity via a reduction in film conductance due to MIR radiation and, unlike classic MIR photoconductors, are unaffected by visible wavelengths. As such these devices offer the possibility of MIR thermal imaging that has an intrinsic solution to the blinding caused by higher energy light sources.  In summary, it is shown that semiconductor NCs with an all ambient fully solution processed deposition and ligand exchange procedure can be used to create simple, robust and cheap devices that are beginning to demonstrate metrics on par with current commercial thermal detector systems. It is also shown that these devices can under certain circumstances demonstrate novel behaviours that offer the prospects of enhanced or novel functionality.</p>


2018 ◽  
Author(s):  
Youngjun Cho ◽  
Simon J. Julier ◽  
Nadia Bianchi-Berthouze

AbstractBackgroundA smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. Photoplethysmography (PPG) and low-cost thermography can be used to create cheap, convenient and mobile systems. However, to achieve robustness, a person has to remain still for several minutes while a measurement is being taken. This is very cumbersome, and limits the usage in applications such producing instant measurements of stress.ObjectiveWe propose to use smartphone-based mobile PPG and thermal imaging to provide a fast binary measure of stress responses to an event using dynamical physiological changes which occur within 20 seconds of the event finishing.MethodsWe propose a system that uses a smartphone and its physiological sensors to reliably and continuously measure over a short window of time a person’s blood volume pulse, the time interval between heartbeats (R-R interval) and the 1D thermal signature of the nose tip. 17 healthy participants, involved in a series of stress-inducing mental activities, measured their physiological response to stress in the 20 second-window immediately following each activity. A 10-cm Visual Analogue Scale was used by them to self-report their level of mental stress. As a main labeling strategy, normalized K-means clustering is used to better treat interpersonal differences in ratings. By taking an array of the R-R intervals and thermal directionality as a low-level feature input, we mainly use an artificial neural network to enable the automatic feature learning and the machine learning inference process. To compare the automated inference performance, we also extracted widely used high level features from HRV (e.g., LF/HF ratio) and the thermal signature and input them to a k-nearest neighbor to infer perceived stress levels.ResultsFirst, we tested the physiological measurement reliability. The measured cardiac signals were considered highly reliable (signal goodness probability used, Mean=0.9584, SD=0.0151). The proposed 1D thermal signal processing algorithm effectively minimized the effect of respiratory cycles on detecting the apparent temperature of the nose tip (respiratory signal goodness probability Mean=0.8998 to Mean=0). Second, we tested the 20 seconds instant perceived stress inference performance. The best results were obtained by using automatic feature learning and classification using artificial neural networks rather than using pre-crafted features. The combination of both modalities produced higher accuracy on the binary classification task using 17-fold leave-one-subject-out (LOSO) cross-validation (accuracy: HRV+Thermal: 76.96%; HRV: 60.29%; Thermal: 61.37%). The results are comparable with the state of the art automatic stress recognition methods requiring long term measurements (a minimum of 2 minutes for up to around 80% accuracy from LOSO). Lastly, we explored the impact of different data labeling strategies used in the field on the sensitivity of our inference methods and the need for normalization within individual.ConclusionsResults demonstrate the capability of smartphone biomedical imaging in instant mental stress recognition. Given that this approach does not require long measurements requiring attention and reduced mobility, it is more feasible for mobile mental healthcare solution in the wild.


2021 ◽  
Author(s):  
◽  
Matt Cryer

<p>Colloidal semiconductor nanocrystals (NCs) with bandgaps less than 1 eV allow the development of mid wave infrared (MIR) sensitive detectors that exploit the benefits of colloidal materials, primarily bandgap selection and solution deposition. Additionally, the electrical behaviour of these films can be examined for characteristics that can increase the functionality of NC based detectors.  The production of devices that are designed to be competitive as ultra-low-cost, room temperature MIR detectors, operating with photonic, rather than thermal detection is detailed. The evolution of the colloidal synthesis, spray deposition methods, substrate materials and post deposition treatments used here lead to highly robust and high performing devices. These devices demonstrate a “colour” sensitivity down to 300 nm in the MIR (≈10 % of scale), with superior responsivities for this class of device, up to 0.9 AW⁻¹, and competitive specific detectivity up to 8 × 10⁹ Jones at 200 Hz and 300 K. Furthermore, these devices utilise a cheap and robust substrate material that allows operation after deformation up to 45 ° without degradation over many cycles. These devices offer a template for ultra-low-cost MIR detectors with performance that rivals microbolometers but with better measurement speed and spectral sensitivity. As such these devices showcase the key advantages of using colloidal NCs in MIR applications.  Planar and fully air processed thin film devices that demonstrate photo-induced memristive behaviour and can be used as a transistors, photode-tectors or memory devices are investigated. Following long term (60 h) air exposure, unpackaged NC films develop reliable memristive characteristics in tandem with temperature, gate and photoresponse. On/off ratios of more than 50 are achieved and the devices show long term stability, producing repeatable metrics over days of measurement. The on/off behaviour is shown to be dependent on previous charge flow and carrier density, implying memristive rather than switching behaviour. These observations are described within a long term trap filling model. This work represents an advance in the integration of NC films into electronic devices, which may lead to the development of multi-functional electronic components.  Building on the previous work the steps taken to move from a planar device, that works well in controlled conditions, to a multi-pixel sensor that can demonstrate MIR video imaging at room temperature in a noisy environment are shown. This is achieved with a 15 pixel detector that consists only of a polymer substrate and solution patterned NC pixels. This device can detect a 373 K object with the device at 298 K in a noisy environment. This performance is enabled by photogain at 5 V bias that reaches a maximum External Quantum Efficiency (EQE) of 1940 ± 290 % for a pixel with a 3.3 µm bandgap. Through the use of four separate bandgaps it is shown that “multicolour” thermal imaging systems can deliver another layer of information, on top of intensity, to the user. The behaviour of the system is examined under use and it is shown that the photoconductive device behaves as expected with regards to bias, and that trap enabled gain is sensitive to total incident flux, more than the spectral energy distribution of the target. Finally, it is shown that solution patterned QD fabrication methods can deliver electrical reproducibility between pixels that is sufficient to allow an imaging plane of multiple pixels.  The somewhat neglected tin chalcogenide semiconductor nanocrystals are investigated and inverse MIR detection at room temperature is demonstrated with planar, solution and airprocessed PbSnTe and SnTe QD devices. The detection mechanism is shown to be mediated by an interaction between MIR radiation and the vibrational stretches of adsorbed hydroxyl species at the oxdised NC surface. Devices are shown to possess mAW⁻¹ responsivity via a reduction in film conductance due to MIR radiation and, unlike classic MIR photoconductors, are unaffected by visible wavelengths. As such these devices offer the possibility of MIR thermal imaging that has an intrinsic solution to the blinding caused by higher energy light sources.  In summary, it is shown that semiconductor NCs with an all ambient fully solution processed deposition and ligand exchange procedure can be used to create simple, robust and cheap devices that are beginning to demonstrate metrics on par with current commercial thermal detector systems. It is also shown that these devices can under certain circumstances demonstrate novel behaviours that offer the prospects of enhanced or novel functionality.</p>


2017 ◽  
Vol 29 (2) ◽  
pp. 317-326 ◽  
Author(s):  
Jörg Güttler ◽  
◽  
Muhammad Karim ◽  
Christos Georgoulas ◽  
Thomas Bock

[abstFig src='/00290002/05.jpg' width='300' text='A cuffless blood pressure device implemented in a chair' ] In this paper, the authors describe the cuffless blood pressure meter prototype, which is targeting at potential long-term automated blood pressure screening. By the proposed cuffless approach, mental stress is reduced, which increases the reliability of measurement. By using a wireless communication medium to transmit data, care staff can store and access readings more easily. The proposed system was developed using low-cost off-the-shelf parts such as Arduino/Wattuino Uno boards and single-board computers. This enables thereby an unobtrusive implementation of such a compact system into furniture, for example. Its intuitive measurement enables care staff to devote more attention to the patient and less to the blood pressure measurement. The proposed system is described in its hard- and software functionality. Furthermore, experimental results confirm the proposed system’s reliability.


2013 ◽  
Vol 10 (1) ◽  
pp. 153-164 ◽  
Author(s):  
Aleksandra Pavlovic ◽  
Zarko Barbaric

A measurement of energy efficiency in the construction industry aims to reduce permanently energy requirements in design, construction and use of new buildings, sannation and reconstruction of the existing ones. Over the long term, with the expected increasing in the price of energy and the development of awareness about energy conservation and environmental protection, thermal imaging methods will certainly find their wide application in the construction industry. Infrared thermal camera is used to estimate the temperature. However, we do not see all the details of interest on the thermal image, therefore we suggest using the fusion of visual and thermal images of the same part of the building recorded simultaneously. The analysis images of the object obtained by fusion of television and infrared thermal imaging shows the areas of interest for further processing.


Author(s):  
Moveh Samuel ◽  
Samuel-soma M Ajibade ◽  
Fred Fudah Moveh

People counting applications have been used in diverse applications. The ability and accuracy of thermal imaging over conventional image cameras has led to the implementation of thermal cameras in people counting applications. This paper present a thermal people counting smart glass windows. The people counting application would be remotely monitored from a single centralized PC station as it&rsquo;s connected to a multiplex of mass monitoring of 20 thermal camera, all embedded into different glass windows. The thermal cameras would then be able to detect body temperatures of all individuals who pass through any of the camera range and also count the numbers of people who passed through the camera range. The data gotten can then be further utilized in various ways, example is in the control of air conditioning and lightening.


Author(s):  
Raymond T. Greer

A useful resolution specimen for day-to-day SEM instrument standardization should exhibit uniform, periodic microstructure of hundreds to thousands of angstroms, show low contrast, have low cost, be easily prepared, and demonstrate long term stability.Opal (Figure 1) is particularly useful as a resolution specimen; however, several aspects of sample preparation should be emphasized. A high quality gem opal chip a few millimeters in diameter is sufficient for carefully controlled etching in, for example, 10% HF for 30 seconds. The choice of test specimen microstructure size is established by the maximum wavelength of color observed under white light. The iridescence effect is dictated by the microstructure of uniform silica spheres arranged in a cubic closest packing array.


2020 ◽  
Vol 12 (21) ◽  
pp. 3591
Author(s):  
Matheus Gabriel Acorsi ◽  
Leandro Maria Gimenez ◽  
Maurício Martello

The development of low-cost miniaturized thermal cameras has expanded the use of remotely sensed surface temperature and promoted advances in applications involving proximal and aerial data acquisition. However, deriving accurate temperature readings from these cameras is often challenging due to the sensitivity of the sensor, which changes according to the internal temperature. Moreover, the photogrammetry processing required to produce orthomosaics from aerial images can also be problematic and introduce errors to the temperature readings. In this study, we assessed the performance of the FLIR Lepton 3.5 camera in both proximal and aerial conditions based on precision and accuracy indices derived from reference temperature measurements. The aerial analysis was conducted using three flight altitudes replicated along the day, exploring the effect of the distance between the camera and the target, and the blending mode configuration used to create orthomosaics. During the tests, the camera was able to deliver results within the accuracy reported by the manufacturer when using factory calibration, with a root mean square error (RMSE) of 1.08 °C for proximal condition and ≤3.18 °C during aerial missions. Results among different flight altitudes revealed that the overall precision remained stable (R² = 0.94–0.96), contrasting with the accuracy results, decreasing towards higher flight altitudes due to atmospheric attenuation, which is not accounted by factory calibration (RMSE = 2.63–3.18 °C). The blending modes tested also influenced the final accuracy, with the best results obtained with the average (RMSE = 3.14 °C) and disabled mode (RMSE = 3.08 °C). Furthermore, empirical line calibration models using ground reference targets were tested, reducing the errors on temperature measurements by up to 1.83 °C, with a final accuracy better than 2 °C. Other important results include a simplified co-registering method developed to overcome alignment issues encountered during orthomosaic creation using non-geotagged thermal images, and a set of insights and recommendations to reduce errors when deriving temperature readings from aerial thermal imaging.


Author(s):  
Ivan Shipilov

Based on long-term research in the dry steppe zone, recommendations are given for improving low-productive meadow phytocenoses based on the selection of adaptive perennial legumes and cereals, low-cost methods of surface improvement. It was found that the restored grasslands provide up to 13.0–15.0 t/ha of high-quality green mass, which exceeds the productivity of unimproved grounds by 2.4–2.9 times.


Sign in / Sign up

Export Citation Format

Share Document