Novel Sensor Applications
A Systematic Comparison of High-End and Low-Cost EEG Amplifiers for Concealed, Around-the-Ear EEG Recordings
DOI - ReserachGateSensors
Abstract: Wearable electroencephalography (EEG) has the potential to improve everyday life through brain–computer interfaces (BCI) for applications such as sleep improvement, adaptive hearing aids, or thought-based digital device control. To make these innovations more practical for everyday use, researchers are looking to miniaturized, concealed EEG systems that can still collect neural activity precisely. For example, researchers are using flexible EEG electrode arrays that can be attached around the ear (cEEGrids) to study neural activations in everyday life situations. However, the use of such concealed EEG approaches is limited by measurement challenges such as reduced signal amplitudes and high recording system costs. In this article, we compare the performance of a lower-cost open-source amplification system, the OpenBCI Cyton+Daisy boards, with a benchmark amplifier, the MBrainTrain Smarting Mobi. Our results show that the OpenBCI system is a viable alternative for concealed EEG research, with highly similar noise performance, but slightly lower timing precision. This system can be a great option for researchers with a smaller budget and can, therefore, contribute significantly to advancing concealed EEG research.
OpenBCI + 3D-Printed Headphones = Open ExG Headphones – An Open-Source Research Platform for Biopotential Earable Applications
DOI - ReserachGateCHI '23: CHI Conference on Human Factors in Computing Systems
Abstract: Earables research is a promising field in Human-Computer Interaction (HCI) due to the variety of observable phenomena that can be captured by sensors placed in the ear region. However, this field is hindered by the use of custom-built hardware and software, which can create barriers for less experienced researchers and lead to high assessment heterogeneity. To address this challenge, we have developed a platform for electrophysiological ear-centered research. This platform combines the established amplifier hardware and recording software stack of the OpenBCI biosignal ExG system with a low-cost 3D-printed over-ear headphone design, based on electrode positioning best practices from neuroscientific research. To demonstrate the flexibility and potential of this platform, we have conducted two studies replicating classical EEG, EOG, ECG, and EMG patterns, and showing that the system is reliable and easy to use in a promising application scenario: detecting mental workload levels across multiple tasks.
A simplified design of a cEEGrid ear-electrode adapter for the OpenBCI biosensing platform
DOI - ReserachGateHardwareX
Abstract: We present a simplified design of an ear-centered sensing system built around the OpenBCI Cyton & Daisy biosignal amplifiers and the flex-printed cEEGrid ear-EEG electrodes. This design reduces the number of components that need to be sourced, reduces mechanical artefacts on the recording data through better cable placement, and simplifies the assembly. Besides describing how to replicate and use the system, we highlight promising application scenarios, particularly the observation of large-amplitude activity patterns (e.g., facial muscle activities) and frequency-band neural activity (e.g., alpha and beta band power modulations for mental workload detection). Further, examples for common measurement artefacts and methods for removing them are provided, introducing a prototypical application of adaptive filters to this system. Lastly, as a promising use case, we present findings from a single-user study that highlights the system's capability of detecting jaw clenching events robustly when contrasted with 26 other facial activities. Thereby, the system could, for instance, be used to devise applications that reduce pathological jaw clenching and teeth grinding (bruxism). These findings underline that the system represents a valuable prototyping platform for advancing ear-based electrophysiological sensing systems and a low-cost alternative to current commercial alternatives.
Exploring the Recognition of Facial Activities Through Around-the-Ear Electrode Arrays (cEEGrids)
13th Retreat on NeuroIS
Abstract: NeuroIS scholars increasingly rely on more extensive and diverse sensor data to improve the understanding of information system (IS) use and to develop adaptive IS that foster individual and organizational productivity, growth, and well-being. Collecting such data often requires multiple recording devices, which leads to inflated study cost and decreased external validity due to greater intrusion in natural behavior. To overcome this problem, we investigated the potential of using an around-the-ear electrode array capable of capturing neural and cardiac activity for detecting an additional set of variables, namely facial muscle activity. We find that reading, speaking, chewing, jaw clenching, and six different emotion expressions can be differentiated well by a Random Forest classifier. The results are complemented by the presentation of an open-source signal acquisition system. Thereby, an economical approach for naturalistic NeuroIS research and artefact development is provided.
Detecting Daytime Bruxism Through Convenient and Wearable Around-the-Ear Electrodes
12th International Conference on Applied Human Factors and Ergonomics
Abstract: Bruxism is associated with multiple health issues and affects millions of people worldwide. To enable effective interventions, precise, easy-to-use and unobtrusive detection systems are required. Unfortunately, especially for daytime bruxism, such systems still rely on electrodes placed on the face, recording diaries, and manual algorithm tuning. In this work, we present a novel approach for bruxing event detection using comfortable and inconspicuous around-the-ear sensors (cEEGrids) as a form of distal EMG-based measurement. Using Random Forest classifiers on laboratory experiment data, promising F1-scores (up to 0.9) are found for the detection of bruxing events in contrast to a variety of other facial muscle activity events. Thereby, a promising new alternative for feasible awake bruxism detection is demonstrated.
Open-Source Concealed EEG Data Collection for Brain-Computer-Interfaces: Real-World Neural Observation Through OpenBCI Amplifiers with Around-the-Ear cEEGrid Electrodes
Brain-Computer Interfaces
Abstract: Observing brain activity in real life offers exciting possibilities like the support of physical health, mental well-being, and thought-controlled interaction modalities. The development of such applications is, however, strongly impeded by low accessibility to research-grade neural data and a lack of easy-to-use and comfortable sensors. To overcome these challenges, we present the cost-effective adaptation of concealed around-the-ear EEG electrodes (cEEGrids) to the open- source OpenBCI EEG signal acquisition platform. The system combines publicly available electronic components with 3D-printed parts, forming an easily replicable recording system for prolonged use and flexible application development. To demonstrate the system’s feasibility, experimentally induced changes in visual stimulation and mental workload are replicated. Lastly, a novel applica- tion area is investigated, the observation of flow experiences through observation of temporal Alpha power changes. Support for a link between temporal Alpha power and flow is found, indicating an efficient engagement of verbal-analytic reasoning with intensified flow levels.
Feeling the Pain of Others in Need: Studying the Effect of VR on Donation Behavior Using EEG
12th Retreat on NeuroIS
Abstract: Virtual reality (VR) enables people to engage in experiences that reach far beyond physical reality. This has inspired humanitarian organizations (among others the United Nations) to use VR technology to raise the awareness of humanitarian crises by virtually transporting people to the regions affected. As a consequence, these immersive experiences may lead to a change in the readiness to donate. As scientific evidence for this effect is still rare we propose an experimental design which aims at investigating how immersion affects donation behavior. In particular, neurophysiological measurement (EEG) shall shed light on the influence of immersion on emotional and motivational processes. First results from a convenient sample of young men indicate that donation behavior is linked to the dynamics of frontal alpha asymmetry changes.
Flow Neurophysiology
Flow in Knowledge Work: An Initial Evaluation of Flow Psychophysiology Across Three Cognitive Tasks
12th Retreat on NeuroIS
Abstract: To accelerate the development of flow-adaptive IT in NeuroIS research, the present work aims to improve the automatic detection of flow in knowledge work-related situations by observing flow emergence across three controlled tasks. In a pretest, we manipulated the type and difficulty of task and recorded subjective (self-reports) as well as objective (EEG features) measures of flow and mental effort. Results indicate that a novel text typing task resembles the expertise of knowledge workers best which is reflected in elevated flow levels across tasks. Difficulty manipulations based on autonomously chosen task difficulty elicited contrasts in flow and mental effort, which was also reflected in the EEG data by Theta band power modulations. This further highlights the utility of autonomy for stimulating flow. We discuss limitations and improvements for the experiment and how this contributes to further research on flow-adaptive IT.
Predicting In-Field Flow Experiences Over Two Weeks From ECG Data: A Case Study
13th Retreat on NeuroIS
Abstract: Predicting flow intensities from unobtrusively collected sensor data is considered an important yet challenging endeavor for NeuroIS scholars aiming to understand and support flow during IS use. In this direction, a limitation has been the focus on cross-subject models built on data collected in controlled laboratory settings. We investigate the potential of predicting flow in the field through personalized models by collecting report and ECG data from a clerical worker over the course of two weeks. Results indicate that a lack of variation in flow experiences during this time likely diminished these potentials. Through pre-training feature selection methods, model accuracies could be achieved that nonetheless approach related cross-subject flow prediction work. Novel recommendations are developed that could introduce more flow variation in future flow field studies to further investigate the within-subject predictability of flow based on wearable physiological sensor data.
Detecting Flow Experiences in the Field Using Video- Based Head and Face Activity Recognition: A Pilot Study
13th Retreat on NeuroIS
Abstract: Flow represents a valuable daily life experience as it is linked to performance, growth, and well-being. As flow support is still a major challenge due to a lack of automatic and unobtrusive detection methods, NeuroIS scholars face the opportunity to devise measurement approaches for flow experience during IS use and, moreover, flow supporting, adaptive NeuroIS. This work presents the first results from a controlled experience sampling field study in which experiences were observed using video-recordings during a week of scientific writing. Novel behavioral features (face and head activity) with negative flow-report correlations are identified. Together, the results contribute to the NeuroIS community by providing an extended concept of flow as a state of behavioral efficiency, the identification of novel correlates, and recommendations for economical and feasible extensions of the study approach.
Detecting Flow Experiences in Cognitive Tasks - A Neurophysiological Approach
Doctoral Dissertation, Karlsruhe Institute of Technology (KIT), Germany
Awarded "Summa Cum Laude" from the Karlsruhe Institute of Technology (KIT)
Abstract: The experience of flow is a unique sensation of complete task absorption and effortless action that is highlighted as a correlate of peak performances, personal and social growth, and general well-being. For organisations, higher flow frequencies, therefore, relate to a more engaged, skilled, and productive workforce. Especially as global phenomena like increasing knowledge work demand and low worker engagement are developing, organisations could strongly benefit from fostering workers’ flow experiences. However, facilitating flow represents a substantial challenge due to the variety of workers’ abilities, tasks and workplace configurations. Knowledge workers are faced with unstructured and complex tasks, that require numerous domain-specific abilities and cooperation with others. Workplaces are diversifying with boundaries disappearing between centralized and digitally-mediated workspaces. This variety means that only person-, task- and situation-independent approaches can deliver comprehensive flow support. For this reason, research on the experiences neurophysiological basis is increasingly pursued. On this basis, adaptive Neuro-Information Systems (NeuroIS) could be developed that are able to detect flow continuously (especially through wearable sensor systems), and that can provide flow-supporting mechanisms. Presently, despite these efforts, the knowledge on how to detect flow with neurophysiological measures is sparse, highly fragmented, and lacks experimental variety. On the individual level, competing propositions exist that have not been consolidated through cross-situational, and multi-sensor observation. On the group level, almost no research has been conducted to investigate neurophysiological correlates in social interactions, particularly not in digitally-mediated interactions. This dissertation addresses these gaps through the cross-situational observation of flow using wearable ECG and EEG sensor systems. In doing so, limitations in the present state of experimental flow research are addressed that refer to central shortcomings of established paradigms for the controlled elicitation of flow experiences. Specifically, two experiments are conducted with manipulations of difficulty, naturalism, autonomy, and social interaction to investigate the question of how flow elicitation can be intensified, and the experience detected more robustly across situations. These investigations are based on an extensive integration of the theoretic and empiric literature on flow neurophysiology. Altogether, the results suggest flow to be represented by moderate physiological activation and mental workload, by increased attentional task engagement and by affective neutrality. Especially EEG features indicate a diagnostic potential to separate lower from higher flow intensities by the reflection of optimal and non-optimal (individual and group) task difficulties. To catalyse, that the positive promises of fostering flow in individuals and social units, can be realised, avenues to advance flow facilitation research are outlined.
To Be or Not to Be in Flow at Work: Physiological Classification of Flow using Machine Learning
IEEE Transactions on Affective Computing
Abstract: The focal role of flow in promoting desirable outcomes in companies, such as increased employees' well-being and performance, led scholars to study flow in the context of work. However, current measurement approaches which assess flow via self-report scales after task execution are limited due to obtrusiveness and a lack of real-time support. Hence, new measurement approaches must be created to overcome these limitations. In this paper, we use cardiac features (heart rate variability; HRV) and a Random Forest classifier to distinguish high and low flow. Our results from a large-scale lab experiment with 158 participants and a field study with nine participants reveal, that with HRV features alone, flow-classifiers can be built with an accuracy of 68.5% (lab) and 70.6% (field). Our research contributes to the challenge of developing a less obtrusive, real-time measurement method of flow based on physiological features and to investigate flow from a physiological perspective. Our findings may serve as foundation for future work aiming to build physio-adaptive systems which can improve employee's performance. For instance, these systems could ensure that no notifications are forwarded to employees while the system is 'sensing' flow.
Flow in Knowledge Work Groups – Autonomy as a Driver or Digitally Mediated Communication as a Limiting Factor?
40th International Conference on Information Systems
Abstract: As cooperation is becoming increasingly important for complex knowledge work tasks, so is the need to understand how to support it. This study contributes to this aim by manipulating flow experience intensities for individuals working alone or in digitally- mediated small groups. Contrary to related work it is found that flow is not experienced as more intense in groups. Two possible reasons are identified, either an optimization of difficulty through more/less autonomy when working alone/together, or a limitation for sharing intense experiences by working in a communication-restrictive digitally- mediated environment. Furthermore, empiric explorations reveal (i) positive correlations for flow with group performance, satisfaction with and growth of the group, and (ii) that lower/higher flow intensities in digitally-mediated small groups can be classified via machine learning on combinations of ECG, self-report, and task-related data, thus further advancing the development of flow-supportive IS.
Flow and Optimal Difficulty in the Portable EEG: On the Potentiality of using Personalized Frequency Ranges for State Detection
3rd International Conference on Computer-Human Interaction Research and Applications
Abstract: The experience of flow has been centrally linked to peak task performances and heightened well-being. To more effectively elicit these outcomes, flow is increasingly studied using neurophysiological measures. For example, portable EEG is employed to enable automatic state detection required for adaptive system design. However, so far, there is a lack of highly diagnostic findings, and moderately diagnostic ones relate more strongly to a central flow pre-condition – namely optimal task difficulties. Unfortunately, even these metrics might be infeasible in real-world scenarios and for portable EEG systems without midline electrodes. In this work, we discuss how frequency band personalization and separation could provide options to overcome these problems. Results from an experiment with a task manipulated in difficulty highlight that upper Alpha and Beta ranges show differentiating patterns to their lower frequency counterparts (i.e. within bands). These sub- bands could be used to detect instances of higher flow and optimized difficulty using portable EEG.
Flow Neurophysiology in Knowledge Work: Electroencephalographic Observations from Two Cognitive Tasks
5th International Conference on Physiological Computing Systems
Nominated for Best Conference Paper
Abstract: In an effort to study flow experiences in the context of less structured knowledge work (KW), we explored a paradigm we call controlled experience sampling (cESM). Participants worked on a naturalistic, cognitive task (a personal scientific thesis), and a difficulty-manipulated math task. Results show that the cESM approach elicits a consistent flow experience with intensities as least as high as in the math task flow condition. An interesting finding is that given similar flow intensities, different perceptions of stress arise between the two paradigms. EEG results from both tasks suggest increased frontal upper alpha band (10-12Hz) activity with increased task attention, that has higher temporal stability in flow than in a boredom condition, and that is laterally indifferent. Integrating with the presently available literature, the results further consolidate an understanding of flow as a state of fronto-lateral activation.
Exploring Flow Psychophysiology in Knowledge Work
10th Retreat on NeuroIS
Abstract: We report on a first exploration of a new paradigm to study flow physiology in knowledge work that we call controlled experience sampling (cESM) in order to build a bridge for flow physiology research to more unstruc-tured tasks. Results show that the approach elicits a consistent flow experience with intensities as least as high as in an established difficulty-manipulated math task. Yet, significantly lower stress perceptions and heart rate variability (HRV) responses are found in the cESM approach which highlights gaps and conse-quences for the diagnostic potential of HRV features for the understanding of flow physiology and automated flow observation in bio-adaptive systems.
Got Flow?: Using Machine Learning on Physiological Data to Classify Flow
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems
Abstract: As information technologies (IT) are both, drivers of highly engaging experiences and sources of disruptions at work, the phenomenon of flow - defined as "the holistic sensation that people feel when they act with total involvement" [5, p. 36] - has been suggested as promising vehicle to understand and enhance user behavior. Despite the growing relevance of flow at work, contemporary measurement approaches of flow are of subjective and retrospective nature, limiting our possibilities to investigate and support flow in a reliable and timely manner. Hence, we require objective and real-time classification of flow. To address this issue, this article combines recent theoretical considerations from psychology and experimental research on the physiology of flow with machine learning (ML). The overall aim is to build classifiers to distinguish flow states (i.e., low and high flow). Our results indicate that flow-classifiers can be derived from physiological signals. Cardiac features seem to play an important role in this process resulting in an accuracy of 72.3%. Hereby, our findings may serve as foundation for future work aiming to build flow-aware IT-systems.
The Psychophysiology of Flow: A Systematic Review of Peripheral Nervous System Features
9th Retreat on NeuroIS
Abstract: As information systems (IS) are increasingly able to induce highly engaging and interactive experiences, the phenomenon of flow is considered a promising vehicle to understand IS user behavior and to ultimately inform the design of flow-fostering IS. However, despite growing interest of researchers in the phenomenon, knowledge about how to continuously assess flow during IS usage is limited. Hereby, recent developments in NeuroIS and psychophysiology propose novel possibilities to overcome this limitation. This article presents the results of a systematic literature review (SLR) on peripheral nervous system indicators of flow. The findings revealed that currently four major approaches exist towards physiological measurement. Propositions for simple and unobtrusive measurement in IS research are derived in conclusion.
Live Biofeedback
Live Biofeedback as a User Interface Design Element: A Review of the Literature
Communications of the Association for Information Systems
Abstract: With the advances in sensor technology and real-time processing of neurophysiological data, a growing body of academic literature has begun to explore how live biofeedback can be integrated into information systems for everyday use. While researchers have traditionally studied live biofeedback in the clinical domain, the proliferation of affordable mobile sensor technology enables researchers and practitioners to consider live biofeedback as a user interface element in contexts such as decision support, education, and gaming. In order to establish the current state of research on live biofeedback, we conducted a literature review on studies that examine self and foreign live biofeedback based on neurophysiological data for healthy subjects in an information systems context. By integrating a body of highly fragmented work from computer science, engineering and technology, information systems, medical science, and psychology, this paper synthesizes results from existing research, identifies knowledge gaps, and suggests directions for future research. In this vein, this review can serve as a reference guide for researchers and practitioners on how to integrate self and foreign live biofeedback into information systems for everyday use.
Emotion Feedback in Small Group Collaboration: A Research Agenda for Group Emotion Management Support Systems
17th International Conference on Group Decision and Negotiation
Nominated for Best Conference Paper
Abstract: With emotion management continuously emerging as a central factor for positive cognitive and affective small group collaboration outcomes, more investigations are appearing that utilize digital emotion information in group tasks like decision-making, idea generation and structured problem solving. However, no coherent and structured approach has been developed to address the variety of critical aspects of emotion management support. In this article, we outline central research gaps aligned with an established data quality framework. We argue that group emotion management support systems (GEMSS) research should be driven to investigate matters of how digital emotion information is measured, how information should be designed and transmitted, how emotion management can be supported in the context of collaboration scenarios, and how privacy and personal requirement concerns need to be integrated.
Designing Live Biofeedback for Groups to Support Emotion Management in Digital Collaboration
International Conference on Design Science Research in Information Systems
Awarded "Best Prototype" Paper
Abstract: Digital collaboration of individuals has increased in diverse areas such as gaming, learning and product innovation. Across scenarios, adequate intra- and interpersonal emotion management is increasingly acknowledged to be beneficial to cognitive and affective interaction outcomes. Unfortunately, individuals differ notably in their emotion management abilities. Additionally, many types of computer mediated collaboration lack the richness of affective cues traditionally found in face-to-face interaction. We envision psychophysiology-based emotion feedback as an automated tool to improve emotion management, and therefore group performance and satisfaction. The presented prototype presents a first iteration of this idea, centered around information on emotional arousal derived from peripheral nervous system measures.
Other
Facilitating NeuroIS Research Using Natural Language Processing: Towards Automated Recommendations
12th Retreat on NeuroIS
Abstract: Facing rapidly growing numbers of scientific publications, attempts to structure and interconnect scientific articles are becoming increasingly rele- vant. Various approaches to scientific article mapping are discussed in academ- ic literature. However, these are mainly developed upon citation-based similari- ty measures. We propose an extension to these concepts of literature mapping by introducing a knowledge graph-based approach. Our contribution is twofold. We introduce a holistic concept to systematically capture literature in the field of NeuroIS research based on a relational mapping of main research concepts in the field. Additionally, we implement a content-based recommender system for literature in the NeuroIS domain that serves us as a proof-of-concept and a per- formance baseline for further developments. By providing a relational overview of the research domain and a dedicated recommender system, we hope to facili- tate the entry to the field for new researchers and enhance further research.
Chatbot-based Emotion Management for Distributed Teams: A Participatory Design Study
Proceedings of the ACM on Human-Computer Interaction
Abstract: Fueled by the pervasion of tools like Slack or Microsoft Teams, the usage of text-based communication in distributed teams has grown massively in organizations. This brings distributed teams many advantages, however, a critical shortcoming in these setups is the decreased ability of perceiving, understanding and regulating emotions. This is problematic because better team members’ abilities of emotion management positively impact team-level outcomes like team cohesion and team performance, while poor abilities diminish communication flow and well-being. Leveraging chatbot technology in distributed teams has been recognized as a promising approach to reintroduce and improve upon these abilities. In this article we present three chatbot designs for emotion management for distributed teams. In order to develop these designs, we conducted three participatory design workshops which resulted in 153 sketches. Subsequently, we evaluated the designs following an exploratory evaluation with 27 participants. Results show general stimulating effects on emotion awareness and communication efficiency. Further, they report emotion regulation and increased compromise facilitation through social and interactive design features, but also perceived threats like loss of control. With some design features adversely impacting emotion management, we highlight design implications and discuss chatbot design recommendations for enhancing emotion management in teams.
BrownieR - The R-Package for Neuro Information Systems Research
10th Retreat on NeuroIS
Abstract: Neuro-Information-Systems (NeuroIS) research has become an established approach in the information systems (IS) discipline for investigating and understanding user behavior. Our outlined package with the name brownieR is a freely-available open source R-package for analyzing NeuroIS data (i.e. the combination of physiological and behavioral data). The central purpose of this work is to instruct researchers how brownieR can be used in IS research by providing a practical guide on how to conduct the analysis of bio-physiological data combined with behavioral data (e.g., from the web, experimental tasks, or log files). Further, the article provides an analysis framework and covers the different stages involved in analyzing physiological data.