TRUST-ME - TRUST-ME: TRUstworthy enhancement of job SaTisfaction and productivity using Micro-sensing in work Environments

For almost a century, employee productivity and job satisfaction have been a topic of interest. They are sometimes in conflict, with employers pursuing productivity at the expense of employee satisfaction. Other times they are aligned, as satisfied employees tend to be more productive, and employees that feel productive are usually satisfied. Over the years, awareness of the importance of good work conditions for physical and mental health has grown, and legislation to protect workers has followed. However, new concerns such as precarious employment (i.e., the “gig-economy”) have also led to increased stress and worsened mental health among the workforce. Technology has seemingly tipped the scales the wrong way in recent years. Today, so-called “technological supervision” – the automated management of employee schedules, from warehouse staff to hotel housekeepers to software developers, in a quest to maximize efficiency – is increasingly creating physical and mental exhaustion among those managed in this way. The COVID-19 pandemic, with the increase in remote work, has only exacerbated these problems. There is thus a need for approaches that increase productivity, job satisfaction, and employee well-being without sacrificing one for the other.

Recent developments in sensing technology (e.g., video- and audio-based and wearable sensors) and advanced artificial intelligence (AI) algorithms are promising personalized digital tools for monitoring well-being and boosting productivity. At the same time, awareness of privacy issues and the possibility of unpredictable and undesirable outcomes of AI have grown, prompting interest in the development of privacy-preserving technology and trustworthy AI. By combining expertise in sensing technology, privacy, user experience design, and explainable AI, this project will develop a new approach for monitoring job satisfaction, wellbeing, and productivity. The project will (1) offer insights into the complex relations at work in AI-driven privacy-aware workplace monitoring and (2) create powerful yet explainable AI that provides actionable productivity insights.

The TRUST-ME project will utilize the expertise from two separate institutions. Swiss partner USI has expertise in security, privacy, and trustworthy AI. Slovenian partner JSI has expertise in machine learning for sensing and modeling psycho-physiological constructs, including employee wellbeing. The proposed project will thus explore the problem domain via two critical strands of investigation: (1) multimodal monitoring and modeling of job satisfaction and productivity (led by JSI) and (2) multimodal, secure, private, and explainable AI for productivity assessment (lead by USI).

The first strand will employ physically unobtrusive workplace sensors for knowledge workers, such as computers (which can sense typing patterns, mouse movement, and application usage), cameras, and eye trackers. These will be used to obtain psychological constructs of interest, e.g., satisfaction, wellbeing, engagement, self-perceived productivity, and affect. We will model these constructs' interrelation using multi-modal sensor inputs and multiple sources of ground truth based on novel deep-learning methods involving deep supervision and bi-directional learning. This approach will represent the problem domain more accurately, alleviating the impact of noise in ground truth otherwise obtained via (subjective) questionnaires.

The second strand will implement tools and techniques for maximizing user security and privacy in this context, including quality and integrity of data, access to data, and adequate privacy and confidentiality controls. We will use federated learning to build models on user-trusted local devices instead of sending potentially sensitive user data to a central location (where employers could misuse it). Additionally, we will develop multimodal explainable AI methods to provide transparency and end-user understandable reasons for the findings of the TRUST-ME models. We will exploit this to create a dashboard that will provide users with insights into their well-being and productivity while clarifying how the system came to these conclusions. Both employees and employers will receive information at the right granularity without intruding on employees’ privacy. TRUST-ME will lay the foundation for future sensing interventions to improve job satisfaction and productivity while respecting ethical values such as privacy and autonomy.

Both strands of the investigation will come together in a validation study with academic knowledge workers, software developers, and administrative staff. This study will provide inputs into the design of the TRUST-ME approach, data for training and evaluating the machine-learning models, and feedback on the dashboard presenting their outputs.