Assessing the impact of new technologies requires looking beyond productivity and efficiency to understand how they affect employees and their working conditions. As part of SEISMEC, researchers from Technische Universität Berlin (TU Berlin) explored a human-centric approach to technology assessment across 17 industrial pilots to figure out how to solve the problem of measuring workers’ well-being, autonomy, trust and participation alongside traditional technical performance indicators.
When a company introduces a new machine or piece of software, it usually checks whether production has gone up, errors have gone down and tasks take less time. If the figures improve, the implementation is considered a success. But a machine that increases output can also leave its operator more tired at the end of a shift, and software that reduces errors can give employees less say over how they organise their work.
Researchers (Arina Rahma, Quang Thanh Ngo, Nafisah Animashaun and Nayab Bushra) explored the topic and presented outcomes of their study, “Beyond Technical KPIs: A Human-Centric Framework for Technology Implementation”, at the AI Hype, Hope & Humanity Conference in Rotterdam on 30 September and 1 October 2026.
Why productivity figures are not enough
Industry 5.0, the European Commission’s vision for the future of industry, asks companies to consider workers’ well-being as well as productivity. Doing so in practice is difficult. Throughput, error rates and time spent can be measured in the same way in any factory. Trust, stress or a sense of control at work cannot, as they depend on the person and the workplace. As a result, most assessments focus on technical performance and leave the human side out.
What the 17 pilots discussed
To understand what else organisations should be measuring, the researchers held a workshop with managers from the 17 SEISMEC pilots. The pilots come from sectors as different as aerospace, automotive, agrifood, healthcare, energy, retail and tourism.
Using a shared online board, participants discussed how new technology had affected four areas of their work: organisation, safety and health, privacy and ethics, and tasks and functions.
The researchers grouped the results into nine themes, which they call social KPIs because they describe how technology affects people rather than production.
- Three of these themes concern individual employees: how much control they keep over their own tasks, whether they trust the technology, and whether they receive enough training to use it.
- Three concern the tasks themselves. Automation often reduces physical effort but adds monitoring and decision-making, so the workload becomes more mental than physical. Safety and health conditions belong in this group too, along with the question of how all these changes can be measured.
- The last three concern the organisation as a whole: who decides how technology is introduced, how privacy and fairness are protected, and whether the technology suits the company’s way of working.
Three questions for every assessment
According to the study, technology changes work at three levels that influence each other: the individual, the task and the organisation. A management decision changes how tasks are designed, and the way employees experience those tasks affects how well the technology performs.
The researchers therefore suggest that organisations assess new technology with three questions.
- Does it perform well?
- Does it change people’s work?
- How was the change decided and managed?
Adoption rather than adaptation
The study also looks at how technology is introduced. In many workplaces, employees are told to adapt to a new tool once it has already been installed. The researchers recommend involving them from the beginning, taking their views into account and then checking how well the technology has been adopted.
When employees are involved early, they get to know the technology before it changes their routine. With suitable training and a clear idea of how it benefits them, they are more likely to accept it. The researchers describe this as prioritising adoption over adaptation. SEISMEC’s Guidebook for Worker Participation gives practical advice on how to involve workers in this kind of process.
Limitations
The authors point out two limitations. The workshop collected the views of managers, not of the employees who use the technologies. And because the pilots work in such different conditions, their results could not be compared with one another. Future work will need to include workers’ own experiences.
Technical KPIs remain necessary for judging whether a technology works. This research adds a second question to the assessment: whether it works for the people using it.