Discover the HumanTech work: Wearable technologies for construction
Welcome to the 'DISCOVER THE HUMANTECH WORK' blogs, a series where we dive into the HumanTech work packages (WPs) with our WP Leaders!
Let’s start this series with WP4, wearable technologies for construction, led by Bruno Walter Mirbach, Senior Researcher at the Department of Augmented Vision at DFKI.
This WP is composed of four tasks:
- T4.1: Inertial sensors - wearable camera integration, led by sci-Track
- T4.2: Intention prediction and exoskeleton integration, led by Tecnalia
- T4.3: Wearable camera digital twin localisation, led by DFKI and
- T4.4: XR-glass integration and BIMxD visualisation framework, led by HoloLight
In this blog, Bruno presents the work carried out in the first half of the project, the benefits and the impact of the work done, its liaison with the other WPs and what is next!
WHAT WP4 ACHIEVED
Work package 4 focuses in the development of wearable technologies for construction. These technologies will support workers in physical and stressful situations to enhance their efficiency, safety, and satisfaction. Moreover, extended reality (XR) glasses and deep-learning methods applied to body cameras shall ease the use and effectiveness of digitalisation methods in construction.
A key achievement in the first half of the project has been the development of a visual-inertial body sensor network (BSN). Combining a network of Inertial Measurement Units (IMU) with camera-based visual tracking allows for robust pose tracking of the worker. We will use this as input to an artificial intelligence (AI) algorithm that detects workers' intentions and automatically activates an exoskeleton when needed.
The integration of the IMU network with the camera system has been achieved by the HumanTech partner Sci-Track, with support from RICOH, who has developed an entirely new prototype of a dual-fisheye 360° camera to integrate the visual BSN. In parallel, Tecnalia has built a test bench with IMU and physiological sensors to determine the actions in which workers need support and develop an intention recognition algorithm for exoskeleton activation.
HoloLight has integrated the visualisation of a BIM model in the HoloLens into its software framework. Both HoloLight and DFKI, in collaboration with sci-track, are moreover working on a camera-based localisation of a worker with respect to a BIM model. We can use these functionalities for an augmented reality (AR) visualisation of progress monitoring or issue investigation through a BIM model.
In HumanTech, we are not working in a silo – let’s discover how this WP is linked to the others, the impact and the benefits of this WP.
We will demonstrate the exoskeleton with automatic activation function in a use case of Pilot 2: human-robot collaboration and wearables. Therefore, from the start of the project, there has been a close collaboration with the pilot owner and other involved partners — in particular, from WP5: construction robotics and human-robot collaboration — to define use cases for the exoskeleton.
WP4 is also very closely linked to WP2 and WP3. WP2 provides the BIMxD backend, the communication and processing platform from which the BSN and the XR glasses receive content from the BIM model generated within WP3. We can either visualise this content in the XR glasses or use it to determine the camera's pose with respect to the BIM model. Therefore, we have organised multiple workshops between WP4, WP2, and WP3 to define extensions of BIM standards as well as interfaces and services of the BIMxD backend.
WP3 partners have strongly supported WP4 partners in scanning lab space for generating BIM models. On the other hand, the development of a camera localisation and the augmented visualisation of BIM content allows to visualise the benefits of the methods developed in WP3 (dynamic semantic digital twin generation), thus providing added value.
Moreover, RICOH's development of the novel 360° camera brought an unforeseen benefit to the project. Initially designed for integration into the BSN, the camera turned out to be an ideal environment perception sensor for the robotic platform we developed in WP5, into which we will now integrate it.
The exoskeleton with an automatic activation function is one of the project's key exploitable results because it overcomes the downside of existing exoskeletons, which is the need for manual activation or the fact that it impedes natural motion. With the exoskeleton's so-called transparency, the range of applications will increase, as will the acceptance of exoskeletons in construction.
Computer vision and deep-learning methods, such as the automatic localisation of a camera with respect to a BIM model and the XR visualisation, can strongly support the demonstration and exploitation of the methods developed in WP3.
WHAT IS NEXT
The clearly indicated final step for the development exoskeleton with automatic activation function is the hardware- and software integration of the visual BSN with the exoskeleton, the optimization of the pose tracking and intention recognition algorithms and the system validation. The final system with be demonstrated within a use case of Pilot 2.
An ongoing task is the research on methods to localize accurately the pose of a camera with respect to a BIM model and to implement the communication between the body sensors and the BIMxD backend.
In the coming weeks, we will continue to share our progress on our different work packages — stay tuned to our news, social media (LinkedIn and Twitter), and to stay up to date!
Tech4EUconstruction cluster at the European Robotics Forum 2024
Our second workshop at the European Robotics Forum was a great success! It allowed us to share our knowledge on the benefits of AI and robotics in achieving automation in construction — based on our research and development activities — with more than 100 professionals in the AI and robotics area.
We organised a workshop on “AI and Robotics in Construction” in collaboration with our sister projects, the EU-funded BEEYONDERS and RoBétArmé, for the second year in a row, which attracted a huge audience.
Our project coordinator, Jason Rambach (DFKI), introduced HumanTech, highlighting the groundbreaking technologies we are developing to revolutionise the construction sector.
Antonio Alonso Cepeda (ACCIONA) and Dimitris Giakoumis (CERTH-ITI) also introduced our sister projects, with which we created the Tech4EUconstruction cluster.
Watch the interview we did with them, in which they reflect on the successful outcome of our workshop, the importance of the cluster for our projects and the opportunity it offers us to collaborate and advance AI and robotics in construction.
https://www.youtube.com/watch?v=SBN9XnfpyUg&list=PLLFSbOsrWIuaKCMQwGqsuvQ3zEEhhalnV
During the workshop, Gabor Sziebig (SINTEF), Renaud Detry (KU Leuven) and Maria Teresa Lázaro (ITAINNOVA) addressed key aspects of AI and robotics necessary for introducing robotic automation in construction sites, focusing on robot vision, navigation, control and HRI.
Patricia Helen Rosen (BAuA) shared insights from the first end-user evaluation we developed in HumanTech.
Our colleagues Gabor Sziebig and Sascha Wischniewski (BAuA), together with Herman Bruyninckx (KU Leuven), Jose Carlos Jimenez (TECNALIA) and Alberto Landini (STAM), held an insightful panel on challenges and lessons learned within our projects.
To conclude, we had an interactive feedback round with the attendees.
Thanks to all who joined us!
Want to keep updated with the latest news from HumanTech and our Tech4EUConstruction cluster? Subscribe to our newsletter.
Andrea Serrano Campo on creating a more diverse, inclusive and equitable STEM field
Our colleague Andrea Serrano Campo, innovation engineer at ACCIONA, participated in our first-ever Hackathon, which we recently celebrated at ACCIONA's facilities in Madrid, Spain. On the occasion of International Women's Day 2024, we spoke with her about her vision for a more diverse, inclusive and equitable STEM (science, technology, engineering, and mathematics) field, where women remain significantly underrepresented.
Andrea is an MS civil engineer. She was born in Santander (Spain) and lived in the US and Italy during her student years. A few years ago, she moved to Madrid, where she worked as a consultant in the transportation sector for more than six years.
She now works in the construction sector as an innovation engineer at our partner organisation ACCIONA, a Spanish company with a presence on five continents and over 65 countries, focused on developing and managing sustainable infrastructures and renewable energy solutions. In particular, she works in the Construction Technology Centre's Technology and Innovation Division, supporting different engineering projects.
When asked what has brought her to where she is now, Andrea reflects on her fascination with science, technology and research since childhood. "Perhaps the fact that my father was an engineer only fueled that curiosity further. My family has always supported my academic and personal growth, allowing me to pursue my interests and become the professional I am today," she says.
For International Women's Day 2024, we took the chance to ask her about her vision for a more diverse, inclusive and equitable STEM field, where women make up just 29.2% of all workers. She also shared how she thinks HumanTech could help address the construction industry's skills shortage and gender gap (only 8% of building industry workers are women), leading to a more balanced workforce.
This is what she told us:
Q: Hi Andrea! Can you share your experiences with female role models who have positively influenced your career journey?
A: Throughout history, there have been numerous pioneering women in civil engineering and computer science whose work has laid the foundation for advancements in these fields. Women like Ada Lovelace, Grace Hopper, and Margaret Hamilton made groundbreaking contributions to computing and software engineering, shaping the way we understand and interact with technology today.
In the field of civil engineering, without a doubt, I would highlight Emily Roebling (1843-1903). She was the first woman to be the chief engineer on-site, and what a project it was — none other than the Brooklyn Bridge!
Their achievements have inspired countless individuals, regardless of gender, to pursue careers in STEM. Their dedication, innovation, and perseverance serve as a source of motivation for aspiring engineers and technologists, highlighting the importance of diversity and inclusion in driving progress and innovation in the field of technology.
Q: What advice would you offer to fellow women aspiring to pursue a career in your sector, civil engineering?
A: If a woman asks me for my advice now, this is what I would tell her:
- Have confidence in your abilities and believe that you belong in the field. Your gender does not define your capabilities.
- Stay curious and committed to lifelong learning. Keep up with advancements in technology, regulations, and industry trends by taking courses, attending workshops, and pursuing certifications relevant to civil engineering.
- Recognise that challenges and setbacks are a natural part of any career journey. Stay resilient, learn from failures, and use them as opportunities for growth and development.
- As you progress in your career, serve as a role model and mentor for other women aspiring to enter the area. Share your experiences, offer guidance, and support efforts to increase diversity and representation in the industry.
"Have confidence in your abilities, stay curious and committed to lifelong learning, recognise challenges as a natural part of your career and become a role model for other women aspiring to enter the STEM area."
Q: That is such an inspiring advice! In your opinion, what measures could be taken to promote greater diversity and inclusion in STEM? What changes do you think would contribute to creating a more equitable environment for women?
A: To create a more diverse, inclusive, and equitable STEM field for women, targeted efforts must be made to tackle the specific challenges and barriers they face.
For example, encourage and support girls' interest in STEM subjects from a young age, address prejudice and stereotypes that discourage women from pursuing careers in this area, develop mentorship and networking programs that connect female students and professionals with experienced mentors and peers, or promote female representation in leadership roles.
Q: Reflecting on your personal experiences, are there any aspects of your journey you would have liked to see handled differently to make it more inclusive?
A: That´s a good question… Because although, theoretically, everything has evolved over the years, there are still pending issues and milestones to achieve.
Something I wish had been different in my case is the availability of programs encouraging young people, especially girls, to foster their interest in engineering from an early age. These could include workshops, science and technology camps, and visits to engineering companies and institutions to showcase engineering as a viable and exciting career option.
Also, it would be helpful to enhance the visibility of women and minorities in engineering roles by acknowledging their contributions through awards, publications, and conferences and establishing them as role models for future generations.
Q: At HumanTech, we are focused on advancing digitalisation within the construction sector. How do you think this evolution can make the built environment more inclusive and diverse?
A: The advancement of digitalisation in construction can significantly contribute to making it more inclusive for women. Digitalisation allows greater access to educational resources, online courses, webinars, and virtual workshops, making it easier for women to acquire knowledge and skills in construction-related fields without traditional physical or temporal barriers. It can also improve workplace safety through technologies such as virtual reality for safety training and real-time monitoring of working conditions, making the sector more attractive and accessible.
"The advancement of digitalisation in construction, which HumanTech focuses on, can significantly contribute to making it more inclusive for women."
Andrea's role at HumanTech
At HumanTech, the team Andrea works with is involved in various tasks, such as:
- Integrating wearables into a single intelligent system that can support construction workers on demand, transparently and without obstruction.
- Setting up the basis for deploying human-robot collaboration (HRC) technologies in construction scenarios, from demolition to new build, providing support to operators, ensuring safety, and overall improving working conditions.
- Creating and delivering new educational resources to train and improve health and safety and specialised green skills to make the sector more attractive, especially for young blue-collar workers, females, technical colleges, and universities.
Meet Andrea's colleague, Anurag Bansal, innovation manager at ACCIONA’s Construction Technology Center, who leads the work on two of our five pilots to validate our different HumanTech technologies.
Want to know more stories from our HumanTech team? Subscribe to our newsletter and follow us on LinkedIn and Twitter to keep up to date!
HumanTech 1st Robotic Integration Hackathon
From 26 February to 1 March 2024, we celebrated our first Robotic Integration Hackathon at ACCIONA’s Construction, Technology, and Innovation Division’s facilities in Madrid, Spain. Colleagues from different partner organisations involved in our work on Construction Robotics and Human-Robot Collaboration tested and integrated some of the construction technologies we are developing, which we'll use in our pilot projects.
It was the first time we integrated some of the HumanTech technologies developed by our partners over the past few months into single systems. On the one hand, a mobile robotic platform (from Baubot) to help construction workers build walls by handing over bricks. On the other, a robotic arm to fill concrete joints with elastic material (teleportation and autonomous operation by TECNALIA and SINTEF).
"Robots will be an everyday coworker on construction sites, and we are paving the way for them in HumanTech", says Gabor Sziebig, research manager in robotics and automation at SINTEF and leader of our work on construction robotics and human-robot collaboration.
It was truly exciting to see our technologies in action and a great example of how we are building the future of construction.
Big thanks to our colleagues at ACCIONA and Anurag Bansal for welcoming and hosting us, and to Gabor Sziebig and SINTEF for their efforts in organising this key event, which is a really exciting milestone for our project.
Stay tuned to learn more and watch the interviews we conducted with some of our colleagues at the hackathon! Follow our news and social channels (LinkedIn, Twitter and YouTube – where you can find the HumanTech Hackathon playlist) and subscribe to our newsletter.
https://humantech-horizon.eu/2024/06/11/humantech-hackathon-gabor-sziebig/
HumanTech Mid-Term Review Meeting: Consolidating our innovations to drive digitalisation in construction
At HumanTech, we have reached a new milestone: successfully celebrating our Mid-Term Review Meeting! From 22 to 24 January 2024, representatives from the 21 partner organisations that comprise our team gathered in Zurich, Switzerland, to comprehensively review our progress since we started working in June 2022 and agree on our next steps. A process that has helped us align our priorities to further our mission — to achieve breakthroughs in cutting-edge technologies, contributing to a safer, more efficient and digitised European construction industry.
“We successfully completed the HumanTech project Mid-Term Review Meeting with presentations and demos that are highly convincing of the project's scientific excellence and contribution to bringing AI to the construction industry” — Jason Rambach, HumanTech Project Coordinator.
To warm up and take advantage of the fact that we were in a perfect place to learn, we made an insightful visit to a construction site at our partner’s IMPLENIA headquarters.
The meeting started with an introduction by our Project Officer at the European Commission and our Expert Project Reviewer.
Next, Jason Rambach, our Project Coordinator (from DFKI), presented an overview of our objectives and the current state of our work, providing a solid foundation for the day’s discussions.
In-depth analysis of our technical developments
Leaders of the different areas of HumanTech made detailed presentations on the technical work we are developing, highlighting our most notable advances.
- Our progress on ‘BIMxD Formats and Standardization’ was presented by Rachele Bernardello (University of Padova).
- Ruprecht Altenburger (ZHAW) delved into ‘Dynamic Semantic Twin Generation’.
- Bruno Mirbach (DFKI) brought insights into ‘Wearable Technologies for Construction’.
- Gabor Sziebig (SINTEF) showcased our advancements in ‘Construction Robotics and Human-Robot Collaboration’.
- Gloria Callinan (TUS) shared details on ‘Human Factors - Training and Usability Assessment’, a key aspect of the project.
- Fabian Kaufmann (RPTU) highlighted the ongoing work on ‘Pilots, Evaluation, and Validation’, providing insights into the practical implementation of our technologies.
- Giulia Pastor (AUSTRALO) explored the ‘Outreach, Exploitation, and Collaboration’ efforts we have undertaken.
Throughout the presentations, we received constructive feedback and recommendations from our Project Officer and Reviewer, which will help us to further boost our success.
Demo session and feedback
The meeting concluded with an impressive live session of our innovative and interdisciplinary demos, showcasing our:
- scan-to-BIM pipeline (ZHAW, DFKI, RPTU),
- wearable localisation (Sci-track – DFKI),
- object pose estimation for grasping bricks (DFKI),
- and Human-Robot-Collaboration interfaces (SINTEF).
“On these reviews, the demos have always been my highlight. Especially on a project the size of HumanTech, it is really inspiring how partners from different disciplines collaborate on these prototypes and present an integrated proof-of-concept for complex problems”, says our colleague Markus Miezal, Co-Founder of Sci-track. He had the opportunity to demonstrate an initial version of a visual-inertial body tracking sensor of a worker within a Building Information Model (BIM) — an integrated prototype that includes a 360-degree camera from our partner, RICOH.
This meeting was packed with valuable discussions and learnings that will fuel our project’s crucial next phase as we continue to shape the future of construction.
Special thanks to our HumanTech team, who did a fantastic job, and to IMPLENIA and our colleagues Patrick Roth and Sebastian Mattes for being such good hosts!
Stay tuned for more updates on our exciting journey.
Welcoming new EU projects to the Tech4EUconstruction Cluster
At the Tech4EUconstruction cluster, we are proud to announce that four innovative European-funded projects have joined our mission of revolutionising construction technology: REINCARNATE, BIM2TWIN, RECONMATIC, and InCUBE. Each brings unique strengths and will enrich our collaborative efforts towards accelerating the digitisation of the EU construction industry and boosting its sovereignty.
This marks a significant step towards collaborative innovation. By pooling expertise and resources, we aim to address industry challenges, drive technological advancements, and shape the future of construction towards a sustainable and digital era.
Get to know the new projects
REINCARNATE
The REINCARNATE project mission is to revolutionise sustainable practices in construction. Focused on circular economy principles, it aims to breathe new life into materials, promoting recycling, reuse, and reduced environmental impact. It is developing a platform to provide information on the life cycle and reuse potential of construction materials and methods to predict and extend product lifetime and will demonstrate ten innovations based on this data.
BIM2TWIN
With a focus on Building Information Modeling (BIM), BIM2TWIN seeks to create interconnected digital twins of construction projects, promising heightened efficiency and accuracy throughout the construction lifecycle.
The project’s primary aim is to provide the construction sector with a breakthrough platform for construction management. This innovative tool is based on the Digital Building Twin (DBT) concept enriched by several functionalities in relation to real-time data collection and end-user applications. Relying on lean principles and adopting a comprehensive, holistic approach, BIM2TWIN’s ambition is to reduce operational waste, shorten schedules, reduce costs, enhance quality and safety and reduce carbon footprint. The DBT platform that BIM2TWIN is developing will provide situational awareness across the entire construction process, supporting a closed loop Plan-Do-Check-Act construction mode.
RECONMATIC
With a vision to streamline and enhance construction processes through automation, RECONMATIC proposes a set of innovative tools and techniques for managing construction and demolition waste (CDW) throughout the whole life cycle of buildings and infrastructures, addressing European targets for waste and energy consumption reduction in construction.
InCUBE
InCUBE envisions to unlock the EU renovation wave through cutting-edge standardised and integrated processes based on industrialisation, innovative renewable energy technologies, digitalisation, and new market entrants. All while accounting for social inclusion, gender mainstreaming and upskilling of the current and potential workforce. The solutions emerging from InCUBE will be validated in 3 large-scale demo sites: Zaragoza (Spain), Trento (Italy) and Groningen (Netherlands).
As we continue our collective journey, we invite you to stay tuned for updates on the progress of these groundbreaking projects. Together, we are committed to boosting a new era of construction where technology, sustainability, and innovation converge for a safer, greener, and more efficient industry.
Visit the Tech4EUconstruction cluster website for more details about these projects, and follow the #Tech4EUconstruction hashtag on social media to stay in the know about our updates!
Unlocking the future of research: Joana Goerg, Master's student at RPTU Kaiserslautern-Landau and intern at SINTEF
Joana Goerg is the protagonist of the fifth edition of our series "Unlocking the future of research". She is currently involved with two of our partners — a great example of collaboration within our HumanTech Team! On the one hand, she's studying for a master's degree at the RPTU Kaiserslautern-Landau University, with a strong reputation in research and teaching. On the other hand, she's doing an internship at SINTEF Manufacturing, one of Europe's largest independent research organisations, where she's working at HumanTech.
Learn about her passion for research and engineering, her work with robots and automation, and the ideas she is contributing to at HumanTech.
"I am fascinated by working in such an international company with such modern approaches. I hope I can continue to think ahead and implement new ideas!"
Even as a child, I was interested in research, and my fascination with bridges led me to study civil engineering. After completing my bachelor's degree in Mosbach (Germany), I am currently studying for a master's degree in deepening structural engineering.
My university in southwest Germany is the Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau (RPTU), where I got the fantastic opportunity to do an internship at SINTEF Manufacturing and work on the HumanTech project.
Before starting the internship, besides the amazing Norway, where SINTEF is based, I was very interested in the company's way of working, which has helped me to progress.
At SINTEF, I have been able to use my knowledge of construction technology to work with robots. In the HumanTech project, I contributed with a good idea of how bricklayers work, which enabled us to organise a live demonstration in a practical way and better adapt robots to the operations on the construction site.
I also spend a lot of time analysing point clouds, which need to be compared with each other and with a previously created Building Information Model (BIM) to enable automated construction site progress recording. I am doing initial tests in the lab to see how detailed changes to an object can be detected. In addition to the walls and columns, it should also be possible to detect colour differences, for example, due to a finished coat of paint.
My work aims to discover how detailed differences can be recognised and reliably detected. Later in the project, the detection of point cloud differences may be tested on real construction sites. Until now, I have taken 3D images myself, but a mobile robot will later do it automatically. The segmentation of the point clouds also needs to be further automated.
I am fascinated by working in such an international company with such modern approaches. I hope I can continue to think ahead and implement new ideas!
Want to discover more stories of the PhDs, junior researchers, master students and interns working with us? Read the fourth edition of our #FutureofResearch series with Irati Rasines, a PhD student at the University of the Basque Country and our partner Tecnalia, a leading private research and technology organisation in Spain.
Subscribe to our newsletter and follow us on LinkedIn and Twitter to learn more about our advances.
Lining up next steps in our remote-controlled demolition pilot
On 15 November, the HumanTech team involved in our remote-controlled demolition pilot met in Kaiserslautern, Germany, to discuss solutions and next steps.
In particular, Chae Soungho and Nakajima Makoto, from our Japanese partner Kajima, visited our colleagues at RPTU Kaiserslautern-Landau, Christian Glock, Marius Schellen and Maximilian Kunz, and were joined by our project coordinator, Jason Rambach (DFKI) and our RICOH colleagues So Okumura and Hideaki Kanayama.
A fruitful and clarifying meeting that will help us move forward with this exciting work!
Subscribe to our newsletter and follow us on LinkedIn and Twitter to keep up to date on all things HumanTech.
HumanTech's progress update in month 16
As we approach the halfway point of HumanTech, we have taken the chance to ask our team to share the highlights of our latest developments and what our next steps will be.
Here is what ten of the partners involved in our project have told us.
Jason Rambach, HumanTech project coordinator, DFKI
At HumanTech, we are headed for the project half-point and our first review in January. In the last months, we have seen many of our technologies, such as the robotic perception and human interaction, the BIMxD platform and the full Scan2BIM pipeline, starting to take shape, as shown in our exciting demonstrator session in our last General Assembly Meeting in Oslo at the end of August. Currently, we are very excited to start to start work on integrating the technologies into our robotic platform in February 2024.
Meanwhile, we have proven the scientific excellence of our results at the ICCV 2023 conference, the premier event for computer vision internationally, where we presented a publication on Scan2CAD with retrieval and deformation of objects and received 3 awards at the BOP Object Pose Estimation challenge. Our next stop is WACV 2024 in January, where we will present our semantic segmentation work on panoramic images with depth information. In addition, we have started using the Horizon Results Booster service to improve our exploitation perspectives and will develop more user evaluation workshops in the coming months.
Gabor Sziebig, SINTEF
Our work in recent months has been twofold. On the one hand, we have made progress with human-robot collaboration scenarios. We are getting the first results on how construction professionals can use this system — and, later on, on our pilot sites. On the other hand, we have advanced on the overall system integration, where the simulator developed by RPTU is coming alive, and more and more functionalities are accessible. A minor contribution towards BIMxD generation is also worth mentioning, where we are able to refresh and update the algorithms we have developed in previous R&D projects.
Arantxa Renteria, Tecnalia
Regarding our work in wearables, we have defined our use case. We are acquiring signals from our Body Sensor Network from several volunteers to train the database and infer the detection of movements and gestures. We will use this data to develop an algorithm to predict the user's intention and activation of our exoskeleton. The plan is to have a first prototype of this algorithm by January 2024. Future plans include developing the controller for the exoskeleton, integrating components (kinematics information from SciTrack) and tests.
About our work on robot teleoperation and learning by demonstration, we have developed drivers for different elements of the teleoperation console (haptic device, clutch) and a constraint-based admittance teleoperation controller (Force-feedback and Clutchless). We have defined the architecture of control middleware (ROS-control) and started SW developments. There is a first version of the robot platform simulator (UDP, control of the speed of the tracks), and we have integrated the control middleware and the simulator. Our next steps include developing means to enable/disable robot tools, real robot platform CAN driver and ROS node and completion of the control-middleware.
On robotic learning from demonstration, we have implemented an algorithm in a robot simulator with clean signals and a 6D mouse, a learning algorithm in a real robot and the first version of the user-friendly interface, created a signal processing block to homogenize teleoperated data, and developed a robot independent GUI. Our future work involves the 3D manufacturing of the mastic applicator, stabilization of the learning algorithm according to the sensed signals, implementation of artificial potential fields to adapt the generated trajectories to new situations, and extension of the learning algorithms to handle forces.
Finally, we worked on human factors at the beginning of the project by defining questionnaires. We will resume this task when the project validation phase starts.
Anurag Bansal, ACCIONA
In the last period, we visited our partners from DFKI and BAUBOT to discuss our progress and preparations for the Hackathon we have planned for next year in ACCIONA. In addition, we have provided samples of bricks, which we will use during next year's SINTEF and DFKI demonstrations.
On the other hand, we are planning to do two poster presentations at international conferences focused on Innovations in Building & Construction in 2024. We would like to present a poster on Cobots (ACCIONA user case for human-robot collaboration for handing over bricks to masonry worker) and another on Exoskeletons (ACCIONA user case for usage of exoskeletons for assisting workers during lintel placement and other construction activities).
Lastly, we had conversations with different ACCIONA work-site teams and explained to them what HumanTech is about to see if they could provide us with their availability for next year's workshop/focus group.
Patricia Rosen, BAuA
Together with our partners Tecnalia, TUS and ACCIONA, we collected our first user insights about some of our HumanTech technologies. We collected data at three different sites from potential users in the construction industry and were able to gain first results on comparing interactive robots, exoskeletons and extended reality (XR) glasses, for example, in relation to the perceived organisational relevance. Potential users also shared their expectations about the different technologies, as well as foreseen risks and opportunities, with us. The description of our procedure, analysis and results are part of our deliverable on 'Worker Assessment Report'.
We are currently planning additional user assessments with specific target groups and evaluating our technologies more in-depth. We are looking forward to completing the human perspective with valuable information provided by different employees.
Based on the results we gathered, we submitted an abstract to the annual German Human Factors conference in the spring of 2024.
Hideaki Kanayama, Ricoh Europe
We are currently advancing in two areas:
Spherical camera prototype
We have been developing a compact spherical camera with a wide field of view of 360 degrees and hardware-based trigger input for seamless integration with the body sensor network.
The camera mounted on the helmet captures an ego-centric view of the worker, and its pose is estimated with an image-based machine-learning algorithm. The estimated pose compensates the drifting error of IMU-based pose that is caused by the electromagnetic field in the environment, which achieves more accurate pose estimation. Furthermore, this camera will be used in forthcoming tasks with other partners. I.e., the cameras are mounted on the UGV with stereo settings and exploited for real-time advanced perception and human safety, such as workers’ 3D pose and position estimation with a wide field of view.
Development of 3D integration algorithm for multi-sensor data
At Ricoh, we are developing a spherical RGB-depth scanner that can capture a 5 m space with a 360° field of view within 1 s. The device is handheld and portable, allowing it to scan areas where automated mobility cannot enter.
At HumanTech, our scans are aligned and integrated with wide-area 3D data acquired by UGV/UAV to create a complete digital twin. We have developed an image-based localization that combines Sfm (Structure from Motion) and RGBD-SLAM (Simultaneous Localization and Mapping), which enables the alignment among all of the multiple sensors that can acquire a spherical RGB image. While this technology has been developed for static scenes, the next step is to develop a daily update algorithm for scenes in actual construction sites.
Sebastian Mattes, Implenia
Implenia, as a construction company, represents the end user’s point of view: How can the solutions we are developing be used, which requirements and framework conditions should be observed, and what does a real construction site look like? Our approach is to go directly on-site as early as possible to check if our thoughts are correct.
In the last months, we have been involved in preparations for our bridge inspection pilot. Regarding progress management, continuous scan recordings are being done on a site in Germany. As for the training of the AI algorithms, there is a need for more data also from a different project. We are checking out which project would fit best and is also reachable for our partners to collect weekly or biweekly scans. Meanwhile, several aspects to take care of came up with the first project.
- Data privacy: How to avoid capturing persons? How do we remove them from our data?
- Darkness: As the days get shorter in winter, the daylight hours after the normal end time of projects become less. Sometimes, we would like to have a coloured scan or use the data otherwise, but as they are based on photos captured by the scanner, they aren’t useful in dark environments — even if it doesn’t matter for the point cloud and captured geometry.
Our next steps are mainly:
- To organize access to the next scanning project
- To give feedback on the realistic use of the developed use cases
- (Perhaps) To evaluate and give input on training data for worker
Michael König, STRUCINSPECT
We are currently working on finalising our process for the bridge inspection pilot and are doing the first test with data we received from ZHAW. A big challenge is still in the BIM integration of standards that will be handled together with the University of Padova and Catenda. We are also involved in the point cloud segmentation and BIM reconstruction pipeline.
Florendia Fourli, Hypercliq
At Hypercliq, we have collaborated closely with our partners to examine the components of the HumanTech system and how they interact. We have pinpointed the necessary interfaces between humans, software, robots, and wearables and started defining the interoperability modules to facilitate their operation.
Additionally, to align our efforts with the EU's Data strategy, we have conducted a comprehensive review of the current state of relevant initiatives at the European level. These initiatives have resulted in reference architectures, as well as completed or ongoing European R&D projects that address the construction sector's needs. We specifically focused on their approaches to system architecture or the frameworks they've established or followed.
Building on the insights gained from these activities, we have developed the initial version of the HumanTech System Architecture. This serves as the foundation for further technical advancements, with the goal of providing HumanTech systems as components that can function independently or as part of an integrated solution. Finally, we have supported and documented the definition of the HumanTech use cases to be used as the basis for implementing the HumanTech pilots.
Elena Petrich, European Builders Confederation (EBC)
During the project's first year, we provided feedback on the usability of technology, started discussing an SME-friendly training programme and supported the project’s developments through effective dissemination through its network.
EBC is currently envisaging, in cooperation with its French member CAPEB, a workshop targeting women in construction with the primary objective of gathering comments and observations from construction professionals on the human-centred technologies currently being developed by HumanTech.
As HumanTech progresses, EBC will play a role in the objectives of training, marketing, and sharing information on innovation among construction SMEs, the whole construction value chain and policymakers.
Learn more about our project and subscribe to our newsletter to keep updated with our progress!
Unlocking the future of research: Irati Rasines, PhD student at the University of the Basque Country and Tecnalia
For the fourth edition of our series ‘Unlocking the future of research’, we have spoken with Irati Rasines, a PhD student at the University of the Basque Country and our partner Tecnalia, a leading private research and technology organisation in Spain.
Learn about her research, focused on developing solutions for easy Robot Learning from Demonstration and partially funded by HumanTech.
My study focuses on enabling robots to learn from human operators based on a few demonstrations performed via teleoperation.
Robot LfD (Learning from Demonstration) presents a great opportunity to enhance learning efficiency by bridging the gap between human expertise and automated machinery.
Through teleoperation, skilled operators can impart their knowledge directly to robots in real-time, rapidly acquiring complex tasks. This direct transfer of expertise ensures that robots learn from the most experienced sources, providing them with a substantial advantage in terms of skill acquisition and refinement.
Robot LfD in construction: Revolutionising safety protocols
The implications for the construction industry are profound, with robots poised to swiftly adapt to new tasks and scenarios, significantly reducing training times and enhancing project execution. In fact, one of the most compelling aspects of LfD in the industry is its potential to revolutionize safety protocols. By allowing robots to learn from skilled operators, we can delegate potentially hazardous to automated systems, minimizing human exposure to dangerous environments.
"One of the most compelling aspects of Robot LfD (Learning from Demonstration) in construction is its potential to revolutionize safety protocols."
Balancing a dual role: A PhD and professional career
Sharing a common research line between a PhD and a European project offers a multitude of benefits that synergistically amplify the impact and efficacy of both endeavours. As the collective effort of a team to achieve a goal enables it to generate innovative solutions that may not have appeared when one is working on an individual problem. But on the other hand, balancing the roles of a full-time worker and a PhD student is undeniably demanding.
"Striking a harmonious equilibrium between the demands of a professional career and the rigorous academic pursuits of a PhD program is a constant challenge."
The dual responsibilities require a meticulous juggling act, as both worlds demand significant time, energy, and intellectual commitment.
Despite the considerable challenges, this dual role also provides unique benefits. It offers practical insights and real-world context that can enrich academic pursuits, while academic rigour fosters analytical thinking and problem-solving skills that can be directly applied in a professional setting.
Read the third edition of our #FutureofResearch series with Harsh Manoj Shah, a master's student at the Technical University of Munich and a working student at our partner organisation Holo-Light GmbH.
Subscribe to our newsletter and follow us on LinkedIn and Twitter to learn more about our advances.