30 must-read publications on digital and circular building
The European construction industry is facing significant challenges in sustainability, innovation adoption, and labour shortages, with almost half of its jobs in short supply. However, by accelerating the sector's digital and green transformation, we can significantly improve its competitiveness and resource efficiency and appeal to a new generation of highly skilled workers, thereby creating a more innovative, sustainable, and attractive industry.
At HumanTech, we have joined forces with seven other European-funded projects addressing these challenges in the Tech4EUConstruction cluster. In this article, we share 30 of our scientific publications, providing new insights into the scientific foundation of our project's innovations for researchers and industry professionals.
The science behind the Tech4EUConstruction cluster’s innovations
At the Tech4EUConstruction cluster, we aim to create a lasting impact by exchanging our expertise and technical innovations. This article explores the science behind the groundbreaking innovations we are developing in areas such as building renovation, sustainability monitoring, digital innovation technologies, energy efficiency, renewable energy, and materials & design.
What advancements in AI and robotics will shape the future of the construction industry? Check out the articles below to find out!
Topic 1: Building renovation and sustainability monitoring
1. Monitoring the sustainability of building renovation projects — A tailored Key Performance Indicator repository
This publication, developed by the InCUBE project, aims to assist in identifying suitable key performance indicators (KPIs) that can be used to assess the sustainability performance of buildings as they transition into zero-carbon, resource-efficient, and resilient structures.
2. Towards the digitalization and automation of circular and sustainable construction and demolition waste management – project RECONMATIC
This publication presents Reconmatic, a Horizon Europe Research and Innovation Action project that aims to develop novel tools, technologies, and methodologies that can contribute to multiple construction phases and project types or material and product life cycle stages.
3. Assessing the construction and demolition waste volume for a typical Mediterranean residential building
Released by Reconmatic, this study estimates the construction and demolition waste (CDW) produced by a typical multi-storey residential building in Greece, built in the mid-20th century, made of reinforced concrete and filling masonry walls. It also considers renovation procedures and presents challenges related to the disposal, recycling, and reuse potential of CDW types.
Topic 2: Digital innovation technology
4. From 3D surveying data to BIM to BEM: The InCUBE dataset
This paper introduces the InCUBE dataset, resulting from the project's activities, focused on unlocking the EU building renovation through integrated strategies and processes for efficient built-environment management (including the use of innovative renewable energy technologies and digitalisation). The dataset contains raw and processed data from an Italian demo site in Trento's Santa Chiara district, enabling multiple potential uses, investigations, and developments.
5. Introducing Noise for AirSim’s 3D LiDAR Sensor to Reduce the Sim2real Gap
In robotics, modeling sensor noise is important as it can affect the accuracy and reliability of a robot’s perception of its environment. It also allows for more accurate simulations of robotic systems, which can help improve their performance in real-world scenarios. The Beeyonders’ project proposes introducing a noise model for the 3D LiDAR (Light Detection and Ranging) sensor supported in AirSim to help the community develop more accurate, reliable, and cost-effective solutions.
6. Online Ergonomic Evaluation in Realistic Manual Material Handling Task: Proof of Concept
Work-related musculoskeletal disorders are a major cause of work-related injuries. To address this issue, work task ergonomic risk indices have been developed, but they are subjective and challenging to perform in real time. This work, released by Beeyonders, provides a technique to digitalize this process by developing an online algorithm to calculate the NIOSH index using inertial sensors, which can be easily integrated into the industrial environment.
7. REINCARNATE: Shaping a sustainable future in construction through digital innovation
The heart of the Reincarnate project is the Circular Potential Information Model (CP-IM), a digital platform designed to assess and enhance the recyclability of construction materials, products, and buildings. The CP-IM utilizes advanced technologies to revolutionize the handling of construction waste, turning it into valuable resources and reducing the sector's environmental footprint. Its features include digital tracing, material durability predictions, and CO2 reduction materials design, showcased in eleven European demonstration projects, highlighting significant reductions in construction waste and CO2 emissions.
8. Presenting SLAMD – A Sequential Learning Based Software for the Inverse Design of Sustainable Cementitious Materials
The composition of concrete has become more complex, especially with formulations aimed at reducing carbon footprint. Inverse Design techniques offer a solution by allowing for a comprehensive search to create new and improved concrete formulations. This publication introduces the concept of Inverse Design and demonstrates how an open-source app called SLAMD, developed by Reincarnate, provides necessary workflow steps to adapt it in the laboratory, lowering the barriers to its application.
9. Single Frame Semantic Segmentation Using Multi-Modal Spherical Images
In recent years, the research community has shown much interest in panoramic images that offer a 360º directional perspective. Multiple data modalities can be fed, and complementary characteristics can be utilised for more robust and rich scene interpretation based on semantic segmentation. In this study, HumanTech proposes a transformer-based cross-modal fusion architecture to bridge the gap between multi-modal fusion and omnidirectional scene perception.
10. U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Clouds
In this paper, HumanTech proposes U-RED, an Unsupervised shape REtrieval and Deformation pipeline that takes an arbitrary object observation as input, typically captured by RGB images or scans, and jointly retrieves and deforms the geometrically similar CAD models from a pre-established database.
11. Annotation rules and classes for semantic segmentation of point clouds for digitalization of existing bridge structures
Germany needs to digitize its extensive bridge infrastructure using BIM due to political requirements. This transformation involves using point cloud data and exploring available open-source datasets and various approaches to semantic segmentation. HumanTech aims to bridge the gap between theoretical research on point cloud data and manual inspection by proposing a set of object-oriented classes for semantic segmentation in this study.
12. OPA-3D: Occlusion-Aware Pixel-Wise Aggregation for Monocular 3D Object Detection
Monocular 3D object detection has advanced with the use of pre-trained depth estimators for pseudo-LiDAR (Light Detection and Ranging) recovery. HumanTech proposes a method that jointly estimates dense scene depth, depth-bounding box residuals, and object-bounding boxes, enabling a two-stream detection of 3D objects.
13. Ontology-Based Semantic Labelling for RGB-D and Point Cloud Datasets
Deep learning applications have recently surged in the construction field. Supervised semantic segmentation of 2D or 3D data acquired from buildings requires using annotated data for training, validation, and testing. However, existing datasets lack a common convention and definitions based on construction ontologies. In this work, HumanTech presents a guideline for ontology-based semantic annotation of RGB-D and point cloud datasets, bridging the gap between deep learning and computer science.
14. When Machine Learning Meets Raft: How to Elect a Leader over a Network
The Raft consensus algorithm is widely used to keep data consistent across multiple distributed nodes by having a leader node coordinate operations. However, the system pauses during leader elections, which can happen if the leader fails or gets disconnected from other nodes. In this paper, Reconmatic explores using Machine Learning to monitor and classify the causes of these leader elections, aiming to reduce unnecessary elections and increase system availability.
15. Machine-learning-assisted classification of construction and demolition waste fragments using computer vision: Convolution versus extraction of selected features
Reconmatic has developed a machine-learning-assisted procedure for identifying construction and demolition waste (CDW) fragments using an RGB camera. This approach improves waste sorting efficiency and accuracy, promoting sustainable resource use and reducing environmental impact.
16. Review of Concepts for Construction and Demolition Waste and the Circular Economy
This paper, developed by Reconmatic, examines the classification and management of construction and demolition waste (CDW) and the concept of circular economy (CE) in the construction sector. Its findings can guide practical measures to enhance waste management and inform planning and decision-making for waste reduction and recovery.
17. On Using Hyperledger Fabric Over Networks: Ordering Phase Improvements
Blockchain is increasingly being used in various research disciplines, such as the Internet of Things (IoT) and Software Defined Networking (SDN). Hyperledger Fabric is a popular enterprise-grade blockchain framework known for ensuring transparency in secure communication. One of its key features is the three-phase transaction flow architecture. This study released by Reconmatic focuses on improving the ordering phase by proposing a mechanism for faster communication.
18. RoBétArmé Project: Human-robot collaborative construction system for shotcrete digitization and automation through advanced perception, cognition, mobility and additive manufacturing skills
The shortage and rising costs of skilled workers, along with the need for new infrastructure and the maintenance of ageing infrastructure, are driving an increasing demand for construction automation. The RoBétArmé project aims to revolutionize construction with advanced technologies for shotcrete (sprayed concrete) automation. This paper showcases a novel robotic system for automating all phases of shotcrete application.
19. Adaptive BIM/CIM for Digital Twining of Automated Shotcreting Process
Developing digital twins for construction requires accurately replicating real-world spaces. This study, developed by RoBétArmé, emphasizes the importance of using Building/Civil-Construction Information Modeling (BIM/CIM) to create digital twins for construction, particularly for automated shotcreting of civil infrastructure projects. It highlights the need for simulations, visualizations, and adaptive modeling to monitor and control assets in real time.
20. Leveraging Multimodal Sensing and Topometric Mapping for Human-Like Autonomous Navigation in Complex Environments
Autonomous vehicles need to understand complex outdoor environments and follow traffic rules. RoBétArmé's approach is to imitate human driver behaviour using RGB and LiDAR (Light Detection and Ranging) data combined with a rough topometric map for route planning. Their method shows potential for safer and more human-like autonomous behaviours in urban and semi-structured environments.
21. Cognitive Fusion-based Path Planning for UAV Inspection of Power Towers
The use of Unmanned Aerial Vehicles (UAV) for inspecting critical power infrastructure has advanced significantly in recent years. This paper by RoBétArmé presents a novel path planning method that leverages robot vision derived from LiDAR (Light Detection and Ranging) and RGB data for inspecting power tower insulators.
22. Comparative Study of Surface 3D Reconstruction Methods Applied in Construction Sites
This research from RoBétArmé provides a comprehensive assessment of key methodologies for 3D reconstruction of construction sites. It evaluates monocular and binocular computer vision techniques for their ability to extract detailed 3D surfaces while considering their computational efficiency. The findings contribute to advancing 3D reconstruction techniques in the construction industry, which is essential for its digitalization.
23. Real-time 3D Reconstruction Adapted for Robotic Applications in Construction Sites
Integrating robot vision techniques, especially focused on 3D reconstruction, in the construction industry is crucial to meeting the digitalization needs of Industry 4.0. This study from RoBétArmé introduces a real-time 3D reconstruction pipeline that uses both RGB and depth information using common algorithms.
Topic 3: Energy efficiency and renewable energy
24. Dynamic Energy Analysis of Different Heat Pump Heating Systems Exploiting Renewable Energy Sources
Renewable energy source-fed heat pumps (HPs) may perform up to very high efficiency standards, offering a promising tool in the broader residential heat decarbonization effort. In this context, this paper, by InCUBE, investigates different heating configurations using various renewable thermal sources and an HP-based system to find the most efficient setup. Its findings can guide the ongoing design efforts for green residential heat solutions at the research and commercial implementation levels.
25. An integrated life cycle assessment and life cycle costing approach towards sustainable building renovation via a dynamic online tool
Building stock retrofitting is crucial for achieving the building sector sustainability goals due to its high energy consumption rates. This paper by InCUBE introduces VERIFY (Virtual intEgrated platfoRm on LIfe cycle AnalYsis), an online tool for dynamic life cycle analysis and global warming impact assessments. It evaluates energy retrofitting measures for a multi-family residential building in Athens, Greece, aiming to reduce environmental impact and achieve near-zero energy consumption.
26. BIM-Based Construction Quality Assessment Using Graph Neural Networks
In this paper, HumanTech presents a novel approach for automating construction quality control. This method improves element-wise quality assessments by utilizing the semantic information in Building Information Models (BIM). The approach involves representing the as-designed Building Information Models (ad-BIM) as a graph, encoding elements' topological and spatial relationships. By using this representation, the paper proposes an algorithm based on Graph Neural Networks (GNNs) to infer element-wise built quality status.
Topic 4: Material science and design
27. Data driven design of alkali-activated concrete using sequential learning
Released by Reincarnate, this paper presents a novel approach to developing sustainable building materials through Sequential Learning. The approach can be immediately implemented in practical applications and can be translated into significant advances in sustainable building material development.
28. LLMs can Design Sustainable Concrete - a Systematic Benchmark
In the context of a circular building material economy, resource flows' complexity and material composition variability pose significant challenges. This Reincarnate study demonstrates how Large Language Models (LLMs) can advance material design by adopting a Knowledge-Driven Design (KDD) approach that outperforms traditional Data-Driven Design (DDD) methods.
29. Beyond Theory: Pioneering AI-Driven Materials Design in the Sustainable Building Material Lab
This work, by Reincarnate, focuses on using Artificial Intelligence (AI)-driven materials design to improve the sustainability of building materials with complex formulations. It provides insights into the real-world application of data-driven design, showcasing the successful integration of AI to advance sustainable materials science and boost sustainable building in the construction industry.
30. 14 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathon
Large-language models (LLMs) such as GPT-4 have garnered interest from scientists for their potential in chemistry and materials science. Reincarnate organized a hackathon that showcased various LLM applications, including predicting molecule and material properties, designing novel tool interfaces, and extracting knowledge from unstructured data. This demonstrates the broad impact of LLMs across scientific disciplines beyond materials science and chemistry.
Share these research developments with your network!
Use our hashtag #Tech4EUConstruction on social media.
HumanTech's year 2 in review: A year of hard work and remarkable advances
As we conclude our second year at HumanTech, we have taken the chance to reflect on a period marked by significant achievements and progress in our mission to drive the digitalisation of the construction sector with cutting-edge, human-centred technologies. This year, we have reinforced our core values: collaboration, innovation, and scientific excellence, achieving milestones that push the boundaries of what is possible in construction technology.
Parallel to our continuous advancements in the scientific field, this year also saw the consolidation of our partner's individual results into integrated systems, which resulted in impressive demonstrations.
Milestones and achievements
A successful Mid-Term Review Meeting
A key milestone for the project was our 18-month periodic reporting and review, hosted by IMPLENIA in Zurich in January 2024. The event was unanimously a great success, validated by the entirely positive feedback we received.
The highlight of the review was the closing demo session, featuring live demonstrations of:
- Autonomous site scanning with an unmanned ground vehicle (UGV) by ZHAW
- Localisation and worker body pose tracking with a head-mounted camera and inertial sensors by SciTrack and RICOH
- Automated Scan-to-BIM pipeline by RPTU and DFKI
- Object Pose estimation for robotic grasping by DFKI
Progress in robotics and AI technologies continues
In the following months, we witnessed additional significant progress in bringing robotic and AI technologies to the construction industry. These results were shown primarily in two events: the HumanTech Executive Board Meeting in Genova, organized by STAM, and our Robotic Integration Hackathon, hosted in Madrid at ACCIONA’s construction test site.
During the hackathon, project partners had the first opportunity to test and integrate their components on the newly created BAUBOT robotic platform for HumanTech.
Some key results shown in these two events were:
- Robotic grasping and handover of materials (SINTEF, TECNALIA, BAUBOT, DFKI, ACCIONA)
- Teleoperated mastic application (TECNALIA, SINTEF, BAUBOT, ACCIONA)
- BIM visualization overlaid over real building in XR (HOLOLIGHT)
Demonstrating scientific excellence
During this period, our commitment to scientific excellence has been recognised through numerous awards and publications at major conferences:
- 1st place awards in 3 categories in the Benchmark on Object Pose challenge 2023 at the ICCV 2023 conference
- 3 rd place in the Scan-to-BIM challenge of the CVPR 2023 conference
- 8 scientific publications, among others, in the prestigious CVPRx2, ICCVx1, and WACVx1 conferences
An equally important part of HumanTech’s mission is actively engaging with different communities in the construction field and disseminating our project results. For the second year in a row, we co-organized, along with the Tech4EUconstruction cluster, a highly successful “AI and Robotics in Construction” workshop at the European Robotics Forum (ERF) 2024 in Rimini.
HumanTech resources available
Recently, we have made several resources from our project openly available to everyone through our website. These include all the public deliverables from the first 18 months of the project, source code repositories for many of our scientific publications, our micro-learning units, and a variety of datasets that will be enriched and extended in the future.
Looking ahead
We are not done just yet!
We still have another whole year ahead with one main challenge: Implementing our 5 project pilots! We will do it across diverse settings (office buildings, bridges) and countries (Spain, Switzerland, Germany, Singapore) between the end of 2024 and the beginning of 2025.
This article has been written by Jason Rambach, HumanTech’s Project Coordinator and Senior Researcher and Team Leader in Spatial Sensing and Machine Perception at the German Research Center for Artificial Intelligence (DFKI) in Kaiserslautern, Germany. Learn more about him in this interview.
To stay updated, follow our news and social channels (LinkedIn, Twitter and YouTube) and subscribe to our newsletter.
HumanTech Technologies: Visual-inertial tracking unit
In this edition of our HumanTech technologies series, our colleague Markus Miezal, CEO and Co-Founder of Sci-track, shares the keys to the inertial tracking technology we are developing — a collaboration between our partners at Sci-track and RICOH.
Body tracking and its problems
The term inertial measurement unit (IMU) usually refers to a sensor package containing an accelerometer, a gyroscope and a magnetometer. They measure 3D acceleration (including gravity), 3D rotational velocity and the 3D magnetic field (i.e. a compass), usually at a high frequency (100Hz).
Every smartphone has an IMU. When the display flips, the accelerometer detects a different "down" direction from gravity. A common application for these sensors is orientation estimation. By fusing the information from the accelerometer and the gyroscope, gravity and linear acceleration are separated so that a clean downward direction can be measured. The magnetometer adds yaw information.
When placing these sensors on the body, one can calculate the orientations of every limb, but with additional information, such as a biomechanical model, one can get accurate orientations across the body or — what is the most common use-case — the relative segment orientations, i.e. the joint angles.
The dimensions of the body allow us to better predict the measured accelerations, mechanical limitations of certain joints can be exploited, and in our case, the biomechanical foot model allows us to estimate positional information through ground contact estimation*.
Using a model, however, requires that we know where the sensors are placed on the body. This is estimated in a small calibration step. But also, the model dimensions, i.e. the segment length, have to be known and may be wrong.
Magnetometers are another error source. Especially indoors, any ferromagnetic metal or current-carrying electrical wires locally disturb the earth's magnetic field, so the measured "north" direction can be different for each sensor. We usually omit the use of the magnetometer completely.
Exploiting the relations between segments, we are able to maintain the yaw direction during motion but not in static situations. Also, a global yaw drift is introduced.
Adding a camera
By integrating a camera into the body and, in particular, a dual fish-eye camera, which is capable of capturing almost a full sphere, information about the surroundings and the body with respect to the surroundings can be obtained.
By detecting the body inside the image, the segment length and intrinsic segment orientations can be corrected (e.g. during drift in static conditions).
By monitoring the surroundings, the position information can be corrected and yaw drift omitted. Furthermore, the camera enables localisation and through object detection, context can be added to the estimate.
*Why can’t we get positions out of accelerometers?
Let’s say an IMU is static on a table. Neither accelerations nor rotations occur, so the IMU will measure gravity and zero rotational velocity. If I place the IMU in my car and drive on the highway at 180 km/h at a constant speed and then start measuring, as long as I don’t brake or accelerate, the accelerometer will only show gravity, and as long as I don’t turn, the rotational velocity will also measure zero. As you see, we cannot distinguish a static IMU on a table from an IMU travelling at a constant speed. We have to integrate the acceleration into velocity and to know the position, we have to integrate it again.
This will, however, always result in errors since the measurements are digital and usually biased. Digitalisation implies information loss since the measured value is quantized. For example, 16 bits correspond to 65535 different numbers, which are mapped to the sensor range of ±60m/s². The smallest change is, therefore, 1.83 mm/s². Biased means that the process of converting to a digital number comes with the problem of the zero point being shifted. So, instead of measuring zero, a small other value is measured, which we call bias. The high frequency of the IMUs adds to the problem so that the integrations quickly diverge, which we call drift.
Learn about another of our HumanTech technologies, which promises improved efficiency and accuracy for the architecture, engineering, and construction (AEC) industry: scan to BIM, explained by Mahdi Chamseddine, an M.Sc. researcher at the German Research Center for Artificial Intelligence (DFKI).
Stay tuned to our news, newsletter, and social media channels (LinkedIn, Twitter and YouTube) to follow our journey toward accelerating the digital transformation of construction!
HumanTech Technologies: Scan-to-BIM
In this edition of our HumanTech technologies series, our colleague Mahdi Chamseddine, an M.Sc. researcher at the German Research Center for Artificial Intelligence (DFKI), shares the keys to the scan-to-BIM technology we are developing.
What is scan-to-BIM and why is it important?
Building Information Modeling (BIM) is a process in architecture, engineering, and construction (AEC) that uses 3D models to centralize building project information. BIM integrates design, visualization, and collaboration, thus enhancing efficiency, accuracy, and communication throughout a building's lifecycle, from planning to construction and management.
Scan-to-BIM is the process of generating BIM models from 3D scans of existing structures, buildings, or construction sites.
"Scan-to-BIM is the process of generating BIM models from 3D scans of existing structures, buildings, or construction sites."
Scanning technologies provide accurate 3D point clouds of the target site. The scans reflect up-to-date information and real-world conditions, thus allowing for the generation of "as-built" models. The "as-built" building model can defer from the architectural plans, and thus, an automated approach to BIM generation is desired. This invaluable tool streamlines projects, facilitates efficient facility management, and empowers informed decision-making throughout a building's lifecycle.
How does scan-to-BIM work?
Scan-to-BIM starts with data acquisition. There are different methods for 3D scanning of buildings, such as terrestrial scanners or photogrammetry. The scanning devices capture high-resolution and detailed point clouds of buildings or sites. Some scanning technologies can capture colour as well as 3D points, which allows for more information to be extracted.
The point clouds are then processed to remove noise and clutter. A typical scan will include data that is not relevant for the BIM generation, such as neighbouring trees or buildings and objects outside the designated site. The processed point clouds are then segmented using specialized machine-learning algorithms that are trained to recognize structural elements in scans. The AI model classifies points into relevant classes such as walls, floor, ceiling, doors, columns, and other classes. The segmented point cloud is used to create structured 3D models for the different objects depending on their type and dimensions. The collection of the 3D models, as well as their relationships and other information, constitutes the BIM model.
Scan-to-BIM applications and challenges
Generating a BIM model of existing buildings is labour-intensive, but automating the process can allow us to reap the benefits faster and more cost-effectively.
"Generating a BIM model of existing buildings is labour-intensive, but automating the process can allow us to reap the benefits faster and more cost-effectively."
It allows for automated progress monitoring of construction sites by generating models at regular intervals. Changes to the project are tracked and stored, and deviations from the plan can be detected and highlighted at an early stage, eliminating the need for costly corrections down the line.
Scan-to-BIM can be used, among other things, in renovation, historic preservation, and industrial projects to create precise digital replicas of existing structures. However, some challenges, including maintaining data quality, managing large datasets, scanning hazardous or hard-to-access areas, and adapting to unique and different structures, still exist.
Advancements in scan-to-BIM technology promise increased efficiency and accuracy for the AEC industry.
Learn about another of the technologies we are developing at HumanTech: the 360° ToF camera. Its speed, portability, accuracy, and efficiency in 3D data collection are unparalleled, and it is set to transform the way construction sites are monitored and managed.
Stay tuned to our news, newsletter, and social media channels (LinkedIn, Twitter and YouTube) to follow our journey toward accelerating the digital transformation of construction!
HumanTech Technologies: 360° ToF camera
At HumanTech, we are excited to introduce a revolutionary innovation in 3D sensing technology: A novel portable omnidirectional RGB-D device with a 5-meter range in all directions. This cutting-edge scanner stands out from currently available commercial 3D sensing equipment by its ability to quickly capture a 360-degree dense point cloud in a single shot with just one second of exposure time. Its handheld and portable design enables it to capture data in areas that are challenging for conventional scanners, such as cluttered rooms and small spaces, making it an invaluable tool for the comprehensive monitoring and management of construction sites.
We are committed to driving the digitalisation of the construction industry — making it safer, more efficient, greener, and more attractive to a new generation of highly skilled professionals. To this end, we are achieving major breakthroughs in cutting-edge technologies with a human-centred design.
One example is the 360° ToF camera we have developed. Its speed, portability, accuracy, and efficiency in 3D data collection are unparalleled, and it is set to transform the way construction sites are monitored and managed.
Enabling frequent digital twin updates
Our main goal with this innovative device is to enable highly frequent digital twin updates and progress monitoring at the construction site, ensuring that professionals have the most recent information at their disposal.
It can also be used to support the creation of a complete digital twin by complementing the extensive scans taken by conventional terrestrial laser scanning and photogrammetry. 3D data from our sensors can also be aligned and integrated with data from other sensors using machine-readable marker detection and additional re-localization based on feature point matching between images obtained from each sensor. This improves the estimated location's accuracy, even in scenarios where some markers are not detected.
Verifying effectiveness through real-world applications
In order to introduce this device in the construction industry, we are verifying its effectiveness in practical scenarios through experiments at actual construction sites.
Collaboration and transparency are key in HumanTech. That's why our partners RICOH and DFKI have published the unique RGB-D dataset collected by this scanner in real buildings. This dataset, which consists of spherical RGB-D images with instance-level semantic and room layout annotations, is available for the research community to explore.
We expect our RGB-D scanner and this dataset to stimulate the development of novel algorithms that bridge the gap between research and practical applications in the workplace.
The prototype of the second version of the device has been completed and is already fully operational from April 2024. This version will significantly improve accuracy, extend the measurement range, and shorten shooting time. Further development is underway to achieve continuous dynamic updating and progress monitoring of the digital twin model.
As we continue to develop our technologies and expand the opportunities they offer, we look forward to seeing their impact on the building sector!
Meet our colleague Hideaki Kanayama, an engineer at RICOH — whose team at HumanTech is dedicated to developing the 360° ToF camera, among other technologies — and who provided the information for this article.
For more details on our developments, stay tuned to our news updates, newsletter, and social media channels (LinkedIn, Twitter and YouTube) to follow our journey toward accelerating the digital transformation of construction.
HumanTech Hackathon: Gabor Sziebig on the benefits of our technologies in construction
How can our HumanTech technologies benefit the construction industry and its workers? Who are the partners behind the robots we tested in our first Robotic Integration Hackathon? Watch this interview with our colleague Gabor Sziebig, Research Manager in Robotics and Automation at SINTEF Manufacturing, to learn more!
In our first Robotic Integration Hackathon, which we held at our partner ACCIONA's facilities in Madrid, we took the chance to interview some of our colleagues involved, who provided insights into their work developing the technologies we tested — a mobile robot to help construction workers build walls by handing over bricks and a robotic arm to fill concrete joints with elastic material.
Below, watch our interview with Gabor Sziebig, who is leading HumanTech’s Work Package 5, focused on Construction Robotics and Human-Robot Collaboration, to find out more details about the groundbreaking technologies we are developing and get a glimpse of our advancements, with which we are advancing digitalisation, automation, safety and efficiency in construction.
https://www.youtube.com/watch?v=w_FMM4jCUVk
The HumanTech partners involved
This initiative is a great example of the collaborative work that allows us to advance towards our goals at HumanTech, as six different partner organisations were involved in it: Baubot (which developed the mobile robot), Tecnalia (which created the control software that acts as the robot's brain), with a strong support from SINTEF, the University of Kaiserslautern-Landau in Rhineland-Palatinate (RPTU) and the German Research Center for Artificial Intelligence (DFKI) are also involved in it, as well as ACCIONA, on whose construction site we carried out this hackathon.
In the following weeks, we will publish the rest of the interviews we conducted. Stay tuned to learn more about our innovative technologies! Follow our news and social channels (LinkedIn, Twitter and YouTube - where you can find the HumanTech Hackathon playlist) and subscribe to our newsletter.
Discover the HumanTech work: Overall Framework Definitions
Welcome to the fourth edition of our ‘DISCOVER THE HUMANTECH WORK’ blogs, a series in which we discuss the HumanTech work packages (WPs) with our WP Leaders!
On this occasion, we dive into our WP1 — Overall Framework Definitions, led by DFKI. It is dedicated to setting the overall project framework in terms of elicitation of concepts, processes and information flows, detailed definition of usage scenarios, design of the overall framework architecture, technical specifications and user requirements, and ethics approach.
We have spoken with Jason Rambach, HumanTech’s Project Coordinator and Senior Researcher and Team Leader in Spatial Sensing and Machine Perception at the German Research Center for Artificial Intelligence (DFKI). He has shared the main activities carried out in the first project period, how this WP is linked to others and our plans for the remaining part of the project.
WP1 had a critical role early in the project: to establish a common ground and a shared understanding of the overall project objectives among the partners coming from different backgrounds (AI, robotics, civil engineering).
Therefore, significant activities such as in-person workshops during meetings and several online meetings were carried out in the first months. This led to the conclusion of the first phase of the WP, with three important deliverables that defined research requirements, a component-level architecture, and user requirements.
This work gave a significant boost to the technical WPs, mainly WP3, WP4 and WP5. A very important contribution was the definition of use cases and flow diagrams for the pilots, which proved highly valuable in the following months.
What remains to be done for WP1 is finalizing the HumanTech architectures and describing them in deliverable D1.4. For this, the task leader Hyperqlic has been monitoring all technical work packages closely over the last months and updating the architecture accordingly.
The final architecture was presented to the partners at the upcoming Executive Board Meeting in Genoa.
Take a look at our progress on WP8, focused on dissemination and communication and presented by Giulia Pastor and Andrea Torres from AUSTRALO, and stay tuned to our news and social media (LinkedIn and Twitter) to stay up to date!
Discover the HumanTech work: Dissemination, communication and exploitation
Welcome to the third edition of our ‘DISCOVER THE HUMANTECH WORK’ blogs, a series in which we discuss the HumanTech work packages (WPs) with our WP Leaders!
Let’s continue this series with WP8 — focused on dissemination and communication. Giulia Pastor and Andrea Torres from AUSTRALO will present key insights on the work done and the most exciting achievements so far.
WP8 is the work package responsible for the project’s dissemination, communication, and exploitation activities. It is led by AUSTRALO, with DFKI in charge of exploitation and standardisation and Hypercliq handling IPR activities.
The whole HumanTech team has invested a lot of effort in the first two years of activities to ensure that the exciting work carried out by our technical partners is well-promoted and all the HumanTech stakeholders are aware of what is happening in the project.
In addition, a significant part of WP8’s job is to MAKE THE PROJECT RESULTS OPEN to everyone interested, in line with the open science principles.
To achieve this important objective, our technical partners have published six open-access scientific publications, which can be consulted and downloaded on our HumanTech website.
They also participated in 24 events, workshops, conferences, and media appearances and won 3 prestigious awards:
- 2 first place awards in the Object Pose Estimation Challenge (BOP Challenge, ECCV 2022) | Partner: DFKI
- 3 first place awards at the Object Pose Estimation Challenge (BOP Challenge, ICCV 2023) | Partner: DFKI
- 3rd place in the CV4AEC workshop's Scan-to-BIM challenge at CVPR 2023 | Partner| DFKI + RPTU
In HumanTech, communication and dissemination are the keys to the project’s success.
All the key scientific stakeholders must be aware of the project activities; however, we must keep in mind that HumanTech stands for Human-centered technologies for a safer and greener construction industry and has the main scope of providing new technologies to make this sector more worker-secure and green.
With this in mind, we created different blog posts and social media campaigns, underlying, on one side, the project's technical achievements and, on the other side, the benefits that the new technologies we are developing will bring to our end users, the workers, and the construction companies.
In the NEWS section of our website, you can find insightful articles on our technologies and their benefits, together with articles summarising the first-year achievements and progress updates before our first review meeting.
To explain different aspects of HumanTech — from our pilots to our learning materials to skill construction workers — in a clear, didactic and engaging way, we have conducted several video interviews with different team members. Also, videos and animations to showcase different project updates, demos, learning pills and activities within our Tech4EUconstruction cluster.
In WP8, we believe that our colleagues, the multidisciplinary professionals from 21 organisations in 10 countries who make up the HumanTech team, are best placed to explain the work they are developing. This is why we have given them space to present themself, their background, skills, vision of the project and the importance of their work for the construction industry in the series of interviews: ‘Meet the HumanTech team’.
In parallel, we launched a new series, 'Unlocking the Future of Research', in which researchers, PhD students and junior staff working on the project explain what it means for a young professional and/or researcher to work on an EU project and what this can do to their future careers.
In HumanTech, we value our community.
In June 2023, we joined forces with our sister projects BEEYONDERS, and RoBétArmé, to create the collaborative cluster Tech4EUconstruction. Funded by the European Commission, under the call HORIZON-CL4-2021-TWIN-TRANSITION-01-12, the three projects aim to develop and demonstrate new technologies to digitalise further and automatise the European building sector, increasing its safety and attractivity for workers. Furthermore, the cluster seeks to stimulate the EU’s sovereignty in the industry, decreasing the need for technological imports.
The main objective of our Tech4EUconstruction cluster is to share knowledge between the projects on different aspects:
- Mutual exchange of technical expertise and project innovations
- Implement joint communication campaigns to raise cluster awareness
- Share knowledge and plans on exploitation actions
- Mutually promote the projects’ principal activities and achievements
- Co-organisation of events, workshops, panels, etc.
In less than one year, we organised two successful workshops at the European Robotics Forum 2023 and 2024. We also launched a new campaign called Words of Innovation, in which experts from our projects delved into essential and innovative aspects of their work by defining simple keywords. They briefly explained the technologies and strategies they are developing to address the challenges facing today’s European construction industry.
In addition, we invited other EU projects working in the same field to join our cluster: we can proudly say that now we have eight EU-funded projects onboard!
Looking ahead, the Tech4EUConstruction cluster will organise new social media campaigns (stay tuned: soon, we will launch an insightful campaign related to scientific publications), participate in joint workshops during central EU and international conferences and events (next step: SustainablePlaces 2024), and invite the recently funded EU projects working on our same topics.
And this is just the beginning!
Beyond dissemination and communication.
WP8 is not just the work package dedicated to dissemination and communication; it also covers the project's Exploitation, standardisation, and IPR activities. Thanks to the expertise of our Exploitation and IPR managers - Jason Rambach and George Kartsounis, and the great service offered by the Horizon Results Booster, we deeply analysed three Key Exploitable Results (KER1 Intention-controlled exoskeleton, KER2 Scan2BIM software and KER3 Spherical ToF camera), preparing the first Exploitation plan for the project. In addition, we also established a shared understanding and agreement regarding IPR among partners involved in developing the KERs.
Take a look at our progress on WP6, Human Factors – Training, Usability and Assessment, presented by Gloria Callinan, Project Support Officer at the Technological University of the Shannon, and stay tuned to our news and social media (LinkedIn and Twitter) to stay up to date!
Discover the HumanTech work: Human factors – Training, usability and assessment
Welcome to the second episode of the 'DISCOVER THE HUMANTECH WORK' blogs, a series where we dive into the HumanTech work packages (WPs) with our WP Leaders!
Let’s discover WP6, Human Factors – Training, Usability and Assessment, presented by Gloria Callinan, Project Support Officer at the Technological University of the Shannon, Development Unit Thurles.
This WP is composed of four tasks led by a team of incredible experts:
- T6.1 Micro-learning units development and coordination – led by TUS
- T6.2 Subjective and objective assessment of worker's technology acceptance – led by TECNALIA
- T6.3 Workflow capturing and extended reality (XR) training – led by DFKI and
- T6.4 Wearables safety, gender and ethics considerations – led by BAUA
Let’s dive into the work behind this WP, its impact, its liaisons with the other WPs and what is next.
The importance of WP6 – what has been done in this WP.
The construction sector is among the least digitalised and thus offers significant potential to improve the efficiency of construction processes and building operations and enhance health and safety on construction sites. The digital transformation in the construction sector will require workforce upskilling and reskilling.
Training and assessments will focus on the following thematic areas:
- Technologies supporting workers' safety and well-being in future digital construction
- Human-robot collaboration in construction automation
Work Package 6 is the meeting of technology and human factors in construction through the HumanTech project. In response to this challenge, WP6 has carried out two main activities supported by all WP 6 task leaders and WP project partners.
1. Creation of new training materials for use by training centres to be delivered to craft trades and apprentices and to higher education institutes to upskill construction professionals. Chambers and registered bodies can also access the freely available materials for continuous professional development. 3 modules have already been developed on HumanTech training material: HumanTech and Digitalisation (module 1), Green Technology in Construction (Module 2), and BIM Fundamentals Module 3, and are available on Zenodo.
Another 9 modules are under development with support from the HumanTech pilots covering 360-degree Cameras and mounting on Robotics and Drones, Robotics in Construction, UAV and UGV and construction, Digital twins, Exosketons, Building Smart Data Dictionaries and others.
2. Subjective and objective assessment of worker's technology acceptance. The use of advanced technology such as exoskeletons, smart glasses and wearable sensors can have a huge impact on the behaviour of the worker. Similarly, the use of robots on construction sites with human interaction is a major challenge. Although technologies are designed to support workers, they can have the opposite effect in the work environment, especially when different technologies are combined. Workers may feel monitored, restricted in their movements or stressed by an information overload. In the context of a good work environment, special consideration should be given to the needs of workers.
Led by partners Tecnalia and BAUA, workers and apprentices participated in a focus group moderated by a HumanTech partner in Spain and Ireland. Technologies were presented and outlined in a face-to-face environment, and surveys were then completed on workers’ perspectives.
One online workshop was held with French female construction workers moderated by the European Builders Confederation. Workshop participants were assured that data from these objective measures were anonymised and would be used only to evaluate the technological solutions, not any given worker’s performance.
Participants were recruited by Acciona at Alicante and Zubieta, in Spain, among the stakeholders working on the construction site (workers and supervisors). A total of 22 participants took part in this first physical workshop, organised in Acciona premises. Another 27 participants took part in the second physical workshop, organised in the Acciona offices at another construction site. The Irish workshop took place in Limerick, Ireland, and was delivered by TUS to apprentice electricians and carpenters at the Raheen Training Centre campus of the Limerick Clare Education and Training Board. It was attended by 26 construction apprentice participants and 4 tutors.
The broad results indicate higher reluctance towards interactive robots than exoskeletons and XR glasses, mainly due to perceived low manoeuvrability, physical rigidity and sedateness. The results are available in report format D6.3 – HT Worker Assessment Report.
In HumanTech, we are not working in a silo – let’s discover how this WP is linked to the others.
WP6 is highly dependent on progress in other work packages, the evolution of technology such as exoskeletons, smart glasses, and wearable sensors, and the progress of the pilots, the work of which will be used for the final assessment due in 2025. Subjective assessment of the human factor-related aspects was performed using scientifically validated questionnaires for lab and field research. Objective evaluation is due later in the project and is an objective perspective of user acceptance that will be assessed by means of measurement of users psychophysiological signals such as EEG (electroencephalogram), GSR (Galvanic Skin Response) and BVP (Blood Volume Pulse). For example, WP1 user requirements and architectural definitions are relevant and fundamental for human factors in both training and assessment. The BIMxD platform of WP2, hyperspectral material scanning of WP3, body sensor network of WP4, demolition task planning in WP5 and mobile platform, along with the pilot of WP7, are all relevant and significant to WP6. Finally, a communications foundation from WP8 is vital for the dissemination of the WP6 training materials.
The feedback from workers and apprentices will help inform the work of the technology partners, and the provision of training from the HumanTech project will mean greater exploitation and sustainability of the HumanTech approach.
The impact and the benefits of WP6.
Two aspects of our work are likely to have the most impact.
- The delivery of 12 bespoke micro-learning units, which are the culmination of HumanTech's work and unique to HumanTech results, will upskill a range of construction sector actors across VET, higher education, and Continuous Professional Development. The target is upskilling 200 trainees and 20 tutors.
- Worker assessment will have a significant impact on workers' attitudes toward technology. Open questions for each technology included, for instance, the participant’s belief on how their working task will change under the use of the technology, expected benefits and problems (each in the short- and long-term), as well as the most important resources needed for a successful implementation. Participants stated that they see positive effects on the reduction of physical strain, musculoskeletal injuries, and disorders. For XR glasses, participants named the specific benefits of worker training, learning, and skill development. Furthermore, they see benefits in the specific application of XR glasses for prototyping as well as in the design and planning phase, also through visualisation of future on-site activities.
WHAT IS NEXT
The two main (although there are many more) activities planned for WP6 are the completion of the remaining 9 micro-learning units as modules to be delivered by training centres and higher education institutes. 200 participants will benefit as pilots from the HumanTech training, and 20 educators will be upskilled in delivery. In addition, the final assessment using technology developed in HumanTech will be tested on workers again in Spain and apprentices in Ireland.
Planning for the next assessments includes:
- Organisation of additional workshops in other countries to go deeper into the subjective analysis (and thus complement what has been done in T6.2).
- Performance of an objective assessment by evaluating physiological sensor data collected in dedicated training sessions. To do so, the technological developments of HumanTech wearables and interactive and collaborative robot systems developed in WP4 and WP5 will have to be mature enough to be tested in pilots in WP7.
Take a look at our progress on WP4, wearable technologies for construction, led by Bruno Walter Mirbach, Senior Researcher at the Department of Augmented Vision at DFKI, and stay tuned to our news, social media (LinkedIn and Twitter), and to stay up to date!
HumanTech's 3rd Executive Board Meeting in Genova
We held our third Executive Board Meeting on 20 and 21 March, hosted by STAM in Genova. During two days, we reviewed our progress over the last period, conducted workshops and demonstrations of our pilots and technologies, and established our next steps to continue advancing towards our goals.
The meeting kicked off with a warm welcome and an introduction by our coordinator, Jason Rambach (DFKI), providing updates on the project's status. Work Package presentations followed, covering various aspects such as BIMxD Formats and Standardization, Dynamic Semantic Twin Generation, Wearable Technologies for Construction, and Construction Robotics and Human-Robot Collaboration.
We conducted pilot workshops, focusing on Dynamic Semantic Digital Twin and Bridge Inspection and Monitoring, among others. The first day concluded with discussions on Outreach, Exploitation, and Collaboration and capped off with a fascinating XR BIM visualization demo by Holo-Light.
The second day began with a presentation on Human Factors: training and Usability Assessment. Pilot sessions continued, featuring Human-Robot Collaboration and Wearables, Remote-Controlled Demolition, and Robotic Mastic Application. Jason Rambach concluded the day with closing remarks, wrapping up two days of productive discussions and workshops.
Overall, the meeting served as a valuable platform for collaboration, knowledge sharing, and strategic planning, driving us closer to our collective goals. We're grateful for the dedication and contributions of all participants, and we look forward to continuing our journey towards innovation and excellence in the construction industry.
Many thanks to our partners STAM, Francesca Canale and Stefano Ellero, for organising it!