andrea@australo.org

HumanTech Pilots: Dynamic sematic digital twin

From 10 to 13 March 2025, HumanTech team partners from research institutes, SMES, and construction companies gathered at the Department of Biomedicine of the University of Basel construction site to evaluate one of the use cases of our dynamic semantic digital twin system. This technology contributes to our broader mission to digitalise the construction sector, enhancing safety and improving efficiency.

Read the insights from the team involved in this pilot's development:

Applying BIMxD and scan-to-BIM in real-world scenarios

Fabian Kaufmann, Researcher at RPTU

The main objective of this pilot — one of our five HumanTech Pilots — is to demonstrate and validate HumanTech solutions related to BIMxD and scan-to-BIM in real-world applications. This includes five use cases:

• Construction progress monitoring
• Construction site item localisation
• Falling hazard identification using an interactive augmented user interface
• Detection of blocked emergency exits
• Scan-to-BIM to provide an as-is model of the project

Key milestones include the development of technical components such as a 360-degree depth camera and automated unmanned aerial vehicle (UAV) and unmanned ground vehicle (UGV) scanning pipelines for efficient data capture, segmentation models, scan-to-BIM reconstruction algorithms, and open BIM authoring tools. During the pilot implementation, each component was integrated to deliver the corresponding use case.

"The main objective of this pilot is to demonstrate and validate HumanTech solutions related to BIMxD and scan-to-BIM in real-world applications."

Turning point clouds into actionable insights

Sebastian Mattes, Reality Capturing Specialist at Implenia

In this pilot, we explored how point clouds can be used effectively on construction sites, with a focus on two key areas: safety and productivity.

We employed different technologies to capture the data: a laser scanner, a drone, and a 360-degree camera with depth-measurement. By using reference markers, we could align these datasets, compare their accuracy, and overlay them with a digital model.

Capturing the point clouds is actually the smallest part; the real challenge lies in processing the data and extracting meaningful information from millions of points.

"Capturing the point clouds is actually the smallest part; the real challenge lies in processing the data and extracting meaningful information from millions of points."

On the safety side, we aimed to detect and visualise falling hazards and ensure that emergency exits were unobstructed. From a productivity perspective, we developed a pipeline to generate a model from the point cloud, track the building progress, and identify materials or objects present on-site.

https://www.youtube.com/watch?v=1v897czNIYo

Hololight’s XR solution for the falling hazards detection use case

Harsh Manoj Shah, Technical Project Manager at Hololight

As part of the use case focused on detecting falling hazards, we at Hololight developed an XR-based BIM visualisation system called BIMSpace XR. It is a mixed reality application that enables real-time, true-to-scale visualisation of high-quality BIM models directly on construction sites. Using industry-standard markers, the system ensures precise alignment between the digital model and physical structures, giving construction professionals an immersive and interactive way to inspect safety-critical elements.

"The BIMSpace XR solution gives construction professionals an immersive and interactive way to inspect safety-critical elements directly on-site."

The BIMSpace XR application is integrated with Hololight Stream, a remote rendering solution that enables real-time streaming of augmented reality (AR) and virtual reality (VR) applications from powerful servers to the Microsoft HoloLens 2 XR device. The solution also supports real-time issue tracking through the BIM Collaboration Format (BCF), allowing users to detect, annotate, and resolve design deviations, safety hazards, or construction errors on-site. These issues are synchronised with BIM collaboration platforms like Catenda Hub, enabling seamless project coordination and communication across teams.

In addition to progress tracking by comparing as-planned and as-built conditions, the solution can extend to MEP reviews, virtual safety training, and multi-user collaborative sessions, offering interactive and scenario-based learning experiences on-site.

Detection of blocked emergency exits using BIM and point cloud analysis

Marius Schellen, Reasearcher at RPTU

This use case focuses on improving safety on active construction sites and operational buildings by detecting blocked emergency exits in real time. Leveraging a detailed BIM model — either of a building under construction or in use — all doors marked as emergency exits (with the fire exit property set to true) are identified. From these, the system derives the required emergency corridors that must remain clear at all times to comply with safety regulations.

Using HumanTech’s advanced technologies, an up-to-date point cloud of the site or specific area is analysed. Points falling within the defined emergency corridors are examined to detect potential obstructions. A rule-based system then determines whether any objects actually block the escape path, ensuring that a minimum width is maintained for safe evacuation.

"Using HumanTech’s advanced technologies, emergency exits can be monitored in real time — helping teams resolve safety hazards quickly and efficiently."

This functionality is fully integrated into the HumanTech BIMxD backend. Through a browser-based interface, users can upload or select an IFC model along with the corresponding point cloud to automatically detect obstructions with just a few clicks.

If an issue is found, a BIM Collaboration Format (BCF) report is automatically generated. This report can include attachments, instructions, or notes, and is assigned to the relevant project personnel, helping teams resolve safety hazards quickly and efficiently.

Item localisation with 360-degree 3D scanner

Hideaki Kanayama, R&D Engineer at RICOH

As part of this pilot, we conducted a field demonstration of our newly developed 360-degree one-shot 3D scanner at a construction site and applied it to the use case focused on construction site item localisation. High-speed scanning in the field is crucial to reduce the effort required to capture images while minimising disruption to site activities.

To compare the design BIM model and 3D scans, we developed a pipeline that aligns the scanned data with BIM using AR markers as reference points. This allows for visualising progress changes in comparison to the BIM model.

We also developed a data preprocessing pipeline to facilitate item localisation using Open-Vocabulary Segmentation, a technology developed by DFKI. Since conventional AI models often struggle with 360-degree imagery, we converted these images into a format more compatible with existing models — significantly improving their performance.

Enabling dynamic semantic digital twins through automated data capture

Michael Peter, Patipol Thanuphol, Researcher at ZHAW

As part of this pilot, ZHAW developed and integrated an automated data acquisition system designed to support the creation of Dynamic Semantic Digital Twins (DSDT) in construction. The system coordinates drones, robots, and handheld scanning workflows, enabling efficient, minimally disruptive data collection throughout the construction lifecycle.

"We developed an automated data acquisition system that coordinates drones, robots, and handheld scanners — enabling efficient, minimally disruptive data collection throughout the construction lifecycle."

Capture requests are initiated and managed through BIM-based inputs, and the collected data is automatically processed, aligned, and integrated using a unified backend. A key innovation is the standardised marker system, which ensures accurate spatial referencing across different capture sessions and devices. Surveyed markers are paired with detectable visual targets, and their detection and transformation are automated within the pipeline.

The system also provides recommended marker information to project partners, simplifying downstream processing and dataset merging. These developments enable high-quality, consistent, and scalable 3D data capture — supporting use cases such as progress tracking, safety monitoring, and automated as-built model generation.

Bringing the end-user perspective

Patricia Rosen, Researcher at BAuA

Like in other HumanTech pilots, our focus in this pilot was to include the end-user perspective in the technology development process to obtain valuable feedback on the current system status, but also to gain insights on potential risks and benefits for occupational safety and health (OSH) from the users’ perspective. We collected feedback on different usability-related aspects from nine potential end-users.

"Like in other HumanTech pilots, our focus in this pilot was to include the end-user perspective in the technology development process to obtain valuable feedback."

We mainly focused on the use case for falling hazard detection, given its clear link to OSH concerns. Even though technology maturity has not yet reached its full potential, most users see strong benefits related to different efficiency aspects. However, further adaptation will be necessary to ensure an overall safe operation.

This pilot has demonstrated how emerging technologies can be integrated into practical workflows to improve safety, efficiency, and collaboration on construction sites. From automated data capture and falling hazard detection to real-time BIM visualisation and item localisation, the combined efforts of the HumanTech partners show the transformative potential of digital solutions when developed with end-users in mind.

Learn more about how our human-centred innovations are contributing to a safer, more efficient, and technology-driven construction by following our news and social channels (LinkedIn, Twitter and YouTube) and subscribing to our newsletter!

by andrea@australo.org

HumanTech Pilots: Bridge inspection and monitoring

Between June 2023 and February 2025, HumanTech partners from different organisations gathered at a highway bridge in Kleinandelfingen, Switzerland, to test our semi-autonomous system for bridge inspection and monitoring. This pilot is part of our broader ambition to digitalise the construction sector — enhancing safety, efficiency, and sustainability through innovative technologies.

Our goal was to identify an affordable drone system with maximum technological support to simplify bridge inspections — crucial for maintaining ageing infrastructure and addressing the growing shortage of skilled inspectors and experienced drone pilots. With this in mind, we aimed to create a best practice demonstrating how future inspections and analyses could be conducted.

This pilot — one of our five HumanTech Pilots — represents an important step toward digitalising inspection processes through robotics, paving the way for more efficient and AI-supported methods in construction monitoring.

Read the insights from the team behind its development:

https://www.youtube.com/watch?v=LpP4lN7UdiM&t=59s

How HumanTech is digitalising bridge inspections

Sebastian Mattes, Reality Capturing Specialist at Implenia

As part of the HumanTech project at Implenia, we are focusing on bridge inspections. Our ageing infrastructure — particularly bridges — needs more and more attention. At the same time, we face a shortage of skilled labour, as many experienced inspectors are reaching retirement age.

To reduce the workload and improve efficiency, we have developed a best practice that combines drones and AI in a digital system, enabling semi-autonomous inspection and evaluation.

The system allows for the overlay of previous inspection data, making it possible to accurately track changes in cracks over time. This process is designed to support and enhance the work of human inspectors.

"To reduce the workload and improve efficiency, we have developed a best practice that combines drones and AI in a digital system, enabling semi-autonomous inspection and evaluation."

Custom drone technology for precision infrastructure monitoring

Michael Peter, Patipol Thanuphol, Research Associate at ZHAW

The Control Systems Group at the Institute of Mechatronic Systems (ZHAW) has developed a companion system for the DJI Mavic 3E drone, designed to overcome the challenge of capturing images underneath the bridges where GNSS signals are unavailable.

Traditionally, photogrammetry in these environments has required manual piloting, but ZHAW’s system leverages drone-based visual SLAM to enable stable positioning, making operations more reliable and significantly reducing the burden on the pilot.

The custom hardware setup includes a high-resolution camera with flash, a microcontroller with Bluetooth and Wi-Fi, laser pointers for field-of-view indication, and a receiver for remote triggering. This solution extends the capabilities of state-of-the-art commercial drone technology, allowing for capturing perpendicular images, regardless of camera orientation, even under challenging conditions.

"This solution extends state-of-the-art commercial drone technology to capture perpendicular images, regardless of camera orientation, under difficult conditions."

As a proof of concept, the system demonstrates that high-quality results can be achieved without advanced piloting skills, highlighting ZHAW’s expertise in merging advanced control systems with accessible commercial platforms. Beyond solving current challenges, this development also opens the door to future autonomous solutions for infrastructure inspection.

AI-powered 3D modelling and defect detection for structural inspections

Jo Luick, Project Lead at STRUCINSPECT

STRUCINSPECT specialises in advanced structural inspection solutions, leveraging cutting-edge technology to help ensure the safety and longevity of critical infrastructure.

In HumanTech, we used photogrammetric processes to create a precise 3D replica of a bridge by analysing images captured by two different data acquisition systems. The process included georeferencing and scaling to ensure accuracy, resulting in two 3D models that allowed for a detailed comparison.

We then applied automated AI-driven defect detection to identify structural issues such as cracks, spallings, exposed reinforcements, corrosion, and efflorescence. These detected defects were mapped onto the 3D model and exported in the Industry Foundation Classes (IFC) format, creating a comprehensive digital representation of the bridge’s condition.

This method not only improves the accuracy of structural assessments but also streamlines the inspection workflow, making defect documentation more efficient. Integrating AI and photogrammetry marks a significant advancement in structural inspection practices.

"This method not only improves the accuracy of structural assessments but also streamlines the inspection workflow, making defect documentation more efficient."

Turning bridge data into digital models

Marius Schellen, Researcher at RPTU

In addition to overseeing the progress of this pilot, RPTU was responsible for coordinating its development and implementation. On the technical side, RPTU provided an approach for BIM reconstruction based on the scanned data. To support this, results from previous research were adapted and applied to explore methods for semantic segmentation and scan-to-BIM reconstruction. A demonstration of the BIM reconstruction will be included as part of the pilot. However, further development is needed to fully automate the process.

This pilot demonstrates how innovation across disciplines can transform infrastructure inspection. By addressing real-world challenges, HumanTech is paving the way for smarter, safer and more sustainable construction practices. The results from this pilot not only showcase the potential of semi-autonomous systems today, but also lay the groundwork for future fully automated solutions.

Learn more about how our human-centred innovations are contributing to a safer, more efficient, and technology-driven construction by following our news and social channels (LinkedIn, Twitter and YouTube) and subscribing to our newsletter!

by andrea@australo.org

HumanTech Pilots: Robotics and wearables for collaborative bricklaying

Between 13 and 31 January 2025, HumanTech project partners gathered at ACCIONA Construction's demo site in Madrid to integrate and test our robotics and wearables system for collaborative bricklaying. This is one of the technologies we have developed as part of our broader mission to digitalise the construction sector and make it safer and more efficient.

The goal for this session was to demonstrate two innovative technologies for assisting bricklayers at construction sites:

• An active exoskeleton that supports workers during heavy lifting, particularly at or above shoulder height.
• A robotic brick-handover system that can detect and grasp bricks from a stack and hand them to the worker.

This pilot — one of our five HumanTech Pilots — represents a key milestone in our efforts to enhance safety and efficiency in construction processes through human-robot collaboration, setting the stage for a smarter and more ergonomic approach to bricklaying.

Explore insights from the team behind its development:

The pilot's main objective

Fabian Kaufmann, Researcher at RPTU

The human workforce will be present on construction sites at least in the mid-term, and likely even in the long-term. Therefore, robotic systems must be able to co-exist with humans on-site. This pilot aims to demonstrate human–robot collaboration enabled by advanced perception and control algorithms. In an integrated bricklaying scenario, our robot hands over bricks, picked up from a pallet, to the worker, and the worker is supported by an on-demand activated exoskeleton to assist with physically demanding tasks.

"This pilot aims to demonstrate human–robot collaboration enabled by advanced perception and control algorithms."

https://www.youtube.com/watch?v=03h0T0rUNOs

Transforming Bricklaying: How Technology is Empowering Workers

Anurag Bansal, Head of Strategic Innovations at ACCIONA

This pilot could be a game-changer for the industry. As a construction company committed to innovation, we were excited to see HumanTech’s robotics and wearables system for collaborative bricklaying in action at Acciona. These technologies have the potential to transform the way we work — making construction safer, more efficient, and sustainable.

The active exoskeleton provided much-needed support for workers lifting heavy materials at or above shoulder height, reducing strain and fatigue. Meanwhile, the robotic brick handover system seamlessly detected, grasped, and passed bricks to workers, improving workflow and minimising downtime.

Seeing these solutions firsthand reinforced the idea that automation isn’t about replacing workers — it’s about empowering them. If widely adopted, this technology could revolutionise construction sites, enhancing productivity while prioritising worker well-being. This is a major step towards a smarter, safer future for the industry.

"If widely adopted, this technology could revolutionise construction sites, enhancing productivity while prioritising worker well-being."

Gabor Sziebig, Research Manager at SINTEF

Statistics and analyses show that the construction industry faces many challenges: slow adoption of new technologies, high injury rates, low productivity, and a decline in recruiting skilled workers. I believe construction sites will change significantly in the coming years.

Looking five years ahead, I think the most modern construction sites will be adapted for robots. Initially, it is natural to imagine them operating on site when human workers are not present. If humans work two shifts, robots can take the third. It will begin with simple tasks — robots that paint or transport materials already exist, and Norwegian companies are already using drilling robots. Robots inspecting the construction site during off-hours is also easy to envision.

Over time, robots will increasingly work alongside construction workers and engineers on-site, as we have demonstrated in this pilot.

"Over time, robots will increasingly work alongside construction workers and engineers on-site, as we have demonstrated in this pilot."

Use case 1: Robotic collaboration for brick handover

Brick pose estimation and best candidate selection

Sai Srinivas Jeevanandam, Computer Vision Researcher at DFKI

For the robot to effectively pick up bricks, it must understand two critical aspects: (1) the poses of the bricks — including their precise location and orientation in the environment — and (2) which brick to select for grasping. We at DFKI have developed ZebraPose, a state-of-the-art, in-house deep learning algorithm designed to predict poses directly from a single image.

Recognising the novelty of this task and the limited availability of relevant real-world data, we created a synthetic dataset consisting of rendered images of brick stacks. These synthetic images were specifically designed to replicate real-world brick arrangements. We used this dataset to train our deep learning model, enabling it to predict brick poses accurately. By leveraging synthetic data, we eliminated the need for time-consuming and error-prone manual annotation, significantly accelerating the model development process.

In addition to pose estimation, we developed a robust best-candidate selection algorithm. Given the poses of all bricks in a stack, this algorithm evaluates each one and selects the most suitable candidate for the robot to grasp.

Finally, we packaged everything into an end-to-end pipeline with an API interface for easy integration. This pipeline is designed to operate in real time, making it suitable for deployment on edge devices such as the NVIDIA Jetson, where computational resources are limited.

Mobile robotic system

Joseph Thomas Thekkekara, Technical Project Manager at BAUBOT

BAUBOT provided a collaborative mobile demonstrator platform based on a modular concept, built from scratch for HumanTech. This system can support a wide range of external robotic applications, particularly those related to the construction industry.

It was designed with external software and hardware interfaces to facilitate the integration of external robotic applications into the existing robotic architecture.

https://www.youtube.com/watch?v=jdzQBCzLsxo

Use case 2: Wearables for supported bricklaying

Intelligent exoskeleton with intention prediction

Alfonso Domínguez, Senior Researcher at Tecnalia

Within the HumanTech project, Tecnalia has introduced the first prototype of an advanced upper-limb exoskeleton. The decision to focus on the upper limbs stems from the fact that shoulders, along with the back, are among the most affected body regions in construction due to the physically demanding nature of the work.

What makes this exoskeleton unique is its ability to activate based on the user’s intent, inferred from their kinematics using data from a Body Sensor Network (BSN). Its main goal is to reduce physical strain and minimise effort for the worker, while providing a more intuitive and less intrusive user experience. By simplifying its operation, the system also reduces the mental effort needed to control the exoskeleton, making it a seamless extension of the worker’s natural movements.

"What makes this exoskeleton unique is its ability to activate based on the user’s intent, inferred from their kinematics using data from a Body Sensor Network (BSN)."

Mobile magnetometer-free inertial tracking

Markus Miezal, CEO and Co-Founder of Sci-track

As support for the exoskeleton, Sci-track provides its magnetometer-free inertial tracking approach, executed on-body using a Xenoma e-skin suit. The suit features very flat IMUs integrated into a tight full-body suit that can be comfortably worn beneath protective clothing.

Usually, inertial sensors include a magnetometer — a 3D compass used to measure the local magnetic field and determine yaw direction. While this sensor is designed to point towards magnetic north, it can easily be disrupted. Metal structures, power tools, and motors — all common on construction sites — influence the Earth’s magnetic field locally, resulting in inaccurate readings. Our approach eliminates the use of magnetometers, relying instead on motion-based intrinsic alignment.

Additionally, our algorithms can run on a body-worn, mobile phone-sized computer. This ensures timely communication with the exoskeleton and enhances privacy, as the data never leaves the wearer.

https://www.youtube.com/watch?v=0pdJqjwoEa4&feature=youtu.be

This pilot showcases how human-robot collaboration can transform physically demanding construction tasks into safer, more efficient, and more sustainable processes. By combining robotics, wearables, and intelligent algorithms, HumanTech is paving the way for a more resilient and future-ready construction industry.

Learn more about how our human-centred innovations are contributing to a safer, more efficient, and technology-driven construction by following our news and social channels (LinkedIn, Twitter and YouTube) and subscribing to our newsletter!

by andrea@australo.org

HumanTech Pilots: BIMxD-based autonomous robotic task planning

In December 2024, HumanTech partners gathered at RPTU Kaiserslautern’s demo centre for a hands-on demonstration and final implementation of one of the project's innovative pilot projects: BIMxD-based autonomous robotic task planning. This session marked a significant milestone in our collaborative journey to revolutionise the construction industry with advanced robotics and digital tools.

This pilot aims to showcase how cutting-edge technologies like our BIMxD platform and robotic systems can transform traditional construction processes by introducing automation and precision, enabling more efficient and streamlined operations.

Professionals across the HumanTech team share their insights in this article, detailing this groundbreaking pilot's technologies and outcomes.

Introduction to the pilot and demo session

Jason Rambach, HumanTech’s Project Coordinator and Senior Researcher and Team Leader at DFKI

This pilot aims to demonstrate the potential of fully digitalised construction sites in enabling robotic automation. BIMxD, the digital twin platform we developed at HumanTech, is used as an interface for remotely planning robotic tasks, in this case by our partner Kajima in Singapore. The planned task information is then transmitted and converted into robotic navigation and task execution by our BAUBOT robotic platform, along with other technologies developed at HumanTech.

This pilot is a key example of successful partner collaboration in HumanTech, integrating components created by different partners. These include RPTU (scan-to-BIM and autonomous navigation), CATENDA (BIMxD platform), STAM (ontology for robotic demolition task), SINTEF (robotic end-effector), BAUBOT (robotic platform), RICOH (surround-view streaming with spherical camera), and KAJIMA (end-user).

"This pilot aims to demonstrate the potential of fully digitalised construction sites in enabling robotic automation."

https://www.youtube.com/watch?v=TXtGg97KQdg&list=PLLFSbOsrWIuaVm5-kJQuKvh6bgsbJHcrH&index=7

Overview of the pilot objectives and test site

Marius Schellen, Researcher at RPTU

The objective of this pilot is twofold: On the one hand, the interface of the BIMxD platform and robotic control is being demonstrated. On the other hand, the autonomous robotic navigation and marking, based solely on the BIM model without any markers, is being implemented and demonstrated.

To this end, a dedicated test building was erected to suit the scenario and the HumanTech robotic platform. The test building spans an area of 90m2, with doors and windows offering distinct features for the robot’s localization.

Beyond the physical implementation, the BIMxD interface was set up by Catenda, and the Robotics Research Lab of RPTU implemented the interface and robot controller for the scenario. SINTEF contributed to the pilot by providing the marking end-effector.

Livestream of the session

Hideaki Kanayama, Researcher at RICOH

RICOH offers a service platform, Ricoh Remote Field, designed to support on-site work and remote monitoring through 360° video streaming. This platform enables high-quality, low-latency video streaming with a full 360° field of view. The setup is straightforward, requiring only a single THETA camera and a wireless network connection. The system can be accessed from anywhere via a PC, smartphone, or tablet.

In this pilot, the THETA cameras were mounted on our robotic platform and a wall at the demo site. The video streaming system allowed a remote operator in Singapore to visually monitor the site in Germany (explained below), verify the robot's path planning, and check for any anomalies in the environment.

BIMxD interface

Torvald Andresen, Catenda

We have developed BIMxD, an in-browser interface for inputting data to the task planner. It allows for uploading a 3D model of the worksite, which it shows in an integrated model viewer. A user may draw up expected key points in the model, and relevant data like BIM coordinates, GUIDs (globally unique identifiers), and selected sides of relevant objects are further communicated to the task planner. Authorised users can interact with the model and highlight key points remotely. These points are used to generate the robot movement plan.

The interface is multipurpose and used in several places in HumanTech. This specific model viewer allows other relevant features, like seeing several models of the site and highlighting issues to be addressed in the same view.

Task planner and demolition ontology

Francesca Canale, Project Engineer at STAM

We have developed a task planner to enable the system to plan the robotic actions needed to execute demolition activities autonomously. Powered by a structured demolition ontology, the system understands both the environment and the impact of each demolition action. This ontology extends the ifcOWL framework, adding new classes like "Robot," "Tool," and "Opening," allowing the task planner to match robots with the right tools for each job. Acting as a bridge between user input, the BIM model, and the robots, the task planner ensures efficient and precise execution of demolition activities.

Path planning and autonomous navigation

Maximilian Kunz and Jörg Husemann, Researchers at RPTU

We utilise information from a BIM model for localisation and autonomous navigation of the robot. To localise within the building, we use a LiDAR scanner and compare the detected walls with the walls in the BIM model. By doing so, we can detect our current room based on the structure we scanned and our position within the room without the need to place markers or additional sensors in the environment. A map is created from the navigation model. Then, a path to the target position is planned based on this map. Finally, the platform drives the calculated path while checking its location based on its odometry and the scans of the LiDAR scanner.

https://www.youtube.com/watch?v=ucHGMGu7rtU&list=PLLFSbOsrWIuaVm5-kJQuKvh6bgsbJHcrH&index=6

Robotic platform

Joseph Thomas Thekkekara, Technical Project Manager at BAUBOT

Baubot provided a collaborative mobile demonstrator platform based on a modular concept built from scratch for HumanTech. The mobile robotic system can serve as a platform for a wide range of external robotic applications, particularly those relating to the construction industry.

The system was built with external software and hardware interfaces that facilitate the integration of external robot applications into the existing robotic architecture. The marking tool developed by SINTEF, the autonomous navigation system provided by RPTU, and the Task Planner developed by STAM were integrated into the pipeline to enable the Baubot robotic system to perform the indented task.

Marking tool

Gabor Sziebig, Research Manager at SINTEF

The marking tool is a small and compact way to highlight the task of the collaborative mobile demonstrator platform. The housing is made by rapid prototyping, and the electronics are adapted to the requirements of the platform: power source and communication with the overall control system of the platform. The marking tool holds a classic whiteboard marker pen and can be easily replaced. A sensor is also integrated into the housing, which detects contact with any surface by the marker pen.

Remote task planning from Singapore and end-user perspective

Rongbo HU, Researcher at Kajima Corporation

In this demonstration, engineers from the Kajima Technical Research Institute Singapore (KaTRIS) set up a station consisting of two terminals in a Construction Robotics Lab, each with a screen to command the robot remotely from Singapore.

On the left screen, videos were streamed live from the Ricoh camera installed inside the testing room in Kaiserslautern. This allowed Kajima engineers in Singapore to observe the surroundings and make informed decisions. On the right screen, there was a digital twin of the testing cell embedded in the web application for the engineers to send commands to the robot. Using the digital twin, Kajima engineers first studied the environment from the camera's live feed, followed by clicking the corresponding wall inside the digital twin to perform autonomous robotic drilling.

As the end user of this technology, Kajima Corporation is able to study the implications and benefits of remote control of autonomous robots. By incorporating remote operation into future workflow, manpower can be shared between Kajima’s offices worldwide. Engineers no longer have to frequently travel to sites to perform specific tasks and workers can dynamically operate from a safer environment. More importantly, as construction robotics solutions incorporate such features, human intelligence can come in remotely to resolve tasks that are too complicated for robots, thus filling the gap. This can potentially be a game changer in how we build.

This BIMxD-based autonomous robotic task planning pilot exemplifies HumanTech’s vision of integrating digitalisation, automation, and collaboration to advance the construction industry. By leveraging the expertise and innovations of multiple partners, this pilot demonstrated the feasibility and potential of fully digitalised construction workflows, from remote task planning to autonomous robotic execution.

As we move forward, the insights gained from this pilot will be instrumental in refining our technologies for broader adoption across the industry. Upcoming demonstration sessions in Madrid and Switzerland will further validate our solutions' scalability and real-world application, bringing us closer to shaping a smarter, more efficient, and sustainable construction future.

Learn more about how our human-centred innovations are contributing to a safer, more efficient, and technology-driven construction by following our news and social channels (LinkedIn, Twitter and YouTube) and subscribing to our newsletter!

by andrea@australo.org