In December 2024, HumanTech partners gathered at RPTU Kaiserslautern’s demo centre for a hands-on demonstration and final implementation of one of the project’s innovative pilot projects: BIMxD-based autonomous robotic task planning. This session marked a significant milestone in our collaborative journey to revolutionise the construction industry with advanced robotics and digital tools.
This pilot aims to showcase how cutting-edge technologies like our BIMxD platform and robotic systems can transform traditional construction processes by introducing automation and precision, enabling more efficient and streamlined operations.
Professionals across the HumanTech team share their insights in this article, detailing this groundbreaking pilot’s technologies and outcomes.
Introduction to the pilot and demo session
Jason Rambach, HumanTech’s Project Coordinator and Senior Researcher and Team Leader at DFKI
This pilot aims to demonstrate the potential of fully digitalised construction sites in enabling robotic automation. BIMxD, the digital twin platform we developed at HumanTech, is used as an interface for remotely planning robotic tasks, in this case by our partner Kajima in Singapore. The planned task information is then transmitted and converted into robotic navigation and task execution by our BAUBOT robotic platform, along with other technologies developed at HumanTech.
This pilot is a key example of successful partner collaboration in HumanTech, integrating components created by different partners. These include RPTU (scan-to-BIM and autonomous navigation), CATENDA (BIMxD platform), STAM (ontology for robotic demolition task), SINTEF (robotic end-effector), BAUBOT (robotic platform), RICOH (surround-view streaming with spherical camera), and KAJIMA (end-user).
“This pilot aims to demonstrate the potential of fully digitalised construction sites in enabling robotic automation.”
Overview of the pilot objectives and test site
Marius Schellen, Researcher at RPTU
The objective of this pilot is twofold: On the one hand, the interface of the BIMxD platform and robotic control is being demonstrated. On the other hand, the autonomous robotic navigation and marking, based solely on the BIM model without any markers, is being implemented and demonstrated.
To this end, a dedicated test building was erected to suit the scenario and the HumanTech robotic platform. The test building spans an area of 90m2, with doors and windows offering distinct features for the robot’s localization.
Beyond the physical implementation, the BIMxD interface was set up by Catenda, and the Robotics Research Lab of RPTU implemented the interface and robot controller for the scenario. SINTEF contributed to the pilot by providing the marking end-effector.
Livestream of the session
Hideaki Kanayama, Researcher at RICOH
RICOH offers a service platform, Ricoh Remote Field, designed to support on-site work and remote monitoring through 360° video streaming. This platform enables high-quality, low-latency video streaming with a full 360° field of view. The setup is straightforward, requiring only a single THETA camera and a wireless network connection. The system can be accessed from anywhere via a PC, smartphone, or tablet.
In this pilot, the THETA cameras were mounted on our robotic platform and a wall at the demo site. The video streaming system allowed a remote operator in Singapore to visually monitor the site in Germany (explained below), verify the robot’s path planning, and check for any anomalies in the environment.
BIMxD interface
Torvald Andresen, Catenda
We have developed BIMxD, an in-browser interface for inputting data to the task planner. It allows for uploading a 3D model of the worksite, which it shows in an integrated model viewer. A user may draw up expected key points in the model, and relevant data like BIM coordinates, GUIDs (globally unique identifiers), and selected sides of relevant objects are further communicated to the task planner. Authorised users can interact with the model and highlight key points remotely. These points are used to generate the robot movement plan.
The interface is multipurpose and used in several places in HumanTech. This specific model viewer allows other relevant features, like seeing several models of the site and highlighting issues to be addressed in the same view.
Task planner and demolition ontology
Francesca Canale, Project Engineer at STAM
We have developed a task planner to enable the system to plan the robotic actions needed to execute demolition activities autonomously. Powered by a structured demolition ontology, the system understands both the environment and the impact of each demolition action. This ontology extends the ifcOWL framework, adding new classes like “Robot,” “Tool,” and “Opening,” allowing the task planner to match robots with the right tools for each job. Acting as a bridge between user input, the BIM model, and the robots, the task planner ensures efficient and precise execution of demolition activities.
Path planning and autonomous navigation
Maximilian Kunz and Jörg Husemann, Researchers at RPTU
We utilise information from a BIM model for localisation and autonomous navigation of the robot. To localise within the building, we use a LiDAR scanner and compare the detected walls with the walls in the BIM model. By doing so, we can detect our current room based on the structure we scanned and our position within the room without the need to place markers or additional sensors in the environment. A map is created from the navigation model. Then, a path to the target position is planned based on this map. Finally, the platform drives the calculated path while checking its location based on its odometry and the scans of the LiDAR scanner.
Robotic platform
Joseph Thomas Thekkekara, Technical Project Manager at BAUBOT
Baubot provided a collaborative mobile demonstrator platform based on a modular concept built from scratch for HumanTech. The mobile robotic system can serve as a platform for a wide range of external robotic applications, particularly those relating to the construction industry.
The system was built with external software and hardware interfaces that facilitate the integration of external robot applications into the existing robotic architecture. The marking tool developed by SINTEF, the autonomous navigation system provided by RPTU, and the Task Planner developed by STAM were integrated into the pipeline to enable the Baubot robotic system to perform the indented task.
Marking tool
Gabor Sziebig, Research Manager at SINTEF
The marking tool is a small and compact way to highlight the task of the collaborative mobile demonstrator platform. The housing is made by rapid prototyping, and the electronics are adapted to the requirements of the platform: power source and communication with the overall control system of the platform. The marking tool holds a classic whiteboard marker pen and can be easily replaced. A sensor is also integrated into the housing, which detects contact with any surface by the marker pen.
Remote task planning from Singapore and end-user perspective
Rongbo HU, Researcher at Kajima Corporation
In this demonstration, engineers from the Kajima Technical Research Institute Singapore (KaTRIS) set up a station consisting of two terminals in a Construction Robotics Lab, each with a screen to command the robot remotely from Singapore.
On the left screen, videos were streamed live from the Ricoh camera installed inside the testing room in Kaiserslautern. This allowed Kajima engineers in Singapore to observe the surroundings and make informed decisions. On the right screen, there was a digital twin of the testing cell embedded in the web application for the engineers to send commands to the robot. Using the digital twin, Kajima engineers first studied the environment from the camera’s live feed, followed by clicking the corresponding wall inside the digital twin to perform autonomous robotic drilling.
As the end user of this technology, Kajima Corporation is able to study the implications and benefits of remote control of autonomous robots. By incorporating remote operation into future workflow, manpower can be shared between Kajima’s offices worldwide. Engineers no longer have to frequently travel to sites to perform specific tasks and workers can dynamically operate from a safer environment. More importantly, as construction robotics solutions incorporate such features, human intelligence can come in remotely to resolve tasks that are too complicated for robots, thus filling the gap. This can potentially be a game changer in how we build.
This BIMxD-based autonomous robotic task planning pilot exemplifies HumanTech’s vision of integrating digitalisation, automation, and collaboration to advance the construction industry. By leveraging the expertise and innovations of multiple partners, this pilot demonstrated the feasibility and potential of fully digitalised construction workflows, from remote task planning to autonomous robotic execution.
As we move forward, the insights gained from this pilot will be instrumental in refining our technologies for broader adoption across the industry. Upcoming demonstration sessions in Madrid and Switzerland will further validate our solutions’ scalability and real-world application, bringing us closer to shaping a smarter, more efficient, and sustainable construction future.
Learn more about how our human-centred innovations are contributing to a safer, more efficient, and technology-driven construction by following our news and social channels (LinkedIn, Twitter and YouTube) and subscribing to our newsletter!