Getting machine learning and artificial intelligence to work for construction is often a matter of finding where it would be most useful. AI algorithms can scan through terabytes of data, looking for small inconsistencies or hidden patterns that a human might not notice at first. Now, engineers at Thornton Tomasetti have applied that considerable processing power to the tricky work of building-facade inspections.
After two years of development in Thornton Tomasetti’s CORE lab, the result is T2D2, a computer-vision, machine-learning algorithm that can identify damage to building exteriors in photos or video. Not intended to replace the difficult work of facade inspections, T2D2 is instead seen as tool to find hidden damage that might go unnoticed, and speed along a tedious, difficult process.
“It will be somewhat guided—engineers will review the detections—but in general we want T2D2 to be flexible enough to look for conditions across different material types and structure types,” explains Badri Hiriyur, vice president and director of artificial intelligence at Thornton Tomasetti. “It could be used for any type of structure, but our main goal is to look at aging structures [that require] periodic inspections.”
After showing promise in an internal competition, Hiriyur and his team set out to train the algorithm on Thornton Tomasetti’s library of hundreds of thousands of images collected over decades of building inspections. Now able to work directly off of video and photos captured by drones, T2D2 can perform a “first pass” facade inspection before a human engineer has to get up close to the building.
On a typical image, T2D2 goes through several steps. Using its knowledge base of thousands of previously annotated images, it quickly identifies the geometry of the structure and assigns material types. T2D2 then does a pass to identify potential damage that would be found on each type of material. It then outputs a marked-up version of the image, with potential facade damage highlighted and classified for review.
While T2D2 was initially trained primarily on concrete structures, Hiriyur says its capabilities has been expanded. It can now find and classify damage on brick, masonry, stucco and other commonly used facade materials.
Thanks to the machine-learning elements of T2D2, the algorithm gradually improves the more it is used. “As we get more projects, that will generate more data. We’re going to use reinforcement learning to improve the models,” says Hiriyur. “An engineer can review the project as we do it and mark the false positives and negatives, helping us retrain our model.”
“Our engineers have found T2D2 to be valuable. This thing doesn’t get tired,” says Raymond Daddazio, Thornton Tomasetti senior consultant and the firm’s former president. “We’re not replacing the engineer, but just as radiologists can use AI to help find tumors, or as NASA astronomers use AI to find new craters on Mars, [T2D2] makes it easier to do, more efficiently and effectively than before.”
But a clever algorithm is only part of the solution. Hiriyur has overseen the spin-off of T2D2 into its own data analytics firm, and working in partnership with drone survey companies, it can offer a complete facade inspection service. T2D2 has already taken on several customers, and is in discussions with several large owners in New York City to inspect their buildings, perhaps even more often than the city’s mandated five-year interval.
“We’re trying to rethink the way inspections are done in the first place,” says Hiriyur. While many building owners are only interested in performing facade inspections to meet regulatory requirements, the comparatively lower expense of running T2D2 with drones could change that. “This isn’t just to meet city regulations. More frequent inspections mean building owners can do small fixes to prevent big problems. We’re talking about saving millions in repairs,” he says.
