Back

Roof Covering: Benchmarking Engineering AI Against HazardHub

Francisco Galvis

CTO & Co-Founder

A building's roof covering is the outer layer of the roof, such as shingles, metal panels, tiles or a waterproof membrane. When wind blows across a roof, it creates suction that pulls up on the covering, a force engineers call uplift. Each type of covering can withstand a different amount of uplift before it detaches. FEMA's review of the 2004 hurricane season named damage to roof coverings as the leading cause of building performance problems in hurricanes. When a building's roof covering is unknown, a catastrophe model cannot assign the roof a covering class. RMS, for example, then applies an average damage curve, so the loss estimate reflects an average roof rather than the building's own.

In our recent benchmarking study, HazardHub, a legacy data provider, returned a roof covering for fewer than 10% of the 7,009 commercial buildings we sent it. ResiQuant's Engineering AI returned a covering for more than 98% of the buildings we tested from the same set (Figure 1).

Figure 1. Roof covering coverage on real commercial buildings. HazardHub returned a roof covering for 10% of buildings. Engineering AI returned a covering for more than 98%.

Accuracy and precision

Returning a roof covering is only the first step. The covering also has to be correct, and specific enough for the model to use. RMS and AIR use multiple roof covering options because each option behaves differently under uplift. RMS, for example, has separate classes for shingles rated for high winds and for ordinary shingles.

Our engineers reviewed 158 of the buildings where HazardHub listed a roof covering. They checked each roof by hand against aerial and street-level imagery and property records, and they identified the roof covering on 157 of them. To compare the two sources, we sorted both sets of answers into six basic materials: shingle, membrane, metal, tile, wood shake and concrete.

On those 157 buildings, Engineering AI was correct on 92% and HazardHub on 83%. Most of Engineering AI's lead is on metal and membrane roofs, which are common on commercial buildings (Figure 2). Engineering AI was correct on 94% of those roofs and HazardHub on 78%. On shingle roofs, both were correct on 95%.

These accuracy numbers favor HazardHub, because they only count the 10% of buildings where HazardHub gave an answer.

Figure 2. On buildings with metal or membrane roofs, ResiQuant was 94% accurate and HazardHub 78%. On buildings with shingle roofs, both were 95% accurate. Only buildings where HazardHub listed a roof covering.

Engineering AI returns the RMS or AIR class directly, so it also gives the detail inside each material. For a metal roof, it reports whether the fasteners are exposed or concealed. For a shingle roof, it reports whether the shingles are rated for high winds. HazardHub's roof covering field gives the material only.

What it means for underwriters

The more precisely an underwriting team knows a building, the more accurately it can price the building's risk. HazardHub left the roof covering blank on more than 90% of the buildings we sent it. Those buildings enter the model with an average roof. Taking accuracy into account, HazardHub would give a correct roof covering for only about 8% of the 7,009 buildings. Engineering AI returned a roof covering for 98% of the buildings we tested. Where both sources answered, Engineering AI was also more accurate, and it returned the actual class detail that RMS and AIR use.

Accurate pricing matters most in a soft market, when rates fall and carriers compete for the same accounts. A team that prices each building on its actual roof can offer better terms on the buildings that carry less risk. The same team can hold its price on the buildings that carry more risk. A team that prices every building on an average roof cannot tell those buildings apart.

Turn catastrophe risk into resilience

Transform underwriting with seamless automation, unmatched precision, and AI-powered insights tailored for property carriers.

.

Turn catastrophe risk into resilience

Transform underwriting with seamless automation, unmatched precision, and AI-powered insights tailored for property carriers.

Turn catastrophe risk into resilience

Transform underwriting with seamless automation, unmatched precision, and AI-powered insights tailored for property carriers.

Turn catastrophe risk into resilience

Transform underwriting with seamless automation, unmatched precision, and AI-powered insights tailored for property carriers.