Issue 740 - Ifc Importer Upgrades

[sebjf]

This fixes #740

This PR is currently based against #747 to check travis, which is why it is tagged invalid. Once #747 is merged, it should be based against staging.

Description

This PR upgrades the IFC Importer, in order to gain the following improvements:

• Up to IFC 4x3 Addendum 2 Support
• Streaming import via RepoSceneBuilder
• Tree improvements to better match industry conventions
• Support for LOD settings via ModelImportConfig
• Support for multithreaded import (configured via ModelImportConfig)

Product Team Issue: https://github.com/3drepo/3D-Repo-Product-Team/issues/680

Known Bugs/Errata

1. PropertySetDefinitionSets are not supported. This is an IFCOS bug. Reported here: PropertySetDefinitionSets are not imported correctly IfcOpenShell/IfcOpenShell#6330.
2. IfcRelDefinesByObject Representations are not supported. This is an IFCOS bug. Not reported yet because none of the reference tools support it so we don't have a good example.

Major Changes

Structure

The Ifc importer dependencies have been rearranged.

Before, bouncer would link to IFCOS and its dependencies directly & invoke a serialiser for each schema:

bouncer -> ifcImporter2x3, ifcImporter4

Now, a new module, ifcUtils, acts as an intermediary:

bouncer -> ifcUtils -> ifcSerialiser2x3, ifcSerialiser4

ifcUtils is responsible for identifying the file version, etc and instantiating the correct serialiser. The schema-dependent compilation is performed at the "serialiser" level (the individual projects are now called "ifcSerialiser_###"). The serialisers are linked to ifcUtils (instead of bouncer) and are invoked by the schema-agnostic ifcUtils project.

This pattern more closely matches IFCOS.

Dependencies exclusive to IFCOS are now resolved in the ifcUtils CMakeLists, which exports a single variable (IFCUTILS_LIBRARIES) for use by 3drepobouncer & 3drepobouncerTests.

Static Libraries

Our libraries (ifcUtils and the serialisers) are now static. This is because they now reference complex types such as RepoSceneBuilder that are outside of repo::lib, for which dynamic linking would result in a circular dependency. Since each schema gets its own compilation, and the libraries are only ever used in bouncer, dynamic linking offers no real advantages.

IFCOS is still built as a shared library.

Serialiser Pattern

The importer design now follows the serialiser pattern of IFCOS. A "serialiser" is created which outputs geometric entities (IfcGeom::Elements) from an iterator. For each of these elements, the tree above it is built on-demand by recursing the Ifc Relationship objects.

For each tree node built, we build the metadata nodes in the same way, by pulling values from Ifc Value objects, and recursively traversing Relationship types. Geometric entities are written using RepoSceneBuilder, so only exist inside that loop iteration, reducing memory.

Instancing

Instancing is used inside IFCOS, significantly reducing import time by allowing IFCOS to reuse mesh data instead of re-triangulating it each time. These instances are baked however before being handed to RepoSceneBuilder, as other parts of the platform don't properly support true instancing. Baking is performed using the matrix returned by the iterator, which is the same one IFCOS would use to bake before triangulation.

Metadata and Units

The approach to metadata and units has also been changed, to rely more on IFCOS' dynamic type system.

(1) Unit labels are built the same way as before (e.g. from IfcUnit objects and enums), but are now associated with specific Ifc Types (e.g. IfcLengthMeasure) dynamically through a dictionary. This dictionary is initialised on start-up based on the resource schema tables in the spec. (There is no programmatic mapping inside IFCOS.)

(2) The code no longer explicitly casts to simple named types, such as IfcLengthMeasure, to get their values or units. Only a specific subset of types which are involved in indirection to other entities holding metadata, are handled explicitly. Values are extracted from entities by directly accessing their attributes as instances of AttributeValue. AttributeValue holds the type of the underlying primitive at runtime, allowing it to be converted directly to a RepoValue. The Id (RTTI) of the class/type containing the attribute is used to get the units from the dictionary built at start-up.

Only a subset of Value types requiring complex formatting, such as GIS coordinates, are explicitly handled - the others go directly into RepoVariants.

The relationship tree is still searched recursively to extract metadata; determineActionsByElementType has effectively become collectMetadata, except that collectMetadata doesn't affect the tree (which is now built by getParent).

(3) Similarly, predefined property sets are extracted by enumerating over their attributes, as are classifications.

This reduces the number of types we need to maintain in if..else trees, along with the number of SCHEMA_HAS_... #ifdef pairs. It also unifies the handling of IfcValues and AttributeValues, maximising code reuse and allowing, e.g. units to automatically work for attributes in complex types such as IfcSite, as well as in property sets.

An example is the IfcClassification, which had the Location attribute removed in IFC4 and the Specification attribute added instead. Under the new pattern, we don't need to check the version of schema because all attributes are enumerated dynamically, so the correct name and string-ification is retrieved without selectivity.

The approach of using a recursive method to walk both relationships and interpret the objects they reference is still used; determineActionsByElementType has substantially become collectMetadata, though the difference is that the latter doesn't build the tree, as this is now handled separately via getParent.

RepoVariants

RepoVariants are used for metadata, instead of converting all metadata to strings.

LOD Support

LOD support has been added (though most IFC files contain tessellations already as the procedural geometry schema is not easy to use...)

Unit Tests

A full set of unit tests have been added for Ifc. The test files have been created by examining the spec, and the scripts and starter files to build them added to the tests repository.

The Ifc regression tests have been moved from st_3drepobouncerClient.cpp to ut_repo_model_import_ifc.cpp.

Due to the way the import now works - with tree nodes only being built for successfully tessellated geometry - and IFCOS handling all exceptions internally, the importer will no longer create scenes with missing nodes, so this test has been removed.

Dependencies

The IfcOpenShell libraries on travis have been rebuilt from IfcOpenShell 0.8.0 for Noble. The head of 0.8.0 as of writing this contains all the fixes in our local fork.

travis now uses the package manager's copy of OCCT, as IFCOS requires a later version and the infinite loop bug appears to have been fixed upstream.

Test cases