What is digital twin
In real estate it is a virtual copy of a real-world building achieved with the help of 3D rendering technologies. This digital twin can be used for various purposes, such as monitoring, taking management decisions related to the property. It helps people better understand and manage the real thing by providing a detailed and dynamic representation in the digital form.
Requirements for a Digital Twin
- Accurate 3D model of the building existing in one of the industry standard formats, for example the IFC (Industry Foundation Classes)
- Building Information Modeling (BIM): BIM data is vital for construction and maintenance information. It includes details about the building’s components, systems, and materials, as well as their specifications and relationships.
- IoT Devices and data processing logic. These represent the sensors in the real estate used to collect data for the digital twin. These devices can be used for indoor climate control, energy, water consumption management, and more.
- A user-friendly interface for viewing and interacting with the digital twin. This might include 3D visualization tools and user-friendly dashboards.
- Frequent BIM and 3D model updates to keep the digital twin aligned with changes in the real estate asset.
Implementation of digital twin app
- 3D viewer app built with Nuxt.js and Xeokit SDK
- Converting 3D models from IFC to XKT (viewer compatible format)
- Data processing and storage (Node.js + Express, InfluxDB, Neo4j)
Implementing 3D viewer app for a digital twin with Nuxt.js
What is Nuxt.js
Nuxt.js is the most popular open-source framework for building modern single-page web applications with Vue.js. It utilizes the most of Vue.js power and provides additional features and conventions to streamline the development of server-side-rendered (SSR) and single-page applications.
Main benefits of using Nuxt.js:
- Fast to set up and run. It’s a good boilerplate for any new project, there is no need to configure a plenty of libraries to have the essential features for the modern web application, because most of the features you may need are already present in Nuxt or can be added easily as Nuxt plugins.
- Support for Server-Side Rendering (SSR). It allows to improve the initial page load time by rendering the HTML on the server and sending a fully rendered page to the client, which is particularly beneficial for SEO and performance. So, the rendering speed will not depend on each single device, because the content is generated by the server and returned to the client as static HTML.
- Facilitates the app routing by providing a powerful routing system that simplifies the management of routes and pages in your application.
- Facilitates the access control with the help of middlewares. Nuxt.js supports middlewares that can be used to modify the behavior of routes, making it easy to add authentication, authorization, or other access control logic to your application.
- Possibility to create and use app plugins in order to extend and configure the framework to suit your needs. There are many community-contributed plugins available and you can always add your custom plugins easily.
Nuxt.js is a good choice when it comes to the implementation of complex frontend functionality. In our case, using it as a starter for our 3D viewer app we reduce the time and resources needed to start the development process as it has all the essential features we may need for our application, such as authentication that can be easily configured for any OAuth provider, middlewares for implementing the access control, i18n localization module and etc. Also, there are plenty of ready UI component frameworks (Vuetify, Ant Design) that can be used with Vue and Nuxt to speed-up the development process. Moreover, if we have some custom UI design developed for our project we can still use these components, because most UI frameworks have quite good customization options.
Simple application structure:
- Login / Sign up page
- The authentication can be implemented on our backend side (Node.js + Express) with the help of node-passport module that provides a good and reliable toolset for OAuth2 authentication that follows the common standards
- On the frontend we can use the nuxt/auth module to facilitate the authentication, all we need to do is to provide the correct paths to endpoints of our authentication API. After successful authentication the data of the user will be saved to the application properties ($auth) that can be accessed by any component within the app. Having this data we can set up the access control for routes inside the application based on user role and other data returned by the API
- Dashboard with buildings
- On this page the user (staff of the property management company) will see all the buildings he has an access to with their information
- Here we hay show the capabilities (qualities) of sensors installed in this particular building such as: temperature, humidity, CO2, air flow, water consumption
- This page is supposed to have list view operations that will allow use to select and make some changes to the building information such as: title, address, property type, image
- Also, we may add new buildings using this page via special modal similar to the one we use for updating buildings where we may type all the necessary info, link the 3D model from uploaded and converted ones
- The building profile page with 3D viewer
- Here we may see the list of all spaces (rooms) and devices (air flow, heating, plumbing) within the building with the latest data from the sensors associated with them
- Another part of the page is a 3D viewer tab, where the 3D model of the building is rendered
- There are multiple tools that allow you to manipulate the view by hiding/showing objects based on their type, floor, the availability of data, etc. All these feature are about implemented with Xeokit and are covered further in this material
- Charts tab that allows to filter data by time period, selected rooms/devices and browse it in the form of infographics
- Info tab for showing the extended information related to currently selected node (building/room/device) with the latest data show as cards with icons demonstrating the measured quality (capability) and values
- From that page we might open a modal where we select a sensor from available and active ones and link it with the selected room or devices
- Admin dashboard for managing users and their access to buildings and their information
- Set of listing pages such as users, groups, invitations, buildings
- Page for managing sensors and their configuration for incoming data, scheduling, .etc
- Page where the 3D models are uploaded and their conversion to the XKT format required by Xeokit library is triggered
- The uploaded model may be assigned to the specific group or multiple groups, the the users that are in these groups will have an access to this building and it’s information
- As a result of 3D model conversion not only the file needed for its rendering is generated, the whole network including the building itself, all rooms and devices is being parsed and stored to the database. This allows associating sensors with rooms and devices of the building.
Xeokit SDK
Xeokit is an open source JavaScript software development kit created by xeolabs for viewing high-detail, full-precision 3D engineering and BIM models in the browser. It allows us to develop our own browser-based BIM viewer, which we can fully customize, extend with plugins and even integrate it inside the Nuxt application as it is in our case. It is maintained by the developer and multiple contributors, frequently updated and well documented with a lot of examples that demonstrate the essential features we may need in our digital twin solution. The examples of using the SDK may be found here.
Installing the SDK
Xeokit SDK is distributed as a node module as well. So, it is enough to install its latest stable version to our project’s folder following the instructions provided on Github
Implementing a 3D viewer
First of all, it would be a good idea to create a separate Vue.js component inside our Nuxt app that will be responsible for viewing the 3D models. There we will import all the essential functionality and plugins from the Xeokit SDK. Also there the viewer will be initialized based on properties that are passed to the components such as: source URLs of 3D models to fetch and render, some viewer configuration according to available API, .etc
Simple rendering of a 3D model can be done in a few steps within our component:
- Create an HTML element that will be a container for the viewer, set up the necessary style and dimensions
- Import Viewer from Xeokit SDK
- Initialize the viewer by using the constructor of the imported Viewer class passing all necessary configuration parameters, specifying the container element of the viewer canvas
- Specify the initial camera position for the scene. Instructions and available parameters can be found here.
- Create an instance of the loader plugin providing the viewer object, created in the previous step. Depending on the format of our 3D model file we may use WebIFCLoaderPlugin or XKTLoaderPlugin. The plugins should be imported from Xeokit SDK. Each plugin has its own configuration options that we may provide.
- Load the model by providing its source URL to the loader plugin initialized in the previous step. The file should be accessible from the browser for downloading it.
Following the steps above should result with a successfully rendered model. Now we can browse it by manipulating the camera, but it’s not enough for good user experience, we may need to add some 3D tools to improve the viewing experience:
- 2D/3D mode switcher
- Visible types selector that allows to make visible only objects of the selected types such as: doors, walls, roof
- Floor selector to manipulate the visible floors
- Floor map tool which allows to render small thumbnails of each floor in order to improve the user navigation within the viewer. Is implemented with the help of StoreyViewsPlugin from the Xeokit SDK
- The context menu that is triggered on a right mouse button click and may have the following operations available:
- Fit the scene to the current camera view
- Hide/Show objects
- X-Ray objects
- Clear selection
- Distance measurement tool that allow to measure the distance in an interactive way by selecting the starting point and then dragging the ruler that will appear to the end point of the measurement. In most cases, the 3D model will have the same dimensions as the building which allows you to get some accurate measurements. Can be implemented by using the DistanceMeasurementsPlugin
There are plenty of other tools that can be implemented with the help of Xeokit SDK according to your business needs.
Displaying sensors data
The rooms and other spaces will have different types of sensors connected to them, which means that there is a need to visualize them in 3D in a user-friendly manner. The simplest way to achieve this is to use the annotations provided by AnnotationsPlugin in Xeokit SDK. We are free to define the look and feel of the annotation and can put any content there. Each annotation will be attached to the corresponding object in the 3D viewer and will show the latest measurements with values and units.
The good idea will be to add the toggler that will allow the user to choose whether to show sensor data or hide them in order to simplify the 3D model browsing when there is a lot of data rendered.
Moreover, we may use the colors to improve data visualization:
- Set up coloring rules for measurements, for example if indoor temperature is under 18 degrees the space will be coloured with blue accent, if greater the green accent will be added, if it is above 24 the red accent will be added
- Use viewer functions available from Xeokit to adjust the colors of spaces and other objects
- Add selector for the displayed measurement type (indoor temperature or humidity) to prevent overlapping data to be rendered within the viewer
Parsing and storing the building structure
In order to have a clear structure of our building with the possibility to create a database relationship between rooms and sensors it would be a good idea to use a graph database such as Neo4j.
What is a Neo4j database?
Neo4j is a scalable graph database management system dedicated to storing and traversing relationships in computing. Instead of tables or documents it uses nodes and relationships in a graph structure to make the semantic queries possible.
Parsing the IFC model
For this purpose the open source IfcOpenShell toolkit may be used. We can use the Python module and run it from our backend application with Node.js. The documentation can be found here. As a result we will have the whole structure of our building with relationships between nodes in JSON format that allows us to recreate the parsed network in the Neo4j database.
Handling data from sensors
For this purpose we may use the InfluxDB which is fast and reliable time series database. It is designed to handle high write and query loads. InfluxDB is meant to be used as a backing store for any use case involving large amounts of timestamped data, such as IoT sensor data for our digital twin.
What do we need for data handling:
- The running instance of InfluxDB
- All sensors that are installed in rooms should also exist as a nodes in Neo4j database
- The API endpoint that will handle the incoming data from servers and store them to the InfuxDB and update the latest data for the sensor in the Neo4j database
- The API endpoint to retrieve the data from sensors stored in InfluxDB with possibility to filter by the sensor id and selected time period for displaying the data as charts in our viewer application
Preparing 3D model for viewerIn order to render the 3D model of our building which is in IFC format in our viewer application we need to convert it to the XKT format. We can do it with the help of xeokit-convert npm module that has all necessary tools available. It has to be installed to our backend application where the conversion can be triggered by the admin user after uploading the IFC file of our building. It will convert the model generating an XKT file that needs to be stored to some location, from where it can be downloaded by our 3D viewer application with the help of XKT loader plugin and then rendered.