A Tianjin University cultural heritage conservation team has devoted itself to cultural heritage protection and the vivid presentation of human civilization.

Zhang Jiawan is Director of the Information Technology Research Center of Cultural Heritage Conservation and Inheritance at TJU. In the past decade, with the help of modern information technologies, such as IoT and AI, this team has been dedicated to the protection of cultural relics. All the team members who are scientists dealing with computers every day have engaged in cultural heritage conservation.

On October 11, 2017, at the 92nd Anniversary of the Palace Museum, a new exhibition, “Discovering the Hall of Mental Cultivation: A Themed Digital Experience Exhibition" at the Palace Museum was opened to the public. This exhibition merged traditional Chinese culture and contemporary technologies. The newly emerging technologies would not only change the future, but also bring about a brand-new perspective to old things such as cultural relics and museums.

Time is the strongest enemy of the cultural relic preservers. The preservers have to “fight” against not only the changeable climate and unpredictable natural disasters, but also the tourists that do more harm to the cultural relics. As for vulnerable relics, even the vibration caused by the footsteps and the moisture of breathing may result in irreversible damage. Given this, human beings are always looking for a solution that satisfy both sides, so that the tourists are able to appreciate the heritage of our ancestors and no harm is caused to this heritage.　

Modern science and technology is the key solution to the problem. The so called ‘smart museum’ and preventive protection technology are two effective methods proposed by Zhang’s team. They tried to build a bridge with modern information technology that links the visitors with the ancient world, in order that the visitors are able to admire the vivid history, and that the natural and cultural heritage is better preserved.

Making a museum like a real person

The team aims to make a museum talk like a real human being. However, due to some limitations like space, only part of the wisdom of human beings and the splendid art in museums are able to be presented for appreciation. “Even in large museums, the presented cultural relics take up less than one percent of the total relics preserved in them.” Zhang said. He visited many museums in China, and the more he saw, the more he felt it is necessary to put most of the collections into exhibitions and that he had to do something. He noticed that only several thousand relics in a museum were exhibited at most, “so people only have access to a small part of the treasures.”　

Zhang is an expert of information technology, which will help to present the relics as vividly as possible. “The space is limited in the reality, but cyber space is infinite,” Zhang said. In previous attempts, people have thought about putting the collections on the internet, which is the so called virtual museum, or digital museum. However, in Zhang’s point of view, an ideal museum should be one that lives, thinks, and learns like human beings, and more importantly, it should be open to the public, shared by everyone, and interact with everyone and all of them are significant features of this era of the internet.　

Zhang was determined to apply an artificial intelligence solution to “bring the museums back to life”, namely to perceive, analyze and process the key information of a museum or a group of museums by means of new advances in information technology, such as cloud computing, big data, IoT, and mobile communication. In this way, artificial intelligence plays an important role in the improvement of the management of a museum, the conservation of collections and the service for the visitors.

As a matter of fact, the joint function of technology and culture has already made museums more interesting places. Zhang’s team has played a significant role in the construction of the Digital Gallery at the Gate of Correct Deportment, the Palace Museum, which is the first digital gallery in China that merges ancient buildings, traditional culture and modern technologies. They even bring back to life the vivid scenery of ancient folk activities (of a crowd gathered to drink water from a winding canal with one wine cup floating on it so as to wash away ominousness spirits) augmented with reality, through which the visitors can immerse themselves in the game. They can also write the Preface to the Orchid Pavilion Collection (calligraphy work of the well-known Chinese calligrapher Wang Xizhi) as if they were Wang Xizhi himself. What’s more, computers are there to vividly simulate the effect of ink rendering. “Tourists formed a long line to experience it,” Zhang said. He was aware that people were becoming very interested in the human-computer interaction equipment, VR and AR, and holographic, panoramic and 3D technologies. Therefore, smart museums are not the ones with upgraded technologies, rather, they should function like human beings do. Zhang’s team summarized the features of a smart museum: it is erudite, considerate, good at expressing ideas, intelligent and capable.　

From now on, a pair of AR glasses will be necessary when visiting a museum. As long as people put on the AR glasses, they will see vivid scenes and related information. People can talk to the glasses as they walk, asking questions at any time and getting answers. If they go to a fragmented relic, they will see the holographic picture of a complete one. People can even “fetch” and appreciate the relic without actually moving it from the showing case. Also, without paying a visit to the far away Mogao Grottoes, people can enjoy its splendid holographic picture which is exactly the same as the real experience.　　

The museums equipped with an “intelligent brain”.

It is not that the relics can move that matters, rather, they should be intelligent, which makes it necessary to give the relics an “intelligent brain”. It occurred to Zhang that the knowledge graph in the field of knowledge engineering may be helpful in this work. To construct a knowledge graph is like to equipping the museum with a brain.

The knowledge graph is a concept derived from modern information technology. To put it simply, the applying of a knowledge graphic is a method of teaching a machine to read and understand data. Seemingly unrelated knowledge is linked by their relevance to form a semantic network, which is a graph to the machines. In the emerging AI technology, the machine learns through knowledge graphics.　

This is what a traditional museum lacks. For all the time, the commentary of museums is clichéd and boring, which cannot keep people’s interest. The clichés are to be blamed for the fragmentary and irrelevant information about the relics, because many relics only have a few commentary words, which merely acquaint people with what this is and what that is, but the historical and cultural background are rarely shown. “In fact, every relic has its own unique and abundant history, and the understandings vary from person to person,” Zhang said. Likewise, the same problems exist in information sharing among museums. The lack of interconnectivity results in the isolation of each museum.　　

Just as people turn to Baidu for answers, Zhang hoped that there would be a search engine for relics. However, most of the relic knowledge is kept within the professional world, which means that only a small number of experts have access to the knowledge in this field. He said, it would be much more convenient to present the relic knowledge to people, if the massive and multisources of relic data, including the relic itself, its environment, the historical documents, the archaeological materials, the related research achievements, and its network database, could be included in a knowledge graph and organized by natural language, big data and bibliometric, and so on.　　

“These data not only have to be sorted out, but also have to be calculable and searchable,” Zhang said, which means it is necessary to form a norm for Chinese relic knowledge graphs. Only with this “smart brain” can people break the isolation among museums and share this knowledge, and the whole Chinese history and culture would be able to unite to understand the perspective of the relics.

Based on the knowledge graph, visitors may get a personalized tour according to their preference and time. Meanwhile, the most suitable high and new technologies could be utilized to demonstrate and spread the knowledge.　　

At present, Zhang’s team is cooperating with the top experts in Shanxi Museum and Hunan Museum to help to construct a relic knowledge graph and knowledge platform.

“Health Physicians” for ancient architectures and cultural heritage

Apart from those collected in museums, there are still many other historic cultural sites, historic constructions, grottoes and murals. They are located in different areas throughout the whole of China, and they represent the history. Statistics show that there are over 780,000 immovable relics in China, and they are facing challenges from weathering, rainfall, and damage from human activities every day. Even the moisture in the air keeps damaging them.　　

Although there are strict conservation measures, there is still damage done to precious cultural relics like the Grottoes, Murals and Terra-Cotta Warriors, due to the lack of technologies for prevention and protection. “Rescuing conservation is always too late,” Zhang told journalists. Protection by prevention has already been put forward, but there were not suitable measures to carry it out.　

“People previously turned to simulations in labs, but the thing was it was difficult to simulate the coupling effect in the real world,” Zhang said. Any single problem has many causes, but people got stumped by the relations between the problem and their sources, due to the lack of detection means and related data.　

In view of the present bottleneck in the conservation of cultural relics, Zhang’s team has come up with protection by prevention technologies with the relativity model of cultural relics and their damaging sources as its core. This technology has already been applied in some cultural heritage sites, such as the Dunhuang Academy China, the Summer Palace and the Labuleng Temple.

The team has cooperated with the experts in the Dunhuang Academy China. They chose 11 caves and set up 47 detection points and then began to continuously detect the damaging factors to the murals and the painted sculptures in the caves. They thoroughly considered four represented types of damage and the severity, the location of those caves, and the degree of opening to the public, and did a quantitative analysis.　

By comparing the results, they obtained in 2014 and 2016, they found out for the first time a 0.1 millimeter change in the cultural relic over the period of a year. This finding was recognized as a substantial breakthrough by experts in this field.　

“Only when we made a comparison did we know how much the mural was damaged,” Zhang said. “In previous detections, people only detected the environment, instead of the relic itself. How did the relic itself interact with the environment always perplexed the experts?” These technologies would help the experts find out measures to deal with the damage. “We will build a platform that integrates the perception, analysis, assessment and feedback,” Zhang added.　

Although located in the desert, the Mogao Grottoes suffered from unprecedented flooding in 2011 and 2012. In 2012, Zhang’s team developed the first Risk Management and Decision Support System for Cultural Heritage Sites, and carried out the pilot application in the Mogao Grottoes. The system helped in detecting and analyzing the risk of severe natural disasters like flood, violations in the surrounding environment, gradual environmental change, and the risk from the tourists, etc. “If there were sudden downpour in the vicinity of the grottoes, the system will calculate the direction, yield and speed of the flood according to the rainfall, and provide the best measures to deal with it to make sure the grottoes are perfectly safe,” Zhang said.