Why is it important to study fragments in Cultural Heritage Studies?
Ceramic potteries are one of the most frequently discovered archaeological artifacts. Since they are usually short-lived (in archeological terms), researchers find these artifacts useful to analyze chronological and geographical features, given that shape and decoration are subject to significant changes over time and space [0]. This analysis gives a basis for dating the archaeological strata and provides evidence from a large set of valuable data, such as local production, trade relations, and consumer behavior of the local population. Unfortunately, ceramics are fragile; therefore, most of the actual ceramics recovered from archaeological sites are fractured, so the vast majority of the available material appears in fragments. The reassembly of the fragments is a daunting, delicate, and time-consuming task, done almost exclusively by hand, which requires the physical manipulation of the fragments, which ideally should be as short as possible for conservation purposes.
Iberian Pottery Data
In this project, we used a collection corresponding to Iberian wheel-made pottery from various archaeological sites of the upper valley of the Guadalquivir River (Spain). The ceramics are classified into eleven different classes based on their shape. These classes consider the forms of the lip, neck, body, base, and handles and the relative ratios between their sizes. Nine of these classes correspond to closed pottery shapes, and two others belong to open ones [1].
Given that the existing dataset is based on wheel-made pottery techniques, we can assume small asymmetric perturbations in the ceramic. We generate each 3D model as a solid of revolution. First, we extract the shape information from the profile by means of semilandmarks equally spaced in the contours. Then we do the spin, converting the semilandmarks capturing the 2D pottery shape information into a mesh. Lastly, we transform the mesh into a set of voxels.
After we have our complete voxelized model, we fragment it using a method based on the Discrete Voronoi Chain (DVC) algorithm. The DVC algorithm is composed of two steps. First, it generates a random list of Voronoi region centers in the model, each corresponding to a fragment. Second, it assigns the voxel to a section by following a region-growing approach taking each center as a seed until all the voxels have been traversed. However, this procedure can assign voxels in the border between two regions to an incorrect one. Therefore, an additional distance check to the centers of each region is required to guarantee the correct assignment. The resulting number of fragments depends on the type of vessel (closed or open).
Generative Adversarial Networks
A typical GAN [2] framework contains a generative (G) and a discriminative (D) neural network such that G(z) aims (in our context) to generate realistic artifacts, while D(x) learns to discriminate if a sample is from the real data distribution or not.
In our case, z is a voxelized input fragment, different from [3], and G(z) represents the generator function that maps the fragment z to the data space of a complete Iberian voxelized pottery.
This type of network is named Autoencoding GAN (AE-GAN), in which G added a network of encoders that are trained to learn the mapping of each fragment sample to a point in latent space [4], and the decoder learns to map each point to a complete pottery.
The data element x, corresponds to a three-dimensional binary array containing the voxelized pottery geometry. D(G(z)) is the probability that the output of the generator G is a real artifact from the Iberian pottery 3D dataset.
D tries to maximize log(D(x)), which is the probability of having a correct classification of actual voxelized real ceramics, while G tries to minimize log(1 – D(G(z)), which is the probability of D recognizing any of the generated outputs by G.
Results
In order to evaluate the performance under different relative sizes of fragments, we stratified our fragment generation into three main groups: 15-20%, 20-30%, and 30-100% of the initial ceramic model. We can observe that the averaged MSE=0.06 and DSC=0.70 across all classes for the completion of the ceramic is consistent across fragment sizes. This means that our AE-GAN can complete ceramics from smaller to larger initial fragments. Interestingly, we can observe that specific pottery classes are easier to reconstruct across all sizes, such as Classes 10 and 11. We can 
hypothesize that the improvement over the averaged performance of the method over these classes is because we have more examples of these classes in our dataset. Furthermore, these classes correspond to a homogeneous type of open shapes (e.g., plates).
We also used a pottery classifier; this model yields the same performance in test datasets when evaluating real voxelized potteries and completed potteries from three different groups sizes of fragments by our proposed method.
Lastly, we did a domain experts’ case study that yielded that the reconstructed ceramics’ quality perceived by the archaeologists had a mean score of 2.09 with a standard deviation of 0.61 (with range values between [0, 3]). Additionally, we were interested in evaluating if the Iberian style holds in the completed samples. The archaeologists considered the ceramics to have an Iberian style in mean score 3.93 (std 1.16), with 5 fully Iberian Style.
From this preliminary study, we can conclude that archaeologists judge that the model generated a correct Iberian style from an initial fragment and also consider that the reconstructed pottery is between Good and Very Good. At the end of the questionnaire, we included a comment section to enable unstructured feedback. The comments across evaluators agree on the need for better visualization tools, such as including a scale factor and improving the edge inspection of the models, as these are key factors while evaluating the Iberian Style.
Limitations
Asymmetry in generated potteries: Some completed samples do not have adequate symmetry. The pottery used for this work corresponds to lathe potteries, and those are symmetrical by design. This type of problem is caused by the lack of examples in some classes.
Irregularity in generated potteries: For example, incomplete reconstructions or the mismatching alignment of the fragment with the proposed completion model.
Takeaway and Resources
Results suggest that we can generate potteries that conform to the structure of Iberian ceramics and fulfill experts’ validation criteria. Open source code and 3D dataset are available! Future work will focus on mesh directly and weighted fracture simulations to account for class imbalance. More details in this presentation.
Look out for our work @ IJCAI 2023
Thursday, 24th August at 11:45-12:45 (AI and Arts: Arts, Design and Crafts Session)
#ARTS2515 IberianVoxel: Automatic Completion of Iberian Ceramics for Cultural Heritage Studies. Pablo Navarro; Celia Cintas; Manuel Lucena; José Manuel Fuertes; Antonio Rueda; Rafael Segura; Carlos Ogayar-Anguita; Rolando González-José; Claudio Delrieux
References
[0] Eslami, D., Di Angelo, L., Di Stefano, P. and Pane, C., 2020. Review of computer-based methods for archaeological ceramic sherds reconstruction. Virtual Archaeology Review, 11(23), pp.34-49.
[1] Lucena, M., Fuertes, J.M., Martínez-Carrillo, A.L., Ruiz, A. and Carrascosa, F., 2017. Classification of archaeological pottery profiles using modal analysis. Multimedia Tools and Applications, 76, pp.21565-21577.
[2] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y., 2014. Generative adversarial nets. Advances in neural information processing systems, 27.
[3] Wu, J., Zhang, C., Xue, T., Freeman, B. and Tenenbaum, J., 2016. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. Advances in neural information processing systems, 29.