English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
I. INTRODUCTION
The use of mobile devices has spread; they have become an innovative instrument supported by recent technological developments in the educational field, with the added ability to incorporate sensors, cameras, and new applications [1]. Furthermore, mobile learning environments can be enhanced by using advances in areas such as virtual reality (VR), mobile augmented reality (MAR) [2], mobile learning (m-learning), gamification, and learning analytics [3].
For its part, augmented reality (AR) allows real information or objects to be superimposed with virtual information to combine the two [4]. Its application is not limited to the sense of sight, in fact, it is applicable to all the senses, such as smell or touch [5].
One of the main current problems related to the development of contextualized AR applications, especially in primary and secondary education, has to do with the design and construction of 3D objects since it requires technical knowledge in modeling and graphic software [8], and teachers may not benefit from a lot of digital knowledge [8]. Considering that scenario, some studies have managed to confirm the benefits of the photogrammetry technique as a favorable alternative in the construction of 3D objects for different purposes. This is especially true when the design starts with real objects and follows specific characteristics that these objects have [9].
Photogrammetry is defined as the art of extracting information from a physical object by taking photographs [10]. In a more technical definition, Ch'ng et al. [9] define it as "a systematic image-based method for measuring and recovering the positions of points on the surface of photographic features for the purpose of producing 3D digital copies of physical objects".
In this sense, the studies found in the literature related to the use of photogrammetry for the development of 3D objects in AR applications have focused mainly on preserving heritage in museums and tourist places [17]. On the other hand, Portalés et. al [14] developed an application with AR to visualize different buildings in the city, performing a fusion of AR and short-range photogrammetry, which also included a discussion on the software problems experienced. In another study, Ruiz et al. [15] managed to successfully recreate in 3D, through AR, the model of the statue of Queen Victoria together with the interior of the Parliament of Ottawa in Canada. Likewise, Yahuda et al. [16] developed a study on the reconstruction of 3D scans using photogrammetry techniques by incorporating multimedia elements and 3D models visualized by AR. This was done for the use of Virtual Reality Headset devices to enhance the user experience.
Unfortunately, the development of applications directly related to education through the fusion between photogrammetry and AR is still scarce. While some research has been focused on photogrammetry learning at the university level [23], other studies have been oriented towards mechanical and industrial engineering aspects in higher education [27].
Kournoutas et al. [22] carried out an anatomy study creating different parts of the human body with images from an endoscope and using a photogrammetric process to create 3D models which would help to train future doctors. In another contribution, Katrina and Clancy [23] used it to create study material by designing a realistic nervous system model for university teaching. Likewise, some methodologies have been developed and applied in different university contexts [24]. For their part, Waldhausl and Ogleby proposed the 3x3 rule system [25], which can be implemented at any level, although it can show some variation with small objects. However, the photogrammetry technique as an alternative method developed by teachers and students at the secondary level has not been explored, specifically in the educational field of developing digital tools with AR.
This article proposes a method of constructing 3D models with photogrammetry, supporting the co-creation of AR applications in education with the participation of high school students and teachers [26].
This document is organized as follows: the first part includes the proposed method with photogrammetry, then the results are presented. After that, we present the applications developed with the proposed method and evaluation, and finally, a discussion and conclusions.
II. METHOD
For the development of the method, aspects of the literature were considered, such as developing a script and preparing the stage with the necessary equipment [24]. Additionally, we have considered the 3x3 method developed by Waldhausl & Ogleby [25], which leads to using three geometric rules and taking photos at different distances. Three photographic rules were also followed in terms of zoom, avoiding shadows and using a good camera, and other organizational rules regarding preparation and setup.
A. Proposed method
With the contributions of the literature, a new general method is proposed, established in 4 elementary processes, namely: 1. Creation of the co-creation team, 2. Construction process, 3. Optimization, 4. Evaluation and 5. Articulation. The method becomes iterative for each three-dimensional element to be generated.
The first part of the method is the consolidation of a co-creation team with students and teachers from the educational institution, preferably including the technology and computer science teacher from the school. Then, it relies on a process called 3D model construction, which guides the construction of objects; this process has some sub-stages.
The scene preparation is one of the main aspects of the method, as it can lead to optimal or, on the contrary, undesirable results. It requires following some recommendations of the place where the photographs are going to be taken. For this case study, a stage was prepared, with three sides where the distractors that may have remained in the background of the images that were captured were covered with a color surface that contrasts with that of the object.
In the tests carried out, it was necessary to take the photographs during daylight hours and in open places with a roof to help the object have a uniform and natural light as there should be no shadows. Additionally, the roof also prevented reflections, especially in items that had shiny parts.
Therefore, taking photos of the real objects is another fundamental aspect of the process, as they represent the raw material that then generates the 3D model. It is advisable to take as many photos as possible.
The photos are then uploaded to the photogrammetry software chosen among those available on the market. The size is generally not a problem, but the file format must be compatible with the software.
The next step is generating the 3D object, a process carried out by the selected photogrammetry software, which is an online platform in this case, therefore, not needing any additional action from the user.
Finally, the generated 3D object is exported or downloaded in this process, with the required format. The software generally applies various formats such as obj, stl, ftx, among others, as needed.
The next step of the method is optimizing or improving the object since, in many cases, photogrammetry does not achieve a complete model without errors. It is also necessary to adapt to the requirements of the programming language. In our case, a practical guide was developed with Blender [27].
Next, the generated and optimized model must be evaluated after observation, considering the colors, texture, general details, edges, presence of faults or holes. Section III of the evaluation includes a table with the models generated and evaluated.
Finally, articulating the result with the AR application is achieved with the support of the developer, taking into account the requirements of the programming language used. This process is reinforced in section D: Optimization.
B. General Recommendations
Taking pictures to generate good quality 3D models seems simple, but some considerations need to be followed.
1) Covering Possible Distractors. Photos also capture elements around the main object; to avoid this, creating a mini photo studio that covers these alternate objects is recommended.
2) Creating a Photo Studio. It is important to create a mini photo studio [28] that includes two or three sides that cover the back of the object (always facing the camera). It is also essential to include a surface with a color that contrasts with that of the object to avoid causing color confusion in the software, as shown in figure 2.
3) Device to Take the Photographs. Any device can be used to take the photographs, such as a high-end camera, a drone or a mid-range mobile device. However, it should be noted that the use of a camera with good resolution generates better results.
4) Device or Camera Settings. The flash should be turned off, and zoom should not be used to avoid distorting the quality.
5) Appropriate Lighting Place. Bright places that provide uniform lighting to the object and do not create shadows are best. Open spaces with a roof are recommended [9] to ensure that shiny objects do not generate reflections from the sun.
6) Camera and Object Distance. The same distance and angle from the camera to the subject should be maintained in all shots.
7) Camera Tilt Angle. The device used to take the photos (cellphone or camera) must have an inclination of 10 to 20 degrees depending on the size of the object.
C. Taking Photos
A series of steps have been established to take photographs on stage, supported by [25], [28]. Step 1: Place the static object in a fixed place without moving it. Step 2: Focus the camera until the object is clearly visible. Step 3: Capture a minimum of 40 photos per object; consider that each one must match the previous one between 50% and 60% in order to generate continuity, help the software recognize the object correctly, and associate each point properly. Step 4: Rotate the device or camera around the object by achieving uniform distances (equal) in each shot and going 360 degrees around the object. Step 5: Review the photos before uploading them to the photogrammetry software, check to confirm the presence of any problems.
D. Optimization of the Models
The optimization process is related to checking missing or excess parts, either on the surface or the background of the images, and also reducing their size and, at the same time, the number of vertices or polygons [29], to make the images workable in an AR application. This generated the need to develop a step-by-step manual through the Blender software, including importing the images, trimming the excess parts, reducing vertices and finally, exporting the ready-made objects.
In Figure 3, a three-dimensional element exported with defects can be compared against its counterpart, which was optimized and then incorporated into AR.
III. RESULTS
A case study was designed to validate the method applied to the development of 3D elements conducive to AR applications and the appropriate use with high school students and teachers. Fort this study, three applications were developed using Unity and SDK Vuforia, two programming languages used for games [30], as well as AR with markers.
A. Developed Applications
To advertise the gastronomy and tourist places of the department of Putumayo in Colombia, the application called "Mikuna" was developed. This first experience facilitated acquiring knowledge regarding the technical data needed to insert a 3D model developed with a photogrammetric process compatible with mobile applications in Unity.
The second application, "ChemAR" was focused on Chemistry as it is taught in high school, specifically Stoichiometry. Finally, the "AR-mBot" application was developed by taking into account the knowledge acquired, allowing for a refinement of the technique of taking photographic samples. The experience and lessons learned were collected following the methodological process presented in this document regarding the creation of 3D objects.
B. Quality of the 3D Models Created
A total of 18 3D models were created for the three applications developed. Different characteristics that can be observed directly were considered to evaluate the generated models, such as colors, texture, additional details of the object, general faults and edges that may occur for each model [31]. The co-creation team observed each item; a score of 1 to 5 was assigned and recorded in tables 1, 2, and 3, where the total score for each item was calculated. The higher the value rendered meant that the quality was also high.
Table 1 Quality evaluation of 3D models in the Mikuna application.
| Characteristic | Object 1 | Object 2 | Object 3 | Object 4 | Object 5 |
|---|---|---|---|---|---|
| Colors | 3 | 3 | 3 | 2 | 4 |
| Texture | 2 | 2 | 2 | 2 | 2 |
| Model details | 3 | 3 | 3 | 3 | 3 |
| No holes or flaws | 3 | 3 | 3 | 2 | 3 |
| Edges | 3 | 4 | 3 | 2 | 3 |
Table 2 Quality evaluation of 3D models in ChemAR application.
| Characteristic | Object 1 | Object 2 | Object 3 |
|---|---|---|---|
| Colors | 4 | 4 | 4 |
| Texture | 3 | 3 | 3 |
| Details of the Model | 3 | 3 | 3 |
| No holes or flaws | 3 | 3 | 3 |
| Edges | 3 | 3 | 3 |
Table 3 Quality evaluation 3D models in the AR-mBot application.
| Characteristic | 3D objects | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 6 | 7 | 8 | 9 | 10 | |
| Colors | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
| Texture | 4 | 5 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 5 |
| Model Details | 5 | 4 | 4 | 4 | 5 | 5 | 5 | 4 | 4 | 4 | 4 |
| No holes or flaws | 5 | 5 | 5 | 5 | 4 | 5 | 5 | 5 | 5 | 5 | 4 |
| Edges | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 5 | 5 | 5 | 5 |
On the other hand, three models were created with photogrammetry for ChemAR, while the remaining objects were created through Blender due to their complexity when applying this technique and the need to use a laboratory.
Finally, ten 3D models were created for the AR-mBot application, all related to the robot parts of the educational kit for the assembly module.
C. Evaluation by Experts
Five professional experts in mobile application development in Colombia participated in the evaluation of the three applications. They also had experience in the field of augmented reality applied to education. They received a checklist developed by Guimarães and Martins [32]l, which consisted of 22 items distributed according to the following variables: efficacy, satisfaction, and efficiency. Each item was related to the corresponding Nielsen usability heuristic, applied to AR, and was evaluated with a Likert-type scale from 1 to 5, where 1 was very low and 5 was high. The checklist and instructions were shared online by email, together with the developed applications, then the evaluators returned the results. The group of experts were not related in any way to this research project.
As presented in Table 4, after performing an ANOVA analysis of averages, it becomes evident that there is a significant difference between Armbot and the other two. This is due primarily to 3D objects having better quality and managing to display more information. The data can be compared with Table 3 related to the quality evaluation, thus confirming the implementation of the photogrammetry method.
IV. DISCUSSION
Photogrammetry can be a highly precise technique when carried out correctly [9]; however, it requires following some specific recommendations and a practical path described in the proposed method. Furthermore, some errors in this process can lead to adverse results. For example, if the photographs have resolution problems and the object is not well captured or coincides only partially with the same object in the following photograph, the software algorithm will have errors when generating 3D objects.
Photogrammetry is an excellent method that requires some practice and training in taking photographs [23], depending on the complexity of the real object. However, as shown in tables 1, 2 and 3, the quality of the results obtained can increase as the team acquires more experience. Likewise, the generated models can be used within AR development.
On the other hand, this technique can be unsuitable when the real object has areas that are difficult to photograph for different reasons [33].
A. AR and Photogrammetry
For mobile augmented reality (MAR) applications, special attention is required when implementing three-dimensional objects. Precisely, the number of vertices and the size of the 3D object must be controlled; otherwise, the load may be very high for any device and decrease graphics performance. For Unity, the maximum average number of vertices is 100,000 [34]. However, due to quality aspects, it is not recommended to take the reduction to the limit.
V. CONCLUSIONS
Currently, photogrammetry has become widely available and easy to use, increasingly employed in digitizing different real-world elements such as museum and heritage collections, architecture and construction or medicine, among others. This article presents a proven method of photogrammetry that creates custom-made three-dimensional modeling from real objects. It is oriented to students and teachers of secondary and primary educational institutions, and it is a technique that allows a greater diffusion and easy access for the development of new digital tools with AR to support the educational process.
An advantage of photogrammetry has to do with it being a relatively low-cost medium [35]. Therefore, it can be promising for students and teachers with low technical knowledge and scarce economic resources, giving good quality results when creating 3D objects following the method outlined in this study. The low cost is also due to the variety of affordable software and platforms for photogrammetry [17].
