English | ä¸æ–‡
This project combines three-dimensional (3D) modeling and programming knowledge to enable Virtual Reality (VR), Augmented Reality (AR), and related technologies. Its purpose is to combine the modeled 3D model with the OpenSpace3D editor to realize intelligent interaction, allowing the user to perform corresponding operations on the model based on VR technology.
The software can be installed from the following address: https://www.openspace3d.com/lang/en/support/download/
- Marker-based recognition. Displays physical markers representing virtual information on a given projection plane. The figure below shows the steps required to project a 3D object in a mark coordinate system.
- A marker-free system based on image recognition. In this case, the marker will be any image (it's more resource intensive) to find patterns that match the reference image.
GPS for position, compass for orientation and accelerometer for inclination. By drawing multimedia content coincident with the current location on the image seen in the mobile phone.
The basis of AR is to simulate the natural visual process, which is exactly what photogrammetry does. Using the principles of stereo vision in photogrammetry, our brains are able to form a three-dimensional world from two two-dimensional images given by each eye.
In computer vision (CV), eyes are replaced by cameras that capture images from the real world. During capture, the third dimension is lost and the resulting image is stored in a two-dimensional coordinate system in pixels. Therefore, the problem is "simplified" to find the correspondence between the coordinates of a point in the "object" system (3D) and the coordinates of the same point in the "image" system (2D), from which we can know the position of the camera relative to the target system and orientation, so any 3D coordinate can be "projected" onto the image system.
The OpenCV library in Python can be used to learn the basics of AR applications. This is a very powerful library dedicated to computer vision that allows performing tasks such as detecting relevant features in images, camera calibration and computing correspondences between object points and image points.
Some library files for Python 2.7 are uploaded in the liberias folder, including matplotlib, numpy and opencv. The scripts folder contains some calibration, capture, detection, extraction and projection function codes for reference.
After installing the necessary components, the latest library can be installed from: https://sourceforge.net/projects/opencvlibrary/files/
In this project, the previously created 3D model (.obj) is used, and the local camera of the laptop is used to complete the human-computer interaction. Regarding the marks, a plantilla template is used, allowing the model to be arranged according to the marks (model and virtual buttons) used by the application. The model mark is 8x8 cm, and the button mark is 3x3 cm.
