GRNTI 50.07 Теоретические основы вычислительной техники
BBK 3297 Вычислительная техника
One of the main problems of mixed reality devices is the physically correct representation of the brightness distribution for virtual objects and their shadows in the real world. In other words, restoring the correct distribution of scene brightness is one of the key parameters to solve the problem of correct interaction between the virtual and real worlds, but neural networks do not allow to determine the position of light sources that are not in line of sight. The paper proposes a method for restoring the parameters of light sources based on the analysis of shadows cast by objects. The results of the proposed method are presented, the accuracy of restoring the position of light sources is estimated and the visual difference between the image of the scene with the original light sources from the same scene with the restored parameters of light sources is demonstrated.
mixed reality, luminance distribution, High-Dynamic Range Image, 3D-scene, augmented reality, reconstruction of light source position, indirect illumination, direct illumination, point light source
1. Parsons, S. and Cobb, S., "State-of-the-art of virtual reality technologies for children on the autism spectrum," European Journal of Special Needs Education 26(3), 355-366 (2011).
2. Palmarini, R., "A systematic review of augmented reality applications in maintenance," Robotics and ComputerIntegrated Manufacturing 49, 215-228 (2018).
3. Izadi, S., "The Reality of Mixed Reality," Proceedings of the 2016 Symposium on Spatial User Interaction. ACM, 12 (2016).
4. Oculus Rift, https://www.oculus.com (10 April 2019).
5. Epson Moverio, https://moverio.epson.com (10 April 2019).
6. Microsoft Hololens, https://www.microsoft.com/enus/hololens (10 April 2019).
7. William, R. and Craig, A., [Understanding virtual reality: Interface, application, and design], Morgan Kaufmann, (2018).
8. Iis, T. P., Jung, T. H., and Claudia M. D., "Embodiment of wearable augmented reality technology in tourism experiences," Journal of Travel research 57(5), 597-611 (2018).
9. Trout, T., "Collaborative mixed reality (MxR) and networked decision making," Next-Generation Analyst VI. Vol. 10653. International Society for Optics and Photonics, (2018).
10. Sheena, B., Anandasabapathy, S. and Shukl, R. "Use of augmented reality and virtual reality technologies in endoscopic GraphiCon 2019 Intellektual'nye resheniya v komp'yuternoy grafike 23-26 sentyabrya 2019, Bryansk, Rossiya 31 training," Clinical Gastroenterology and Hepatology 16(11), 1688-1691 (2018).
11. Kiljae, A., Ko, D., and Gim, S. "A Study on the Architecture of Mixed Reality Application for Architectural Design Collaboration," International Conference on Applied Computing and Information Technology. Springer, Cham, 4861 (2018).
12. Livingston, M., Zhuming, A., and Decker, J. W., "Human Factors for Military Applications of Head-Worn Augmented Reality Displays," International Conference on Applied Human Factors and Ergonomics. Springer, Cham, 56-65 (2018).
13. Wang, X., Zhdanov, D. D., Potemin, I. S., Wang, Y. and Cheng, H., "The efficient model to define a single light source position by use of high dynamic range image of 3D scene, " Proc. SPIE 10020, Optoelectronic Imaging and Multimedia Technology IV, 100200I (31 October 2016)
14. Mandl, D., Yi, K. M., Mohr, P., Roth, P. M., Fua, P., Lepetit, V., Schmalstieg, D. and Kalkofen D., "Learning lightprobes for mixed reality illumination," 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 82-89(2017).
15. Supan, P., Stuppacher, I. and Haller, M., "Image Based Shadowing in Real-Time Augmented Reality," IJVR 5.(3), 1-7 (2006).
16. Haller, M., Drab, S. and Hartmann, W., "A real-time shadow approach for an augmented reality application using shadow volumes," Proceedings of the ACM symposium on Virtual reality software and technology. ACM, 56-65 (2003).
17. Everitt, C. and Kilgard, M. J., "Practical and Robust Stenciled Shadow Volumes Hardware-Accelerated Rendering," arXiv preprint cs/0301002 (2003).
18. Randima, F. and Kilgard, M. J., [The Cg Tutorial: The definitive guide to programmable real-time graphics], AddisonWesley Longman Publishing Co., Inc., (2003).
19. Kirk, D., [CG Toolkit, User’s Manual], Nvidia Corporation, Santa Clara, CA, (2002).
20. Richter-Trummer, T., "Instant mixed reality lighting from casual scanning," 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 27-36 (2016).
21. A.G. Voloboy, V.A. Galaktionov, E.A. Kopylov, L.Z. Shapiro. Raschet solnechnogo osvescheniya, zadannogo izobrazheniem s bol'shim dinamicheskim diapazonom // Trudy16-oy mezhdunarodnoy konferencii po komp'yuternoy grafike i ee prilozheniyam - GrafiKon'2006, Rossiya, Novosibirsk, iyul' 1-5, 2006, s. 467-472.
22. Voloboy A.G., Galaktionov V.A., Kopylov E.A., Shapiro L.Z. Modelirovanie estestvennogo dnevnogo osvescheniya, zadavaemogo izobrazheniem s bol'shim dinamicheskim diapazonom // "Programmirovanie", № 5, 2006, s. 62-80.
23. Valiev I.V., Voloboy A.G., Galaktionov V.A. Fizicheski korrektnaya model' solnechnogo osvescheniya, zadavaemaya izobrazheniem s bol'shim dinamicheskim diapazonom// "Vestnik komp'yuternyh i informacionnyh tehnologiy", № 9, 2009, s. 10-17.
24. Hariharan, Bharath, et al. "Simultaneous detection and segmentation." European Conference on Computer Vision. Springer, Cham, (2014).
25. Gupta, Saurabh, Pablo Arbelaez, and Jitendra Malik. "Perceptual organization and recognition of indoor scenes from RGB-D images." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2013).
26. Kim, Dong, Muhammad Arsalan, and Kang Park. "Convolutional neural network-based shadow detection in images using visible light camera sensor." Sensors 18.4 (2018): 960.
27. Ji, Jing, Xudong Jiang, and Wei Sun. "Shadow detection using double-threshold pulse coupled neural networks." 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2016.
28. Potemin I.S., Livshits I., Zdanov D., Zhdanov A., Bogdanov N. An application of the virtual prototyping approach to design of VR, AR, and MR devices free from the vergenceaccommodation confli
