Permskiy gosudarstvennyy nacional'nyy issledovatel'skiy universitet
Minsk, Belarus
Perm, Russian Federation
CSCSTI 50.07
Russian Library and Bibliographic Classification 3297
The paper is devoted to using Internet of Things technologies for hardware human-machine interfaces development. Thanks to these technologies, it may be possible to improve the capabilities of visual analytics systems with multiple modalities: movements, audio, etc. It can speed up semantic data filtering and interpretation, increasing the efficiency of analytics. We suggest using ontology engineering methods and tools to automate both the programming of custom hardware human-machine interfaces and connecting them to the third-party software. The proposed concept is tested by solving the real-world tasks of discovering the relationships between the psychological characteristics of the native speakers and their verbal behavior.
visual analytics, Internet of Things, human-machine interface, ontology engineering
1. Aravena P., Delevoye-Turrell Y., Deprez V., Cheylus A., Paulignan Y., Frak V., Nazir T. Grip Force Reveals the Context Sensitivity of Language-Induced Motor Activity during «Action Words» Processing: Evidence from Sentential Negation // PLoS ONE. - 2012. - V. 7, I. 12. DOI:https://doi.org/10.1371/journal.pone.0050287
2. Barsalou L. Perceptual Symbol Systems // Behavioral and Brain Sciences. - 1999. - V. 22. - PP. 577-609.
3. Belousov K., Erofeeva E., Leshchenko Y., Baranov D. «Semograph» Information System as a Framework for Network-Based Science and Education // Smart Education and eLearning. - Springer, 2017. - PP. 263-272. DOI:https://doi.org/10.1007/978-3-319-59451-4_26.
4. Chang Sh.-F., Ellis D., Jiang W., Lee K., Yanagawa A., Loui A., Luo J. Large-Scale Multimodal Semantic Concept Detection for Consumer Video // Multimedia Information Retrieval. - 2007. - P. 255-264. DOI:https://doi.org/10.1145/1290082.1290118.
5. Gallese V., Lakoff G. The Brain Concepts: the Role of the Sensorymotor System in Conceptual Structure // Cognitive Neuropsychology. - 2005. - V. 22, I. 3. - PP. 455-479. DOI:https://doi.org/10.1080/02643290442000310.
6. Hoffman P., Lambon Ralph M. Shapes, Scents and Sounds: Quantifying the Full Multi-Sensory Basis of Conceptual Knowledge // Neuropsychologia. - Elsevier, 2013. - V. 51, I. 1. - PP. 14-25. DOI:https://doi.org/10.1016/j.neuropsychologia.2012.11.009.
7. Ishii H., Ullmer B. Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms // CHI ’97 Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. - ACM, 1997. - PP. 234-241. DOI:https://doi.org/10.1145/258549.258715.
8. Laeng B., Sulutvedt U. The Eye Pupil Adjusts to Imaginary Light // Psychological Science. - 2013. - V. 25, I. 1. - PP. 188-197. DOI:https://doi.org/10.1177/0956797613503556.
9. Lockwood G., Hagoort P., Dingemanse M. How Iconicity Helps People Learn New Words: Neural Correlates and Individual Differences in Sound-Symbolic Bootstrapping // Collabra: Psychology. - 2016. - V. 2, I. 1. - PP. 1-15.DOI:https://doi.org/10.1525/collabra.42.
10. Meteyard L., Cuadrado S., Bahrami B., Vigliocco G. Coming of Age: a Review of Embodiment and the Neuroscience of Semantics // Cortex. - Elsevier, 2012. - V. 48, I. 7. - PP. 788-804. DOI:https://doi.org/10.1016/j.cortex.2010.11.002.
11. Pulvermüller F. Words in the Brain’s Language // Behavioral and Brain Sciences. - 1999. - V. 22. - PP. 253-279.
12. Rose K., Eldridge S., Chapin L. The Internet of Things: an Overview // The Internet Society (ISOC). - 2015 [Elektronnyy resurs]. URL: https://www.internetsociety.org/ resources/doc/2015/iot-overview (Data obrascheniya 17.07.2019).
13. Ryabinin K.V., Belousov K.I., Chuprina S.I., Shchebetenko S.A., Permyakov S.S. Visual Analytics Tools for Systematic Exploration of Multi-Parameter Data of Social Web-Based Service Users // Scientific Visualization. - M.: National Research Nuclear University "MEPhI", 2018. - Q. 3, V. 10, No. 4. - PP. 82-99. DOI:https://doi.org/10.26583/sv.10.4.07.
14. Ryabinin K., Chuprina S. High-Level Toolset For Comprehensive Visual Data Analysis and Model Validation // Procedia Computer Science. - Elsevier, 2017. - V. 108. - PP. 2090-2099. DOI:https://doi.org/10.1016/j.procs.2017.05.050.
15. Ryabinin K., Chuprina S., Belousov K. OntologyDriven Automation of IoT-Based Human-Machine Interfaces Development // Lecture Notes in Computer Science. - Springer, 2019. - V. 11540. - PP. 110-124. DOI:https://doi.org/10.1007/978-3-030-22750-0_9.
16. Ryabinin K., Chuprina S., Kolesnik M. Calibration and Monitoring of IoT Devices by Means of Embedded Scientific Visualization Tools // Lecture Notes in Computer Science. - Springer, 2018. - V. 10861. - PP. 655-668. DOI:https://doi.org/10.1007/978-3-319-93701-4_52.
17. Sanfelice R. Analysis and Design of Cyber-Physical Systems. A Hybrid Control Systems Approach // Cyber-Physical Systems: From Theory to Practice / Rawat D., Rodrigues J., Stojmenovic I. - CRC Press, 2015. - PP. 3-31. DOI:https://doi.org/10.1201/b19290-3.
18. Shchebetenko S. Reflexive Characteristic Adaptations Explain Sex Differences in the Big Five: but not in Neuroticism // Personality and Individual Differences. - 2017. - V. 111. - PP. 153-156. DOI:https://doi.org/10.1016/j.paid.2017.02.013.
19. Staats A., Hammond O. Natural Words as Physiological Conditioned Stimuli: Food-WordElicited Salivation and Deprivation Effects // Journal of Experimental Psychology. - 1972. - V. 96, I. 1. - PP. 206-208. DOI:https://doi.org/10.1037/h0033508.
20. Zhang P., Zhou M., Fortino G. Security and trust issues in Fog computing: A survey // Future Generation Computer Systems. - Elsevier, 2018. - V. 88. - PP. 16-27. DOI:https://doi.org/10.1016/j.future.2018.05.008.
21. Durov A.V. Konspekt lekciy po kursu «Teoreticheskaya i prikladnaya lingvistika» / A. V. Durov. - 2018 [Elektronnyy resurs]. - URL: www.durov. com/study/bilety-1718.docx (Data obrascheniya 17.07.2019).
22. Ryabinin K.V., Baranov D.A., Belousov K.I. Integraciya instrumentariya nauchnoy vizualizacii SciVi s informacionnoy sistemoy Semograf // Trudy 27-y Mezhdunarodnoy konferencii GraphiCon 2017. - Perm', 2017. - S. 138-141.
23. Ryabinin K.V., Chuprina S.I., Belousov K.I., Permyakov S.S. Metody vizual'noy analitiki variativnosti rechevogo povedeniya pol'zovateley social'nyh setey v zavisimosti ot psihologicheskih chert lichnosti // Trudy 28-y Mezhdunarodnoy konferencii GraphiCon 2018. - Tomsk, 2018. - S. 163-167.
