Perm, Russian Federation
GRNTI 50.07 Теоретические основы вычислительной техники
BBK 3297 Вычислительная техника
The paper is devoted to using Internet of Things technologies for hardware human-machine interfaces development. Thanks to these technologies, it may be possible to improve the capabilities of visual analytics systems with multiple modalities: movements, audio, etc. It can speed up semantic data filtering and interpretation, increasing the efficiency of analytics. We suggest using ontology engineering methods and tools to automate both the programming of custom hardware human-machine interfaces and connecting them to the third-party software. The proposed concept is tested by solving the real-world tasks of discovering the relationships between the psychological characteristics of the native speakers and their verbal behavior.
visual analytics, Internet of Things, human-machine interface, ontology engineering
1. Aravena P., Delevoye-Turrell Y., Deprez V., Cheylus A., Paulignan Y., Frak V., Nazir T. Grip Force Reveals the Context Sensitivity of Language-Induced Motor Activity during «Action Words» Processing: Evidence from Sentential Negation // PLoS ONE. - 2012. - V. 7, I. 12. DOI:https://doi.org/10.1371/journal.pone.0050287
2. Barsalou L. Perceptual Symbol Systems // Behavioral and Brain Sciences. - 1999. - V. 22. - PP. 577-609.
3. Belousov K., Erofeeva E., Leshchenko Y., Baranov D. «Semograph» Information System as a Framework for Network-Based Science and Education // Smart Education and eLearning. - Springer, 2017. - PP. 263-272. DOI:https://doi.org/10.1007/978-3-319-59451-4_26.
4. Chang Sh.-F., Ellis D., Jiang W., Lee K., Yanagawa A., Loui A., Luo J. Large-Scale Multimodal Semantic Concept Detection for Consumer Video // Multimedia Information Retrieval. - 2007. - P. 255-264. DOI:https://doi.org/10.1145/1290082.1290118.
5. Gallese V., Lakoff G. The Brain Concepts: the Role of the Sensorymotor System in Conceptual Structure // Cognitive Neuropsychology. - 2005. - V. 22, I. 3. - PP. 455-479. DOI:https://doi.org/10.1080/02643290442000310.
6. Hoffman P., Lambon Ralph M. Shapes, Scents and Sounds: Quantifying the Full Multi-Sensory Basis of Conceptual Knowledge // Neuropsychologia. - Elsevier, 2013. - V. 51, I. 1. - PP. 14-25. DOI:https://doi.org/10.1016/j.neuropsychologia.2012.11.009.
7. Ishii H., Ullmer B. Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms // CHI ’97 Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. - ACM, 1997. - PP. 234-241. DOI:https://doi.org/10.1145/258549.258715.
8. Laeng B., Sulutvedt U. The Eye Pupil Adjusts to Imaginary Light // Psychological Science. - 2013. - V. 25, I. 1. - PP. 188-197. DOI:https://doi.org/10.1177/0956797613503556.
9. Lockwood G., Hagoort P., Dingemanse M. How Iconicity Helps People Learn New Words: Neural Correlates and Individual Differences in Sound-Symbolic Bootstrapping // Collabra: Psychology. - 2016. - V. 2, I. 1. - PP. 1-15.DOI:https://doi.org/10.1525/collabra.42.
10. Meteyard L., Cuadrado S., Bahrami B., Vigliocco G. Coming of Age: a Review of Embodiment and the Neuroscience of Semantics // Cortex. - Elsevier, 2012. - V. 48, I. 7. - PP. 788-804. DOI:https://doi.org/10.1016/j.cortex.2010.11.002.
11. Pulvermüller F. Words in the Brain’s Language // Behavioral and Brain Sciences. - 1999. - V. 22. - PP. 253-279.
12. Rose K., Eldridge S., Chapin L. The Internet of Things: an Overview // The Internet Society (ISOC). - 2015 [Elektronnyy resurs]. URL: https://www.internetsociety.org/ resources/doc/2015/iot-overview (Data obrascheniya 17.07.2019).
13. Ryabinin K.V., Belousov K.I., Chuprina S.I., Shchebetenko S.A., Permyakov S.S. Visual Analytics Tools for Systematic Exploration of Multi-Parameter Data of Social Web-Based Service Users // Scientific Visualization. - M.: National Research Nuclear University "MEPhI", 2018. - Q. 3, V. 10, No. 4. - PP. 82-99. DOI:https://doi.org/10.26583/sv.10.4.07.
14. Ryabinin K., Chuprina S. High-Level Toolset For Comprehensive Visual Data Analysis and Model Validation // Procedia Computer Science. - Elsevier, 2017. - V. 108. - PP. 2090-2099. DOI:https://doi.org/10.1016/j.procs.2017.05.050.
15. Ryabinin K., Chuprina S., Belousov K. OntologyDriven Automation of IoT-Based Human-Machine Interfaces Development // Lecture Notes in Computer Science. - Springer, 2019. - V. 11540. - PP. 110-124. DOI:https://doi.org/10.1007/978-3-030-22750-0_9.
16. Ryabinin K., Chuprina S., Kolesnik M. Calibration and Monitoring of IoT Devices by Means of Embedded Scientific Visualization Tools // Lecture Notes in Computer Science. - Springer, 2018. - V. 10861. - PP. 655-668. DOI:https://doi.org/10.1007/978-3-319-93701-4_52.
17. Sanfelice R. Analysis and Design of Cyber-Physical Systems. A Hybrid Control Systems Approach // Cyber-Physical Systems: From Theory to Practice / Rawat D., Rodrigues J., Stojmenovic I. - CRC Press, 2015. - PP. 3-31. DOI:https://doi.org/10.1201/b19290-3.
18. Shchebetenko S. Reflexive Characteristic Adaptations Explain Sex Differences in the Big Five: but not in Neuroticism // Personality and Individual Differences. - 2017. - V. 111. - PP. 153-156. DOI:https://doi.org/10.1016/j.paid.2017.02.013.
19. Staats A., Hammond O. Natural Words as Physiological Conditioned Stimuli: Food-WordElicited Salivation and Deprivation Effects // Journal of Experimental Psychology. - 1972. - V. 96, I. 1. - PP. 206-208. DOI:https://doi.org/10.1037/h0033508.
20. Zhang P., Zhou M., Fortino G. Security and trust issues in Fog computing: A survey // Future Generation Computer Systems. - Elsevier, 2018. - V. 88. - PP. 16-27. DOI:https://doi.org/10.1016/j.future.2018.05.008.
21. Durov A.V. Konspekt lekciy po kursu «Teoreticheskaya i prikladnaya lingvistika» / A. V. Durov. - 2018 [Elektronnyy resurs]. - URL: www.durov. com/study/bilety-1718.docx (Data obrascheniya 17.07.2019).
22. Ryabinin K.V., Baranov D.A., Belousov K.I. Integraciya instrumentariya nauchnoy vizualizacii SciVi s informacionnoy sistemoy Semograf // Trudy 27-y Mezhdunarodnoy konferencii GraphiCon 2017. - Perm', 2017. - S. 138-141.
23. Ryabinin K.V., Chuprina S.I., Belousov K.I., Permyakov S.S. Metody vizual'noy analitiki variativnosti rechevogo povedeniya pol'zovateley social'nyh setey v zavisimosti ot psihologicheskih chert lichnosti // Trudy 28-y Mezhdunarodnoy konferencii GraphiCon 2018. - Tomsk, 2018. - S. 163-167.
