Digital Technology and Cultures/Understanding the Semiotics of Virtual Reality

Semiotics Definition and Theories

What is semiotics? “Semiotics is the study of signs and codes, signs that are used in producing, conveying, and interpreting messages and the codes that govern their use” (Smith, Moriarty, et. 2004). The theory of semiotics helps us to better understand how meanings are formed so that we can make better messages from our symbols or help us to correct problematic ways of making meaning. There are two main theories of semiotics written by Swiss linguist Ferdinand de Saussure and American philosopher Charles S. Peirce.

Ferdinand de Saussure’s theory stated that the meaning of the sign emerges from the relation between the signifier and the signified. Saussure’s contribution to the study of signs was in shifting from a traditional view with an emphasis on the nature of things to a “relational world view where meaning derives from the priorities human beings construct and perceive among signs in a system” (Mick, 1986). He believed that there is a dual relation between the concept (signified) and the sound-image (signifier). The signifier is the thing provoking meaning making. An example of this is the word cat (signifier) and the sound-image of a cat (signified). We word “cat” lives in the mind of those who use it and give it meaning. The image and meaning of the word cat may vary from person to person. Saussure’s theory was language centered focused and in relation between individuals and society.

Infographic of Semiotics Theories

American philosopher Charles Sanders Pierce differed from Saussure, he stated that there are three kinds of signs; symbols, icons, and indices. Symbols are something that only comes to designate a referent by common use (Pierce Edition Project, 1998). There is no causal link of what it signifies and there does not have to be a physical object. An icon stands for an object which resembles it and not just through a common use. An example of an icon is an emoji or a map. Photographs can also be icons. The indices are the logical relationships with what it represents in reality. An example, smoke is an index of fire. When we see smoke we associate it with fire. We learn all these meanings of symbols, indices, and icons through social interactions or observations of our surroundings.

Semiotician Umberto Nico proposed that every cultural phenomenon may be studied as communication. The importance of signs and symbols has been commonly recognized but only a handful of researchers have developed theory and research programs based on semiotics (Mick, 1986). There are some strengths and weaknesses to the theory of semiotics. Semiotic theory is sometimes criticized as being too focused on the unrealistic visions of representation as opposed to the more practical approaches of communication knowledge. We need to look further at the communication knowledge of the visual meanings created by our society so that we can revise accordingly or decide to keep it the same.

Semiotics can help us understand meanings of signs and how they are constructed in a context of digital technologies. The use of virtual reality technology allows us to be immersed in a virtual world that simulates reality using symbols, signs, and indices. It is people talking with technology and in turn, connecting to other people too (Harway, 1985). We need society and its institutions to teach us how to interpret signs and make meaning of symbols (Berger, 2013). Symbolic representations can affect attitudes about a place or people. With the use of virtual reality we can now share in the human immigrant experience of what it is like crossing the dangerous and dry hot desert sun to cross the United States border wall for a better life.

Imagine you are with your spouse and two children walking through a hot deserted desert with just the food and clothes on your back trying to cross a 20-foot-tall fence in hopes of a better future for your children. This is one of the many stories of immigrant families who have crossed the Mexico-United States border and part of a larger important immigration issue facing political debates. How do we fully understand the hardships of immigrants and share this unique human experience if we have never experienced this ourselves? One way to persuade an audience to empathize and understand this experience of crossing the border is through the use of virtual reality technology. Virtual Borders Arizona is a virtual reality art experience created by Gabija Grusaite and artist Jamie Toll that allows the viewer to experience the challenges and hardships of crossing the dry remote Arizona desert as an immigrant (Mladenovic, 2015). Jamie Toll created three large sculptures; a fried egg, a melting globe, and a human trap as symbols that represent the immigrant experience of survival in search of a better life. They placed the sculptures throughout the Arizona desert and took photographs and videos of the landscape near the border wall. Together the images, videos, and sculptures convey a powerful persuasive visual message of struggle and survival in the human experience of crossing the American border.

In Virtual Borders Arizona, the photograph of the desert is considered an icon. When we see an image of a desert we imagine seeing an oases of dry land with cactus plants, dry shrubs, rocks, and perhaps rattlesnakes. The fried egg, melting globe, and the human trap are all examples of semiotic symbols. A fried egg has several meanings and in this case it is a metaphor for how hot it is outside. We are able to understand the connotation of the symbol because we have a previous experience of cooking a fried egg and have a knowledge that the pan must be well over 100 degrees in order to cook the egg. Connotation is an idea or feeling that a word invokes in addition to its literal or primary meaning. Therefore when we see an image of a fried egg in the middle of a desert we reference our previous experience and knowledge in order to make sense of it’s meaning and provide us with the feeling of standing in the middle of a hot scorching desert.

The example of the Virtual Borders Arizona is how and why semiotics is an important tool to use in the process of understanding meanings in visual messages. Semiotics teaches us not only about the science of signs but also “meanings are based on society and it’s codes; society creates meaning in signs and these meanings can change” over time (Berger, 2013). When we apply the theory of semiotics we are better equipped to understand how meanings are made so that we can break apart problematic meanings of symbols, icons, and indices and revise them accordingly.

Intentionally we are able to make sense of the signs exhibited in Virtual Reality Arizona in order to accomplish persuasive goals of contemporary policy debates around immigration. The artists’ of Virtual Borders Arizona hope that the virtual experience of this imagery and sound allows the audience to participate in the immigrant experience and changes the existing negative connotations surrounding immigration in the United States. They wish to change the communication surrounding ‘immigration’ by persuading the audience to empathize with immigrants who struggle to cross the border for a better life. As Gruisaite explains, “Desert is a very powerful symbol of an extreme climate and landscape, and at some point of life everyone needs to cross a metaphorical desert in order to succeed or to survive.” (Alvarez, 2015). We can all relate to the human struggle of survival through the use of symbolic imagery which helps us create meaning and further dialogue about immigration.

References

Alvarez, Edgar. “VR exhibit sheds light on immigrants chasing the American Dream” Engadget (2015). Accessed on April 19, 2017. https://www.engadget.com/2015/07/15/virtual-borders-arizona-vr-exhibit/

Berger, Arthur A. “Semiotics and Society.” Springer Science and Business Media (2013): 22-26.

Haraway, D. J. “A manifesto for cyborgs: Science, technology, and socialist feminism in the 1980s”. Center for Social Research and Education” (1985).

Mick, David Glen. “Consumer Research and Semiotics: Exploring the Morphology of Signs, Symbols, and Significance.” Journal of Consumer Research 13 (1986): 196-213.

Mladenovic, Oliver, July 29, 2015. “Virtual Reality Experience Showcases Hardships of Illegal Immigrants” Branding Mag Website, April 19, 2017. https://www.brandingmag.com/2015/07/29/virtual-reality-experience-showcases-hardships-of-illegal-immigrants/

Project, P. “What is a sign?” In The Essential Peirce Selected Philosophical Writings (1893-1913). Bloomington, IN: Indiana University Press, (1998): 4-10.

Smith, Kenneth L., Moriarty, Sandra, Kenney, Keith, Barbatsis, Gretchen The Handbook of Visual Communication: Theory, Methods, and Media. New York: Routledge, 2004. Accessed April 21, 2017. https://books.google.com/books?id=ikmM_irMjKUC&printsec=frontcover&source=gbs_ge_summary_r&cad=0#v=onepage&q&f=false