As a result of my declining social media status and in view of the time constraints it was decided to incorporate a visual analysis, based on the same data source. This resulted in a dataset of 105 images collated using Snagit, the software used for the screencasts. In addition:
- Written observations of the screencasts were used for the development of the coding categories. These were subjected to a number of refinements for the purpose of building a methodological framework based on the mobility literature.
- A WordPress document was employed to experiment with the category building, using the tag feature.
The purpose of developing a framework is three-fold:
- First of all it combined some of the strands of the mobility literature in order to reflect and identify the mobility tools and how users create assemblages, the focus of the research questions.
- Secondly it offered an opportunity to contribute to the innovative practice of the Mobilities methods, as part of looking for new ways to ‘capture, track, simulate, mimic, parallel and “go along with” the kinds of moving systems and experiences that seem to characterise the contemporary world’ (Büscher et al, 2011a, p 7).
- Thirdly, developing the framework adds to a personal aesthetic reflection, crystallising features from a mobility experience that aligns with my personal flânerie.
Social semiotics
Gillian Rose (2012) explains that social semiotics looks at the context of visual communication, the modes of production of specific semiotic resources (such as images, any writing, layout, moving images) and the way these are interpreted by social process. The emphasis is thus on the meaning making in social settings. Jewitt and Oyama (2001) reaffirm how this is achieved by stressing that meaning making involves the description of ‘semiotic resources’, i.e. what can be said and done with images (and other visual means of communication) and how the things people say and do with images can be interpreted. Meaning making can be studied through various semiotics works: representational work, interactive work and compositional work which have a simultaneous effect.
Jewitt and Oyama (2001) state that any image not only represents the abstract or real world, but also plays part in some interaction and, with or without any accompanying text, constitutes a recognisable kind of ‘text’. Text here is interpreted in the broadest sense to include a painting, a poster, a film, etc.
Within the limited space of this project it is not feasible to consider all the semiotic factors at work and therefore, for the purpose of this analysis, I considered Kress and van Leeuwen’s chapter on ‘composition’ whereby composition refers to ‘the way in which the representational and interactive elements are made to relate to each other, the way they are integrated into a meaningful whole’ (Kress and van Leeuwen, 2006, p 176). This focus on composition is relevant since it will aid in the identification of the mobility tools as well as the creation of ‘assemblages’ in a social media art curating context.
Social semiotics and the aspects of compositional meaning
‘Composition’ is made up of three interrelated systems (Jewitt and Oyama 2001, Kress 2011, Kress and van Leeuwen 2006):
- Information value: the screen or web page is divided in ‘zones’. This is achieved by the positioning of elements on the left or right, centre or margin, upper or lower part of the screen. Western societies perform the direction of reading left to right. This left-right placement creates a ‘given-new’ structure, with ‘given’ what is already familiar (known) to the user, and ‘new’ which requires special attention. New can be problematic, contestable whilst the given is self-evident. Similarly, top and bottom indicate the split of ‘idealised, generalised or ideological’, with bottom representing ‘real’, the down to earth, the practical.
- Salience: some interface elements are more eye-catching, stand out and grab the attention.
- Framing: this indicates that elements on a page or screen are grouped or separated, achieved by dividing features such as lines, coloured spaces or other means.
The following diagram serves as an 'aide-memoire' and is also recreated in the Thinglink on the left:
Kress and van Leeuwen (2006) consider that these three principles of composition apply to the ‘integrated’ multimodality of the website such as text, images, video and sound and that these may have a spatial and temporal aspect, an unfolding in dimension and time. Furthermore the effect of the compositional elements is interactive and happening at the same time (Kress , 2011).
Social semiotics and contemporary communication
Kress (2011) suggests that in the context of contemporary communication we are now dealing with a level of ‘ordering’. This can be understood as an ordering based not only on the reader’s interests but also as an ordering of layout, with social media apps having a particular design. Adami and Kress (2011) further suggest that semiotic activity is a matter of ‘navigation’ and ‘selection’ from predetermined templates with the user adapting their needs to these templates. Templates are reflected in the design of ArtStack, Pictify and Trover following a top/middle/bottom zone. The top of the computer screen provides navigation, searching and filtering and the centre is dedicated to curating. The bottom of the screen directs the user to FAQ, Terms and Conditions and Privacy information.[1]
Furthermore, ‘navigation’ is now considered a key feature of semiotic action, with the ensuing ‘aesthetic of motion’ associated with the movement (Adami and Kress, 2011). Kress (2011) observes that a new theory with an adequate set of categories is now required to describe contemporary communication. It is this context that the social semiotic approach is employed.
Jewitt and Oyama (2001) and van Leeuwen (2005) indicate that the strength of employing social semiotics is by means of blending it as part of an interdisciplinary approach for critical research purposes. In line with this, I have developed a visual analysis based on a dual framework, with one layer broadly following the writing on ‘compositional meaning’ from the social semiotics literature. This is supported by a second layer focussing on the mobility themes.
[1] It should be noted here that the smartphone app and laptop or tablet access are quite different in this respect. The bottom of the smartphone has a lot more navigational functionality.
