A working prototype was built that enables a blind person to perceive a painting through touch and hearing, exploiting the phenomenon of synesthesia. Seeing a painting presents for the blind person two problems: form and color. A painting is a 2D representation of a 3D object, which engages the perceiving subject in cognitively reconstructing the full 3D objects. This may be solved through some kind of appropriate tactual experience. Color, however, cannot be tactually perceived: it is a purely visual experience. The central, innovative idea of our approach lies in the way we tackled with this problem: the use of synesthesia. Synesthesia is the cognitive phenomenon thereby a perception in one sense modality spontaneously evokes a corresponding perception in another sense modality (as when we say, for example, that a given color is "cold").
Though different associations of sense modalities might occur in synesthesia, we opted out for the one between color and musical sound. The basic correspondence was established by taking the three parameters defining each possible color - hue, luminosity, saturation - and assigning to each of them a corresponding parameter in the musical field: timbre, pitch, volume, respectively. In this way, each possible color, down to the last nuance, would be translated into a different musical sound, in such a way that similarity of color along three dimensions would be exactly matched by similarity of sound along three dimension.
In order to devise an equivalent of the two-dimensional representation of a three-dimensional object, the tactile form created for the painting was not a complete, but a partial one. More specifically, a 3D extrusion of the painting which lets the user touch only a portion of the object. The choice of the painting for the prototype fell on the well known classical painting: Raphael's, The Lady with the unicorn, of the Borghese Gallery in Rome.
The blind person would touch the model and have a tactile experience of the form. At the same time it would perceive a musical sound constituting the equivalent of the color carried by the form in the point of touch. In this way, form perception and color perception would be reunited and synchronous as it happens in vision for seeing people.
The working of this mechanism depends, however, on the possibility of delivering the sound corresponding to the point of the model/painting being touched at each instant by the blind person, while he is exploring it to perceive the form. A magnetic field three dimensional miniaturized tracking device, which could be worn on the blind person's index finger, was used for this purpose. In this way, the system would know which point of the model/painting the blind person would be touching at each instant. A digitized map of the painting would then tell which color would correspond to that point and the correspondence algorithm would instantly translate it into the appropriate musical sound.
Vedi anche:
Immagini:
