Introducing an eye-gaze device to
an individual who is non – verbal can open up a world of possibility for them;
it can allow them to communicate, engage with games and play as well as
allowing them to access and control their environment.
When working with children who have the potential to use eye gaze, it can be difficult to find fun and motivating ways to encourage them to engage with the device. Introducing communication-based programs too early can be too demanding and may ultimately lead to failure using the device.
Smartbox Technologies have developed a program called Look to Learn and describe it as a motivating and fun way to get started with eye gaze technology. Every activity has been developed in consultation with teachers and therapists to improve access and choice-making skills. The software consists of 40 specially created activities that easily allows therapists, families and teachers develop basic eye-gaze interaction with the child. A companion workbook is also available from the Smartbox website to download (free) and helps track and document the child’s progress as they move through the program and the complexity of the activities.
Since the year 2000 Enable Ireland’s Assistive Technology (AT) training service have run a Foundations in AT (5 ECTS) course certified by the Technological University Dublin (TUD). Those of you reading this post will most likely be familiar with AT and what a broad and rapidly evolving area it is. While overall the direction AT has taken over the last decade is positive and exciting, it has also become a more challenging area to work in. As a result, the importance and value of the Foundations in AT course has also increased and this is both reflected in, and as a direct result of the calibre of course participant we’ve had in recent years. The wealth of experience brought by participants each year helps the course evolve and develop, filling in gaps and offering new directions for technology to support people in areas beyond primary needs such as communication, access and daily living. Last month we began what is a new effort on our part to share with a wider audience some of the excellent work produced by Foundations in AT course participants with Shaun Neary’s post Accessible Photography – Photo Editing with Adobe Lightroom & the Grid 3. This month we will look at another area of creativity, music.
Alex Lucas enrolled in the 2018 Foundations in AT course. As soon as we learned about his background and experience, we knew that his involvement in the course was an opportunity for us to learn more about accessible music technology and practice. Alex is an academic (PhD research student in Queen’s University Belfast), a maker, a musician, a developer and a product designer. Before returning to studies, he had gained 10 years’ experience working in mainstream music technology with big name companies like Focusrite and Novation. In Queens he is currently researching “Supporting the Sustained Use of Bespoke Assistive Music Technology” and is part of the Research Group: Performance Without Barriers. He also works with Drake Music Northern Ireland.
We could be accused of having underutilised Alex, but our suggestion for his project was to produce a resource that would act as an introduction to people new to the area of accessible music technology. Alex chose to focus on the mainstream Digital Audio Workstation (DAW) application Ableton Live and Switch input. As well as the project document (download link below) he released 5 really excellent tutorial videos on YouTube, the first of which is embedded here.
Alex kindly agreed to contribute to this post so we asked him why he chose to focus on Ableton, to tell us a bit more about his work in inclusive music and a little about the research he is currently undertaking at Queens. Over to you Alex..
There are many software applications available for computer-based music production. Ableton Live is arguably one of the most popular DAWs. When first released in 2001, Ableton Live set itself apart from other DAWs through a unique feature called Session View.
Session View is a mode of operation which can be thought of as a musical sketchbook providing composers with an intuitive way to create loop-based music; a feature which is particularly useful when creating electronic music. When combined with Ableton Live’s built-in virtual musical instruments and devices for creating and modifying musical ideas, we find ourselves with a rich toolset for composing music in inclusive settings.
How this works with groups?
Music connects people; we see this often when conducting group-based inclusive music workshops, making work of this kind essential to Drake Music NI. There could be up to twelve participants of mixed abilities in a typical Drake workshop. As Access Music Tutors, we approach group workshops by first speaking to each participant in turn to identify their creative goals. One individual may have an interest in playing distorted synthesiser bass sounds, while another may prefer the softer sound of a real instrument such as a piano. Knowledge of an individual’s creative goals and their access requirements is used to select an appropriate device for the participant to use to control a virtual instrument within Ableton Live.
In addition to the Access Switches described in the video’s mentioned above, Drake Music also uses commercially available assistive music technologies such as Soundbeam and Skoog, and mainstream MIDI controllers such as the Novation Launchpad. It’s possible to connect several of these devices to a single computer running Live.
Together, the group make core musical decisions; i.e. genre, tempo, musical key. The workshop will proceed in one of two ways, either we jam together, or record each participant in turn, building up a composition gradually using overdubbing techniques.
OMHI – One-Handed Musical Instrument Trust
There are a handful of other organisations within the UK, working towards providing inclusion in music. One notable organisation is the One-Handed Musical Instrument Trust (https://www.ohmi.org.uk/). Many traditional musical instruments are designed in such a way that they place a fundamental requirement on the musician; they must have two fully functional hands. This assumption results in the exclusion of some individuals from learning a traditional musical instrument. Furthermore, in some cases, accomplished musicians are not able to return to their instrument after losing the function of a hand due to illness or an accident. OHMI aims to address this shortcoming by running an annual competition which invites instrument designers to adapt traditional musical instruments to be played by one hand only. Many fantastic designs are submitted to OHMI each year. I’m particularly impressed by David Nabb’s Toggle-Key Saxophone (https://www.unk.edu/academics/music/_files/toggle-key-system.pdf) which retains all of the functionality of a standard saxophone while being playable by one hand.
Whilst OHMI primarily focuses on the adaptation of traditional acoustic instruments for inclusion and accessibility; my research centres on the challenges faced by disabled musicians in the long-term use of custom-made digital musical instruments.
In partnership with a disabled musician named Eoin at Drake Music NI, together we’ve been designing a digital musical instrument tailored towards Eoin’s unique abilities. Eoin has a strong desire to play electric guitar but as Eoin cannot hold a guitar, due to its physical characteristics, he has been unable to up until this point.
Using a motion sensor and an access switch, coupled with a Raspberry Pi embedded computer, Eoin is now able to play rudimentary guitar sounds using the movements of his right arm. We’ve tested several prototypes and are now in the process of assembling the instrument for Eoin to use both during Drake music workshops and at home.
As a musician, Eoin is the primary user of the device; however we’ve also been considering Eoin’s primary carer, his father Peter, as a secondary user. We’ve designed a high-level interface for Peter to use, hopefully allowing him to easily set-up the device for Eoin to use at home. We’re particularly interested in the longevity of the device, whether or not it’s viable for Eoin and Peter to use independently. Obsolescence can be a problem for assistive technology in general. Our current assumption is that obsolescence may be an issue with custom-made accessible digital musical instruments but hope, through this research to discover useful mitigation strategies.
Some time back, when I was finishing up a photography shoot,
I met a gentleman who had informed me that his photography career had been cut
short due to having a stroke a few years earlier. This was back in 2011, and
options were a lot more limited in terms of cameras, software and accessibility
in general. Earlier in the year, as part of my Foundations in AT course, it was
suggested to me to incorporate my photography background into my project. Now
in 2019, there are a lot more options for accessibility in photography, between
mounts for the cameras, wi-fi connectivity between camera and PC/Phone/Tablet.
However taking the photo is only half the work for a photographer.
Film photographers have to develop their photos, Digital photographers have to edit their photos. Adobe Lightroom is an industry standard program for editing photos. It is also very shortcut friendly. As a result, I was able to make it work with Grid 3 to enable basic editing such as converting to black and white, adjusting colour balance, brightness. Contrast and exposure. Cropping and converting an image from Portrait to Landscape and vice versa could also be achieved via the Grid. In the short time I had to create this grid, it can be easily expanded on, adding access to other modules (such as Export, Slideshow, Book, Print, etc) to access other features like Slideshow Templates, Print Setup, Exporting with previous settings or email a photo. While functionality of this grid is minimal, there is plenty of room for expansion.
I have always been a bit of a gamer. From Tetris on the original Gameboy to Sonic and the SEGA Mega Drive, I was always keen to pass the time away rapidly instructing a cartoon character to bounce from one side of the screen to another. Since I acquired my disability in 1999 though I felt that large parts of this world were now no longer accessible to me. I felt with limited use of my arms and no use of my fingers consoles were out of the question. That changed recently when the Xbox brought out their new accessible controller.
I had tried to use several different games on the PlayStation and the Xbox, my nephew had a PlayStation and I had been able to use the left stick and some of the buttons on the ordinary controller but despite me telling him not to use the trigger buttons which were inaccessible to me I still got hammered several times by him on FIFA.
This new accessible controller seemed as though it would provide me with the opportunity to have the full experience of console gaming again, but who is going to buy an Xbox One and accessible controller just to see if they can use it or not? Thankfully Enable Ireland came to my rescue and they allowed me to borrow their console and controller for the period of a month.
XBox Adaptive Controller (XAC)
controller is simple to use and simple to set up. I needed some help to physically
plug some aids in and out of the controller but apart from that it was a
The controller is setup for people of all abilities. The variety of configurations
is as wide as the number of disabilities of the people who it is geared to
Some games I used just the accessible controller with the coloured plug in
switches that Enable Ireland provided alongside the console.
For other more complicated games, I used the Co-Pilot feature. The Co-Pilot feature allows you
to use the ordinary controller as best you can while using the accessible
controller switches for any bits or buttons on the ordinary controller that you
My setup for Forza, the car racing game, was the simplest of
all. I took 4 of the aid switches and plugged them into the accessible
controller, one was plugged into RT for the accelerator, one was plugged into
LT for the brake, and the remaining two were plugged into the left and right
the d-pad. I placed the RT switch under my elbow to continuously accelerate, which
then meant my hands only had to focus on the three remaining buttons for
steering and braking. That was a huge success, and meant I did not need any
assistance throughout any of the gameplay on that particular game. Though that
does not mean I was a great driver!
FIFA I used the Co-Pilot feature. I used the ordinary controller as I had done
previously with my nephew, steering my player with the left stick while
passing, tackling, shooting, etc with the usual A, B, X, and Y buttons.
I used the Xbox Accessible Controller then for the sprint and switch player options.
I simply plugged in the switches into the RT and LT ports on the accessible
controller and played normally on the ordinary controller while occasionally
tapping the switches to change player or holding them down
with my elbow to sprint.
A very successful and intelligent solution which resulted in a 5-1 victory for
me over my nephew! His face was a picture 🙂
Ryse, GTA & Battlefield
Each of these I played with a similar set up to FIFA (pictured above). I used the Co-Pilot feature, the ordinary controller in conjunction with the accessible controller with four switches plugged into the RT, LT, RB, and LB ports.
These games were a bit more intricate in their controls in
comparison to the others and a little more difficult to use as a result. The
accessible controller meant though that it was possible for me to at least give
it a go.
This controls setup was good and meant that I
actually completed the story mode of Ryse, on easy.
I could play the vast majority of GTA and Battlefield without any difficulty,
but there were certain issues. To use the character’s “special abilities”
in GTA you had to press down on both the left and right sticks. I think you
could set that up but that would require two more switches which I didn’t have.
Also, on occasion, while I had all the right buttons the scenario in the game
was so complex that it involved pressing a number of buttons and steering at
least one, if not both, sticks at the same time. It was almost equivalent to
playing some musical instrument. On one mission I did have to fall back on some
assistance from my nephew.
it is still not quite the same as gaming prior to my disability the Xbox
Accessible Controller has reopened the prospect of gaming properly on a regular
basis and owning a console of my own again. This was a world that I thought had
long left me behind but thanks to Microsoft and Xbox I’m
right back in the game!
A few weeks ago, Lee Ridley (a.k.a. Lost Voice Guy) became the first comedian to win Britain’s Got Talent, now in its 12th year. As well as outshining his competitors along the way, and winning with a clear margin, Lee was a favourite with both the judges and the public.
What also makes Lee’s win even more incredible is that fact that he is the first person with a disability to win the show. For a stand-up comedian, being able to connect with your audience is essential, and he did this with self-depreciating humour, fantastic delivery and some killer one-liners, all done through the use of Alternative and Augmentative Communication(AAC).
AAC provides a means of communication for those whose speech is not sufficient to communicate functionally in all environment and with all partners. Lee uses a combination of two devices to support his communication – an iPad with apps, and a dedicated device called a Lightwriter.
Lee has been on the comedy circuit since 2012, and has won prestigious prizes, including the BBC Radio New Comedy Awards in 2014. Below is an interview that Lee participated in, via email, with Karl O’Keeffe back in 2013, which gives some insights into his process and the unique challenges that using a synthesised voice can present.
Karl: You are the first person ever to do stand up comedy who uses a communication device, so you had nobody to learn from. What are the most important techniques and tricks you have learned so far that you wish someone had told you when you were starting?
Lee: I think one of the most important techniques that I have learnt is how to deal with timing. Obviously it’s pretty hard to know when to leave pauses for laughter and stuff, especially as I have to pre plan this. I can pause whenever I want but you have to be ready to pause when people laugh otherwise the start of the next bit gets lost or they don’t laugh as long. You sort of have to know when it’s coming so you’re ready for it. Obviously every audience is different so I’m never going to get it right every time. I think I’m getting better at anticipating when to pause though.
Karl: I see from your videos that you use both a LightWriter and an iPad. Can you tell me which it better for stand up comedy?
Lee: I use my iPad for my stand up and I use my Lightwriter for day to day conversations. I just find that my iPad is easier to understand slightly. It is also easier to find my material on the iPad and because it backs up to the cloud, it’s a bit more secure and means i can use any Apple device. It’s also a bit sexier than my Lightwriter.
Karl: Do you always use the same voice? Why is the voice important in your performance?
Lee: I use the same voice mostly yes. However I do use other voices in my act as well for comedy purposes. For example, I use a woman’s voice to do an impression of my mother. I think that my main voice is important to me because it has become ‘my’ voice. It’d be weird if I changed it now.
Karl: What app do you use on the iPad for communication?
Lee: I use Proloquo2go, which is a brilliant app. It is very complex but easy to use at the same time. It does everything that I need it to do really.
Karl: What is your favourite app on the iPad?
Lee: I tweet quite a lot so I tend to use Tweetbot all the time. I couldn’t get through long train journeys with the Spotify app either!
Karl: Do you use any other Assistive Technology (computer access etc.)?
Lee: No. I only use Proloquo2go on my iPad and iPhone and then my Lightwriter.
What is your vision of a sensory room? A room with a soft mat, bean bags, bubble tubes, fibre optic lighting? Switch everything on when someone wants to use the room?
Wouldn’t you like to see a little more thought as to how to control the room’s special lighting, music, and objects so that it can be more immersive? How about a projection of a motorbike on the wall while you feel the vibrations in your cushion? Or a picture of an animal while you hear its name, or its sound? Or even a projection of a beach in a blue-coloured lit environment while you feel a breeze? Well, next-generation sensory rooms are here.
The SHX system developed by BJ Live allows all resources and solutions present in the room to act in coordination to create integral stimulation environments. A single control system allows integration of all the interactive and multimedia element of the room.
The SHX system supports 2 projections as well as 4 vibroacoustic elements in the room. There is a range of scenes provided by the SHX control software, combinations of videos, images, noise, lighting, vibration or effects that can be customised to the user.
The good: The system allows you to control the level of stimulation and the method of interaction to adapt the space for each user.
The not so good: Needs time to set up a sensory room installation.
The verdict: With a range of scenes provided by the SHX control software, combinations of videos, images, noise, lighting, vibration or effects can be customised for any user child or adult.
Enable Ireland’s National Assistive Technology Service has gathered together information on a range of accessible toys. It includes a variety of accessible games, apps, and toys. These are not recommendations but simply a selection of items which may be of interest, particularly at times such as Christmas and birthdays, when presents are high on the list of priorities.
Love it or hate it, the game of Minecraft has captured the imagination of over 100 million young, and not so young people. It is available on multiple platforms; mobile device (Pocket Edition), Raspberry Pi, Computer, Xbox or PlayStation and it looks and feels pretty much the same on all. For those of us old enough to remember, the blocky graphics will hold some level of nostalgia for the bygone 8 Bit days when mere blobs of colour and our imagination were enough to render Ghosts and Goblins vividly. This is almost certainly lost on the main cohort of Minecraft players however who would most probably be bored silly with the 2 dimensional repetitive and predictable video games of the 80’s and early 90’s. The reason Minecraft is such a success is that it has blended its retro styling with modern gameplay and a (mind bogglingly massive) open world where no two visits are the same and there is room for self-expression and creativity. This latter quality has lead it to become the first video game to be embraced by mainstream education, being used as a tool for teaching everything from history to health or empathy to economics. It is however the former quality, the modern gameplay, that we are here to talk about. Unlike the afore mentioned Ghosts and Goblins, Minecraft is played in a 3 dimensional world using either the first person perspective (you see through the characters eyes) or third person perspective (like a camera is hovering above and slightly behind the character). While undoubtedly offering a more immersive and realistic experience, this means controlling the character and playing the game is also much more complex and requires a high level of dexterity in both hands to be successful. For people without the required level of dexterity this means that not only is there a risk of social exclusion, being unable to participate in an activity so popular among their peers, but also the possibility of being excluded within an educational context.
Fortunately UK based charity Special Effect have recognised this need and are in the process doing something about it. Special Effect are a charity dedicated to enabling those with access difficulties play video games through custom access solutions. Since 2007 their interdisciplinary team of clinical and technical professionals (and of course gamers) have been responsible for a wide range of bespoke solutions based on individuals’ unique abilities and requirements. Take a look at this page for some more information on the work they do and to see what a life enhancing service they provide. The problem with this approach of course is reach, which is why their upcoming work on Minecraft is so exciting. Based on the Open Source eyegaze AAC/Computer Access solution Optikey by developer Julius Sweetland, Special Effect are in the final stages of developing an on-screen Minecraft keyboard that will work with low cost eye trackers like the Tobii Eye X and the Tracker 4C (€109 and €159 respectively).
The inventory keyboard
The main Minecraft on screen keyboard
Currently being called ‘Minekey’ this solution will allow Minecraft to be played using a pointing device like a mouse or joystick or even totally hands free using an eyegaze device or headmouse. The availability of this application will ensure that Minecraft it now accessible to many of those who have been previously excluded. Special Effect were kind enough to let us trial a beta version of the software and although I’m no Minecraft expert it seemed to work great. The finished software will offer a choice of onscreen controls, one with smaller buttons and more functionality for expert eyegaze users (pictured above) and a more simplified version with larger targets. Bill Donegan, Projects Manager with Special Effect told us they hope to have it completed and available to download for free by the end of the year. I’m sure this news that will excite many people out there who had written off Minecraft as something just not possible for them. Keep an eye on Special Effect or ATandMe for updates on its release.
You may have heard about or seen photos of Enable Irelands fantastic “No Limits” Garden at this year’s Bloom festival. Some of you were probably even lucky enough to have actually visited it in the Phoenix Park over the course of the Bank Holiday weekend. In order to support visitors but also to allow those who didn’t get the chance to go share in some of the experience we put together a “No Limits” Bloom 2017 Grid. If you use the Grid (2 or 3) from Sensory software, or you know someone who does and you would like to learn more about the range of plants used in Enable Ireland’s garden you can download and install it by following the instructions below.
How do I install this Grid?
If you are using the Grid 3 you can download and install the Bloom 2017 Grid without leaving the application. From Grid explorer:
Click on the Menu Bar at the top of the screen
In the top left click the + sign (Add Grid Set)
A window will open (pictured below). In the bottom corner click on the Online Grids button (you will need to be connected to the Internet).
If you do not see the Bloom2017 Grid in the newest section you can either search for it (enter Bloom2017 in the search box at the top right) or look in the Interactive learning or Education Categories.
If you are using the Grid 2 or you want to install this Grid on a computer or device that is not connected to the Internet then you can download the Grid set at the link below. You can then add it to the Grid as above except select Grid Set File tab and browse to where you have the Grid Set saved.
Today May 18th is Global Accessibility Awareness Day and to mark the occasion Apple have produced a series of 7 videos (also available with audio description) highlighting how their products are being used in innovative ways by people with disabilities. All the videos are available in a playlist here and I guarantee you, if you haven’t seen them and you are interested in accessibility and AT, it’ll be the best 15 minutes you have spent today! Okay the cynical among you will point out this is self promotion by Apple, a marketing exercise. Certainly on one level of course it is, they are a company and like any company their very existence depends on generating profit for their shareholders. These videos promote more than Apple however, they promote independence, creativity and inclusion through technology. Viewed in this light these videos will illustrate to people with disabilities how far technology has moved on in recent years and make them aware of the potential benefits to their own lives. Hopefully the knock on effect of this increased awareness will be increased demand. Demand these technologies people, it’s your right!
As far as a favorite video from this series goes, everyone will have their own. In terms of the technology on show, to me Todd “The Quadfather” below was possibly the most interesting.
This video showcases Apple’s HomeKit range of associated products and how they can be integrated with Siri.
My overall favorite video however is Patrick, musician, DJ and cooking enthusiast. Patrick’s video is an ode to independence and creativity. The technologies he illustrates are Logic Pro (Digital Audio Workstation software) with VoiceOver (Apple’s inbuilt screen-reader) and the object recognizer app TapTapSee which although has been around for several years now, is still an amazing use of technology. It’s Patrick’s personality that makes the video though, this guy is going places, I wouldn’t be surprised if he had his own prime time TV show this time next year.