Have you ever considered controlling your computer or mobile devices with your wheelchair joystick?
As well as the basic wheelchair functions such as driving, the CJSM2 –BT also enables control of a computer or mobile devices and so the integration of environmental controls is possible. The same controls that the user drives the power wheelchair with, typically a joystick, can also be used to control an appliance within their environment.
For example for chairs with R-net controls you can replace the old joystick with a CJSM2 –BT as seen in the video below. This R-net Joystick Module has Infra-Red (IR) capabilities included. IR technology is widely used to remotely control household devices such as TVs, DVD players, and multi-media systems, as well as some home-automation equipment. Individual IR commands can be learned from an appliance’s remote handset and stored in the CJSM2.
Integrated Bluetooth technology is also an option, to enable control of computers, Android tablets, iPads, iPhones and other smart devices from a powered wheelchair. To switch between the devices, the user simply navigates the menu and selects the device they wish to control. The R-net’s CJSM2 can easily replace an existing R-net joystick module, with no system re-configuration or programming required.
As well as Curtiss-Wright’s R-net controls, other wheelchair controller manufacturers have Bluetooth mouse options too, including Dynamics Controls with their Linx controller and Curtis instrument’s quantum q-logic controller.
Dragon’s voice recognition software enables people to control their device, create content, browse and write an email, update spreadsheets, surf the Web and create documents using only their voice.
A demo of correcting, formatting and proofreading using Dragon naturally speaking 13.
Dragon NS provides a means voice to text production not only in word processing applications but also to control your computer operations. This for me is the main advantage over other voice to text programmes- which are often “in app” such as the microphone in the Pages app.
Dragon NS versus other voice to text software- my take on it!
Dragon NaturallySpeaking for the PC is much more powerful than the built-in voice recognition software in android or within the iPhone (Siri) i.e. less inaccuracies and more time efficient. It can dramatically cut down the time it takes to create email, word documents and other correspondence on your PC.
It Learns. Dragon NaturallySpeaking actually improves through use. It learns about how you speak, how you sound, what words you use and it creates a database called a voice profile. This voice profile matures over time and allows Dragon NaturallySpeaking to become very accurate with regular use.
Dragon NaturallySpeaking on the PC has “regional accent modelling”. This makes the program far more accurate than basic mobile device speech recognition which uses a generic accent model.
Dragon NaturallySpeaking adapts to your specific vocabulary. Siri or the Google android speech recognition application do not do this, they run off a generic limited vocabulary.
Amount Processed. Free speech software on your phone can only process 30-second chunks of speech. Dragon speech recognition on the PC is continuous for a long as you can talk and doesn’t need a continuous internet connection.
The not so good
Good flow of speech is important, even if just for short passages. Dragon NS writes everything you say; even inflections of speech such as “mmmm” and “eh”. If a user tends to use these inflections in speech, it will type these inflections. Continuously deleting them can be time consuming and frustrating. Training oneself not to use these inflections can be very tricky.
The user needs to be very cognitively able to command the system with their voice, planning out the actions and remembering specific commands.
The verdict
Fantastic software for the right client, especially if for any reason direct access is not an option. Even if a form of direct access is an option for the client, Dragon NS is still a nice option for long passages of text production. For the wrong client, this software would be more of a hindrance and a frustration than a help.
This is the first in a two part post about Enable Ireland’s Immersive Media Beyond Boundaries Garden project. If you want to try the apps for yourself you can get them from Google Play here or there are links and some more information on our website here. This first post (Part 1) will give a brief background to Virtual Reality and related technologies and look at some of the research into its potential in the area of autism. Part 2 of the post will outline how we put our Beyond Boundaries and SecretGarden apps together and how we hope to incorporate this technology into future training and use it to support clients of our service.
Background: VR, AR, Mixed Media, 360 Video?
Virtual Reality, referred to as the acronym VR, is one of those technologies that is perpetually “the next big thing”. If you grew up looking at movies like Tron and The Lawnmower Man (giving away my age here), VR is probably filed away in your brain somewhere between hoverboards (that actually hover) and teleportation. When the concept of a technology has been part of popular culture so far in advance of the capability of its realisation, it can hinder rather than promote its development. The trajectory the evolution of VR has taken however is much closer to a technology like Speech Recognition than hoverboards. VR, as with Speech Recognition, saw a great deal of progress in the latter part of the 1980s. With both technologies, although important, this progress was almost nullified by the hype surrounding and subsequent commercialisation of a technology that clearly wasn’t ready for the public consumption. The reality of what VR could offer at the time led to people becoming disillusioned with the technology.
Before I talk about how VR is being used in the area of autism it’s worth clarifying what exactly is meant by some of the terms that are being used. As an emerging technology there is still quite a lot of confusion around what is meant by Virtual Reality and associated technologies; Augmented Reality (AR), Mixed Reality, Immersive Media and 360 Video. First let’s look at the video below which explains what VR and AR are and how they differ.
So what is Mixed Reality? Well in short Mixed Reality is a combination of VR and AR, in theory offering the best of both. Mixed Reality is also closely associated with Microsoft and other Windows aligned hardware manufacturers. Have a look at the short video below.
360 degree Video and Photography are less interactive than the technologies discussed above. The viewer is also restricted in terms of movement, they can only view the scene from the position the camera was placed. Movement can be simulated to some extent however through the use of hotspots or menus, allowing them to navigate between different scenes. More traditional film techniques like fading between scenes can also be used as in the video below. 360 Degree can be either flat or in stereo. Stereo video or 3D video is captured with a camera that has 2 lens about the same distance apart as a person’s eyes. Each eye then gets a slightly different view which our brain stitch together as a 3D image.
Finally Immersive Media is frequently used as an umbrella term for all the technologies discussed above but would more correctly refer to the less interactive 360 Video and Photography.
Immersive Media and Autism
Since the early days of the technology people have proposed that VR may offer potential as a therapeutic or training tool within the area of neurodiversity. Dorothy Strickland of North Carolina State University’s short paper “Two Case Studies Using Virtual Reality As A Learning Tool For Autistic Children” (Journal of Autism and Developmental Disorders, Vol. 26, No. 6, 1996) is generally accepted as being the first documented use of VR as a tool to increase the capabilities of someone with a disability. In this early study (which you can read at the link above) VR was used as a means to teach the children how to safely cross the street. While VR technology itself has clearly moved on, for the reasons outlined above, its use in this area (up until recently) has not and there is still a great deal about this paper that is relevant today. In particular regarding the children’s acceptance of the headset (which would have been chunkier and more uncomfortable than todays) and their understanding of the 3D world presented by it.
Stepping forward almost a quarter of a century and we are riding the peak of the second wave of commercial VR. Thanks largely to developments made due to the rapid evolution of mobile device in the early years of this decade, VR is becoming more accessible and less disappointing than it was first time around. With the new generation of headsets and their ability to render sharp and detailed 3D environments has come a renewed interest in the use of VR in the area of autism. At a recent CTD Institute webinar on this very subject (Virtual Reality and Assistive Technology) Jaclyn Wickham (@JacWickham), a teacher turned technologist and founder of AcclimateVR outlined some of the reasons why VR could be an appropriate technology to provide training for some people on the autistic spectrum. These included the ability to create a safe and controlled environment where tasks can be practiced and repeated. How the VR experience puts emphases on the visual and auditory senses (with the ability to configure and control both presumably). How you can create an individualised experience and that there are many non-verbal interaction possibilities. Anecdotally this all makes complete sense but we are in the early days and much of the research is still being conducted.
A leading researcher in this area is Dr Nigel Newbutt (@Newbutt) who in June of this year published a short but enlightening update about his progress working with children from Mendip School in the UK. After seeing him present at Doctrid V conference in 2017 I can assure you that progress in this area is being made but even he acknowledges more work is needed. “Our research suggests that head-mounted displays might be a suitable space in which to develop specific interventions and opportunities; to practice some skills people with autism might struggle with in the real world. We’re seeking further funding to address this important question – one that has eluded this field to date.” (Full interview here: From apps to robots and VR: How technology is helping treat autism)
The commercial offerings in the area of VR and Autism (Floreo and AcclimateVR) tend to concentrate on providing a virtual space where basic life skills can be practiced. Another use is as a form of exposure therapy where immersive video and audio of environments and situations are used as a means of preparing someone for the real life experience. You can see examples of both in action at the links above.
Within Enable Ireland AT service our own VR journey was spurred on by a visit and demonstration from James Corbett (@JamesCorbett) of SimVirtua. James could be considered a real pioneer in this area and had in fact met with us previously almost 10 years ago to show us some work he was doing with non-immersive virtual environments (without headsets) in schools. SimVirtua had worked on a Mindfulness VR app called MindMyths and it was this idea of providing a retreat or sanctuary using immersive video that inspired us when it came to working on the Bloom Beyond Boundaries Garden project.
In the second part of this post (coming soon) I’ll give some background to what we hoped to achieve with the Beyond Boundaries garden project and some technical information on how we put it together.
Big news (in the AT world anyway) may have arrived in your mail box early last week. It was announced that leading AAC and Computer Access manufacturer Tobii purchased SmartBox AT (Sensory Software), developers of The Grid 3 and Look2Learn. As well as producing these very popular software titles, SmartBox were also a leading supplier of a range of AAC and Computer Access hardware, including their own GridPad and PowerPad ranges. Basically (in this part of the world at least) they were the two big guns in this area of AT, between them accounting for maybe 90% of the market. An analogy using soft drink companies would be that this is like Coca-Cola buying Pepsi.
Before examining what this takeover (or amalgamation?) means to their customers going forward it is worth looking back at what each company has historically done well. This way we can hopefully provide a more optimistic future for AT users rather than the future offered by what might be considered a potential monopoly.
Sensory Software began life in 2000 from the spare bedroom of founder Paul Hawes. Paul had previously worked for AbilityNet and had 13 years’ experience working in the area of AT. Early software like GridKeys and The Grid had been very well received and the company continued to grow. In 2006 they setup Smartbox to concentrate on complete AAC systems while sister company Sensory Software concentrated on developing software. In 2015 both arms of the company joined back together under the SamrtBox label. By this time their main product, the Grid 3, had established itself as a firm favourite with Speech and Language Therapists (SLT), for the wide range of communication systems it supported and Occupational Therapists and AT Professionals for its versatility in providing alternative input options to Windows and other software. Many companies would have been satisfied with providing the best product on the market however there were a couple of other areas where SmartBox also excelled. They may not have been the first AT software developers to harness the potential resources of their end users (they also may have been, I would need to research that further) but they were certainly the most successful. They succeeded in creating a strong community around the Grid 2 & 3 with a significant proportion of the online grids available to download being user generated. Their training and support was also second to none. Regular high quality training events were offered throughout Ireland and the UK. Whether by email, phone or the chat feature on their website their support was always top quality also. Their staff clearly knew their product inside out, responses were timely and they were always a pleasure to deal with.
Tobii have been around since 2001. The Swedish firm actually started with eyegaze, three entrepreneurs – John Elvesjö, Mårten Skogö and Henrik Eskilsson recognised the potential of eye tracking as an input method for people with disabilities. In 2005 they released the MyTobii P10, the world’s first computer with built-in eye tracking (and I’ve no doubt there are still a few P10 devices still in use). What stood out about the P10 was the build quality of the hardware, it was built like a tank. While Tobii could be fairly criticized for under specifying their all-in-one devices in terms of Processor and Memory, the build quality of their hardware is always top class. Over the years Tobii have grown considerably, acquiring Viking Software AS (2007), Assistive Technology Inc. (2008) and DynaVox Systems LLC (2014). They have grown into a global brand with offices around the world. As mentioned above, Tobii’s main strength is that they make good hardware. In my opinion they make the best eye trackers and have consistently done so for the last 10 years. Their AAC software has also come on considerably since the DynaVox acquisition. While Communicator always seemed to be a pale imitation of the Grid (apologies if I’m being unfair, but certainly true in terms of its versatility and ease of use for computer access) it has steadily being improving. Their newer Snap + Core First AAC software has been a huge success and for users just looking for communication solution would be an attractive option over the more expensive (although much fuller featured) Grid 3. Alongside Snap + Core they have also brought out a “Pathways” companion app. This app is designed to guide parents, care givers and communication partners in best practices for engaging Snap + Core First users. It supports the achievement of communication goals through video examples, lesson plans, interactive goals grid for tracking progress, and a suite of supporting digital and printable materials. A really useful resource which will help to empower parents and prove invaluable to those not lucky enough to have regular input from an SLT.
To sum things up. We had two great companies, both with outstanding products. I have recommended the combination of the Grid software and a Tobii eye tracker more times than I remember. The hope is that Tobii can keep the Grid on track and incorporate the outstanding support and communication that was always an integral part of SmartBox’s operation. With the addition of their hardware expertise and recent research driven progress in the area of AAC, there should be a lot to look forward to in the future.
It is easy for someone to assume that their wheelchair can only be used for driving. However, wheelchair manufacturers have developed their products in recent years and considered the needs of the user such as the need to also interact with their mobile phone, PC or even a TV. As well as the basic chair functions such as driving or controlling the actuators these electronic systems can also enable control of a computer or portable devices and so the integration of environmental controls is possible on most power wheelchairs. The same controls that the user drives the power wheelchair with, typically a joystick, can also be used to control an appliance within their environment. Another benefit of integrating control of other devices within the wheelchair joystick is that it may help to ensure the user maintains a good posture while operating other devices.
For example for chairs with R-net controls you can replace the old joystick with a CJSM2 –BT as seen in the picture here. This R-net Joystick Module has Infra-Red (IR) capabilities included. IR technology is widely used to remotely control household devices such as TVs, DVD players, and multi-media systems, as well as some home-automation equipment. Individual IR commands can be learned from an appliance’s remote handset and stored in the CJSM2. Also Integrated Bluetooth technology is an option, to enable control of computers, Android tablets, iPads, iPhones and other smart devices from a powered wheelchair. To switch between the devices, the user simply navigates the menu and selects the device they wish to control. The R-net’s CJSM2 can easily replace the existing rnet joystick module, with no system re-configuration or programming required.
Although not all power wheelchairs can be fitted with Bluetooth mouse-enabled joysticks, there are some good alternatives that may still work. The BJoy ring is a sensor that can be fitted to most wheelchair joysticks where deflections of the joystick can be translated to mouse movements picked up on a Bluetooth mouse receiver placed on a tablet or PC.
The good: Users can do many daily tasks using one device
The not so good: This capability is only available on high spec wheelchair systems.
The verdict: Using a wheelchair joystick that is Bluetooth enabled will ensure the user maintains a good posture while operating their other devices.
Automatic window openers were design to meet the need of inaccessible windows that are out of reach such as skylights on roofs. Using a wireless remote to activate the window opener we can open and close the window at ease. For people with a mobility restriction window openers can give independence to control ventilation in their living space. The ACK4 Window Opener is suitable for aluminium and timber windows. It has an anti-crush control board to prevent trapped fingers. To control the window, you can either use a wall switch, an infrared remote control or a 433 Mhz radio control transmitter. The window actuator with remote costs about £240.
Using a Broadlink RM Pro could make a window opener become part of your smart home setup and hence then possibly voice controlled.
Motorised blinds and curtains have been around for many years, providing easy access to control blinds and curtain rails. Control of these motorised devices was usually with the use of a radio remote control, which made such devices particularly of interest for people with mobility issues. Internet of Things (IoT) has now extended its internet connectivity to these everyday objects. Embedded with technology, these devices can communicate and interact over the internet, and they can be remotely monitored and controlled.
An example of this is Somfy motorization systems. These systems consist of a range of motorised blinds, curtains and roller shutters. The Somfy myLink™ is a device that turns your smartphone or tablet into a remote control for motorized products featuring Somfy Radio Technology. For voice control, Alexa now works with myLink!
Purchasing new blinds or curtains rails could work out to be quite expensive, and possible wasteful if you already have good blinds in place, however, there are a number of options available to retrofit existing blinds to also consider. They are able to transform your standard home blinds into smart electric blinds and do so at an affordable price. Like the Somfy blinds, they also provide a way to raise, lower and choose an intermediate position of the blind.
Blind Engine
The Brunt Blind Engine can motorize your existing blinds and connect to your smartphone, allowing remote control and scheduling of your blinds anywhere, anytime.
It is designed to be compatible with most roll-type blinds available on the market,
allowing blinds of all different shapes and sizes to be successfully fitted.
The Blind Engine comes with two different gears designed to accommodate string cords and ball chains. With the Brunt App, you can raise and lower multiple blinds at the same time. (No extra monthly charge for the Brunt application)
You can use the Brunt Blind Engine with various voice recognition speakers. Cost online $129
Axis Gear
AXIS Gear is an affordable and easy way to motorize your window shades. Gear is a smart device that lets you easily control and schedule when your shades open and close. Axis say the install and setup of Gear takes minutes and guarantees to fit your shades or your money back.
Included are a solar panel and backup AA batteries. The App allows the creation of schedules and smart home integration.
The SOMA Smart Shades is designed to fit your existing shades and curtains with a continuous-cord. Continuous-cord shades have one looped string or beaded chain that allows you to raise and lower the bottom of the shades. Attach the device to your shades or blinds with a beaded chain or string, download the mobile app, follow the instructions and you’re ready to go. Automated schedules can be created and it is possible to control multiple windows from one mobile app. It is Android & iOS supported. Smart Shades are solar powered with a built-in lithium battery. By installing the SOMA Connect, you can control your shades with your voice, as it works with Amazon Alexa and Apple HomeKit.
Most smart locks are installed on mechanical locks such as deadbolts. They typically upgrade the ordinary lock. Recently, there have been a number of smart locks that have appeared on the market that provide the convenience of being able to lock and unlock your door from anywhere, or passing on to people you trust a passcode to open the door.
Smart locks, like the traditional locks, need two main parts to work: the lock and the key. In the case of these electronic locks, the key is a smartphone or a special key fob configured explicitly for this purpose which wirelessly performs the authentication needed to automatically unlock the door. With some smart locks, the physical key may still be used in case the batteries of the lock have run too low. Generally, the smart door locks operate with 4 AA batteries, so no hard wiring is required.
Most smart locks will feature access by entering a code, instead of fumbling for keys. It is easy to share access with trusted friends/family. They are battery-operated and so should still accept access codes on the touchscreen during home power outages. They all claim to be easy to install with just a screwdriver (providing holes for the lock are already installed in the door).
Smart Lock manufacturers generally have their own app to set up and control the door locks but can often be controlled by other smartphone apps such as Samsung Smartthings or will work with Apple HomeKit, which is software on Apple iOS devices that lets users configure and control smart-home appliances.
Some locks you can use hands-free voice control with Amazon Alexa-enabled devices or Google Assistant-enabled devices. This includes verbally locking or checking the status of the front door. This may require the addition of a wifi adaptor for the lock.
Smart locks can be used with a smart doorbell to allow the user to see or communicate with someone at a door before unlocking. These are now mainstream products, available in computer and online stores. These types of products can help support independent living for people with disabilities.
Below is a range of the newer smart door locks.
Nest x Yale Lock
With Yale known for their locks and Nest known for their connected home, they have come together to make a key‑free deadbolt that connects to the Nest app. As with many smart locks you can lock and unlock your door from anywhere or give people you trust a passcode, instead of a key.
When the Nest/Yale Lock is connected to the Nest app, you can unlock your door from your phone or create passcodes for family and guests. Even set times when passcodes expire. You can get alerts whenever someone unlocks and locks the door. And when Nest knows you’re away, your door can lock automatically.
Yale Real Living® Touchscreen Deadbolt
The absence of the cylinder provides a “clean” appearance and means that lock picking will be difficult. It eliminates the need to manage keys for your door.
In the event the batteries die – a 9V battery provides enough power to enter the code and gain access to the lock.
The touchscreen keypad illuminates for night time access.
It is available in ZigBee® or Z-Wave® configurations
As with all the smart locks, the Schlage Sense Smart Deadbolt claims to be easily installed with just a screwdriver. It has a pre-set, unique 6 digit programming code, and a capacity for 30 access code (4-8 digits)
What makes it different from the other smart locks is that it features a built-in alarm to sense potential door attacks.
It works with Apple HomeKit. With the Schlage Sense Wi-Fi Adapter plugged into an outlet within the home and connected to the home Wi-Fi, you can lock/unlock from anywhere using an iPhone or Android smartphone.
You can use your lock hands-free, through voice control with Amazon Alexa-enabled devices like Amazon Echo and Dot if you pair your Schlage Sense WiFi Adapter with your Schlage Sense lock. Learn more about Alexa features here and similar to Alexa, you’ll be able to use hands-free voice control with Google Assistant-enabled devices like Google Home.
This electronic lock features a battery-operated keypad or touchscreen, eliminating the need for keys. Enter your home with a unique access code or lock your door with the touch of a button.
Kwikset’s SmartCode electronic locks offer a touchscreen and touchpad for a keyless entry that fits on any standard door. Z-wave and ZigBee options available. The touchpad with buttons may have the advantage that it consists of raised buttons for someone with a visual impairment. The back-lit keypad provides increased visibility.
The SmartCode touchpad smart lock with Home Connect technology enables the lock to wirelessly communicate with other devices in the home. The lock allows the user (through a third-party smart home controller) to remotely check the door lock status, lock or unlock the door and receive notifications via email or text. Kwikset say SmartCode is easy to install, program and use. It operates on 4 AA batteries. It also features SmartKey Security as the back-up keyway.
Video doorbells bring both convenience and security to your home by streaming a live view of the doorstep to your smartphone, whether you are on the other side of the door or the other side of the world. For someone with a disability, these could be quite useful products as they let you check who is there before answering. Your smartphone is notified the moment motion is detected or the doorbell is pressed. You can speak to visitors through the doorbell’s microphone and speaker.
Used in combination with a smart lock they could replace older technologies such as a video door intercom for door entry. There are now a number of smart video doorbells products available, as can be seen below.
Ring video doorbell
Nest video doorbell
Blink Video Doorbell
The good: You wont miss another call on the door whether you are on the other side of the door, or the other side of the world.
The not so good: If your home Wi-Fi stops working so does your doorbell.
The verdict: Promotes living independently for people with disabilities by providing a secure door entry system.
One of the dangers for people with disabilities living independently are the risks associated with cooking which can result in a fire. However, there are a number of devices we can use to reduce or even eliminate this danger in cooking. These devices can promote independent living when using the cooker is risky due to old age, memory disorders, disability, or learning difficulties. These devices aim to protect the area of a home most at risk of catching fire, the kitchen.
lnnohome Stove Guard
The lnnohome Stove Guard is a cooker safety device that monitors the hob use and registers when the user is not present. If the cooker has been left on the Timer turns it off. The Stove Guard will also identify a dangerously high temperature or steep temperature rise, and recognizes the alarm signals of fire, gas and carbon monoxide alarms. An Automatic Safety Lock ‘locks’ the cooker so that it cannot be turned on accidentally.
Stove Guard SGK510
Video
lnnohome Stove Alarm
The Stove Alarm is a more affordable solution than the Stove Guard that will improve the kitchen’s fire safety significantly. The Intelligent Heat Sensor, attached underneath the cooker hood, signals an alarm, that alerts the user to a hazardous situation happening on the cooker before it produces toxic gases or starts a fire. The alarm also sounds if it is removed from its location.
