Skyle – out of the blue

Anybody working with Assistive Technology (AT) knows how useful Apple iOS devices are. Over the years they have gradually built in a comprehensive and well-designed range of AT supports that go a long way to accommodating every access need. This is no small feat. In 2009 VoiceOver transformed what was essentially a smooth featureless square of glass with almost no tactile information, into the preferred computing device for blind people. In 2019 Voice Control and the improvements made to Assistive Touch filled two of the last big gaps in the area of “hands free” control of iOS. All this great work is not completely altruistic however as it has resulted in Apple mobile devices cementing their place as the preeminent platform in the area of disability and AT. It is because of this that it has always been somewhat of a mystery why there has never been a commercial eye tracking option available for either iOS or MacOS. Perhaps not so much iOS as we will see but certainly one would have thought an eyegaze solution for the Apple desktop OS could be a viable product.

There are a few technical reasons why iOS never has supported eyegaze. Firstly, up until the newer generations of eye gaze peripherals, eye gaze needed a computer with a decent spec to work well. iPads are Mobile devices and Apple originally made no apologies for sacrificing performance for more important mobile features like reducing weight, thickness and increasing battery life. As eye trackers evolved and got more sophisticated, they began to process more of the massive amount of gaze data they take in. So rather than passing large amounts of raw data straight through to the computer via USB 3 or Firewire they process the data first themselves. This means less work for the computer and connection with less bandwidth can be used. Therefore, in theory, an iPad Pro could support something like a Tobii PC Eye Mini but in practice, there was still one major barrier. iOS did not support any pointing device, let alone eye tracking devices. That was until last September’s iOS update. iOS 13 or iPadOS saw upgrades to the Assistive Touch accessibility feature that allowed it to support access to the operating system using a pointing device.     

iPad Pro 12" in black case with Skyle eye tracker
iPad Pro 12″ with Skyle eye tracker and case

It is through Assistive Touch that the recently announced Skyle for iPad Pro is possible. “Skyle is the world’s first eye tracker for iPad Pro” recently announced by German company EyeV https://eyev.de/ (who I admit I have not previously heard of). Last week it appeared as a product on Inclusive Technology for £2000 (ex VAT). There is very little information on the manufacturer website about Skyle so at this stage all we know is based on the Inclusive Technology product description (which is pretty good thankfully). The lack of information about this product (other than the aforementioned) significantly tempers my initial excitement on hearing that there is finally an eye tracking solution for iOS. There are no videos on YouTube (or Inclusive Technology), no user reviews anywhere. I understand it is a new product but it is odd for a product to be on the market before anybody has had the opportunity of using it and posting a review. I hope I am wrong but alarm bells are ringing. We’ve waited 10 years for eye tracking on iOS, why rush now?

Leaving my suspicion behind there are some details on Inclusive Technology which will be of interest to potential customers. If you have used a pointing device through Assistive Touch on iPadOS you will have a good idea of the user experience. Under Cursor in the Assistive Touch settings you can change the size and colour of the mouse cursor. You will need to use the Dwell feature to automate clicks and the Assistive Touch menu will hive you access to all the other gestures needed to operate the iPad. Anyone who works with people who use eye tracking for computer access will know that accuracy varies significantly from person to person. Designed for touch, targets in iPadOS (icons, menus) are not tiny, they are however smaller than a cell in the most detailed Grid used by a highly accurate eyegaze user. Unlike a Windows based eye gaze solution there are no additional supports, for example a Grid overlay or zooming to help users with small targets. Although many users will not have the accuracy to control the iPad with this device (switch between apps, change settings) it could be a good solution within an AAC app (where cell sizes can be configured to suit user accuracy) or a way of interacting with one of the many cause and effect apps and games. Again however, if you have a particular app or activity in mind please don’t assume it will work, try before you buy. It should be noted here that Inclusive Technology are offering a 28 Day returns policy on this product.

There is a Switch input jack which will offer an alternative to Dwell for clicking or could be set to another action (show Assistive Touch menu maybe). I assume you could also use the switch with iOS Switch Control which might be a work around for those who are not accurate enough to access smaller targets with the eye gaze device. It supports 5 and 9 point calibration to improve accuracy. I would like to see a 2 point calibration option as 5 points can be a stretch for some early eyegaze users. It would also be nice if you could change the standard calibration dot to something more likely to engage a child (cartoon dog perhaps).

Technical specs are difficult to compare between eye trackers on the same platform (Tobii v EyeTech for example) so I’m not sure what value it would be to compare this device with other Windows based eye trackers. That said some specs that will give us an indication of who this device may be appropriate for are sample rate and operating distance. Judging by the sample rate (given as 18Hz max 30Hz) the Skyle captures less than half the frames per second of its two main Windows based competitors (Tobii 30 FPS TM5 42 FPS). However even 15 FPS should be more than enough for accurate mouse control. The operating distance (how far the device is from the user) for Skyle is 55 to 65 cm which is about average for an eyegaze device. However only offering a range of 10 cm (Tobii range is 45cm to 85 cm, so 40 cm) as well as the photo below which shows the positioning guide both indicate that this not a solution for someone with even a moderate amount of head movement as the track box (area where eyes can be successfully tracked) seems to be very small.

the positioning guide in the skyle app. letterbox view of a persons eyes. seems to indicate only movement of a couple of centimeters is possible before going out of view.
Does the user have to keep their position within this narrow area or does Skyle use facial recognition to adjust to the user’s position? If it’s the former this solution will not be appropriate for users with even a moderate amount of head movement.

In summary if you are a highly accurate eyegaze user with good head control and you don’t wear glasses.. Skyle could offer you efficient and direct hands free access to your iPad Pro. It seems expensive at €2500 especially if you don’t already own a compatible iPad (add at least another €1000 for an iPad Pro 12”). If you have been waiting for an eyegaze solution for iOS (as I know many people have) I would encourage you to wait a little longer. When the opportunity arises, try Skyle for yourself. By that time, there may be other options available.

If any of the assumptions made here are incorrect or if there is anymore information available on Skyle please let us know and we will update this post.

Introducing a Joystick

In recent months I have been working closely with a young client to increase her independence during school and play activities. As part of this process, we trialled the Optima and Point It joysticks through the Enable Ireland loan library.

The Optima joystick is the cheaper of the two products, retailing from £175. A newer version, the n-Abler joystick, is also available and has more features and functions, but the selling point of the Optima seems to be its simplicity and therefore suitability for younger users or those with intellectual disability. The large platform of the Optima does provide stability and a natural resting place for the hand. The user was attracted to the clean, clear design and felt comfortable with it straight away. It did not take long to teach the functions of the three large coloured buttons – left-clicking, right-clicking and speed adjustment. It was helpful that the buttons were recessed into the panel as this made it unlikely they would be pressed by accident.

Optima joystick

The Optima is supplied by Pretorian Technologies who were very helpful in answering my queries and also recommended two newer products, the Slimline Joystick for smaller hands and the Ultra joystick, a more compact product for use with the head or chin. Unfortunately as they did not have a distributor in Ireland at the time, we were unable to trial these products. More information is available on their website:  https://www.pretorianuk.com/joysticks. A feature I liked is that these joysticks all come with a variety of interchangeable knobs included, rather than having to purchase these separately.

The Point It joystick, supplied by Housemate, is a smaller model which is easy to fit on a small space next to a laptop or keyboard. It has four buttons around the platform which cover left and right-clicking, double-clicking and speed adjustment. It is a more expensive product with prices starting from €476. Initially this user preferred the Optima joystick as the buttons were larger and she found them easier to use. However what made the biggest difference with the Point It joystick, in this case, was the switch on top of the standard knob. Once she discovered she could easily click and drag items using her thumb, without having to release and readjust her grip, there was no comparison.  This feature was much more intuitive and less effortful than recalling which button was needed each time and having to shift her hand position forward and back.

There are mounting plates available to secure and position the Point It joystick but in this case we found that some Dycem and Velcro did the trick! There is a ball handle version available if preferred and a variety of different knobs that can be purchased for use interchangeably with this version. There is also a Bluetooth version of the Point It joystick available which means it can be used wirelessly with a variety of devices including tablets, smart TVs and other home controls. Finally, there is a Mini Point-It which is even smaller and designed specifically for chin users and others who may need a compact joystick mounted in unorthodox positions. More information is available on the Housemate website (http://housemate.ie/point-it/) or from Edtech who are named suppliers of the product in Ireland (https://www.edtech.ie/).

Starting out, the user explored the joystick using simple games online, anything where she could scan and click to access a sound or image. For example, pressing the play button to activate a favourite video on YouTube!  It was helpful to use some of the standard Microsoft accessibility features in the early days, such as enlarging the pointer. It was also worthwhile slowing down the speed of the cursor, which could be done either through the Microsoft access features or directly through the joystick controls. Now that the client has sourced a Point-It joystick of her own, she has begun using it to access an onscreen keyboard, play games and read e-books, and to continue drawing and colouring through Microsoft Paint. It also provided valuable preparation in developing joystick skills for powered mobility. In this case, the switch feature on the Point It joystick made all the difference to the user and has opened up a world of opportunities for leisure and learning. Given the development of other Point It products by Housemate, who specialise in environmental control, I anticipate that this will be a helpful product for other uses in future. However, I would not hesitate to explore the Optima again with other users looking for a first-time easy-to-use joystick.

Supporting the “Click”

I recently came across two “new” Windows programs when exploring ideas for adding extra control to a roller mouse. The search had started while we’d be looking to adapt an employee’s workplace. In her case, she had a simple request to continue to use her roller mouse – liking the feel and its movement – but she found the “lift” from the roller to click more of a challenge to carry out – particularly over long periods of use.

MetaClick toolbar to automate the mouse click
MetaClick toolbar

The first challenge was a quick one to resolve as there’s a number of well-established supports available to automate the mouse click. Programs such as Point N Click Virtual Mouse and Dwell Clicker 2 are well known and a little more polished. However, in the end, we settled on a program with a small presence on the screen called MetaClick.

The good: It’s free, easy to set up and control the “look”, straight forward layout.

Areas for improvement: Appearance looks dated, lacks the ability to move on-screen buttons to preferred areas.

Sakasa Mouse Settings menu
Sakasa Mouse Settings menu

The second challenge, in this case, related to the “orientation” of the roller mouse. The User found her control to be improved over longer periods when her mouse roller was rotated by 1800 on her countertop. This one was a little trickier to resolve but in the end, an old “prank” program called Sakasa Mouse – software which reverses the direction of the cursor movement – came to the rescue for her.

The good: It’s free, separate settings can be set for the X and the Y mouse planes, simple configuration.

Areas for improvement: No support, hasn’t been updated for several years, “jumpy” mouse movement evident on occasions.

My Computer My Way: Find how to make your device easier to use

Logo for My Computer My Way

My Computer My Way is a free online guide of accessibility features for computers, tablets and mobile phones. The aim is to provide you details to make whatever device you’re using easier to use via built-in accessibility features, browser extensions or via apps that you can install.

It’s been around for quite a number of years and having revisited the site again recently I am glad to see it has been updated to current operating systems features.  So whether you need help now with Android Pie, Windows 10 or iOS12 this useful guide has been updated to include the new built-in accessibility features. 

The layout of Accessibility features is divided into four categories

  • Vision; options include features to help you see and use applications more clearly
  • Hearing; accessibility features and information for people who are deaf or hard of hearing.
  • Motor; ways to make your keyboard, mouse and mobile device easier to use.
  • And cognitive; computer adjustments that will make reading writing and using the internet easier.

Further information

https://mcmw.abilitynet.org.uk/

The good:  provides details on just about every build-in accessibility feature for your device.

The not so good: There is a limited amount of information on apps or applications that might also provide useful features.

The verdict:  A useful tool for individuals who have limited or no access to an assistive technology service and need help to find solutions on their own.

Webcam Face trackers

User at a laptop using a webcam face tracker

Webcam Face trackers allow full control of mouse functions without the use of hands. They can be used to access a computer (Windows, Mac), as well as a tablet or smartphone (Android only at present).

Primary users of these technologies are people with motor impairments.  There are various options for hands-free control of your mouse on a computer screen such as reflective dot trackers, lip and chin joysticks, speech recognition or even eye trackers.  Webcam Face trackers are another possible option for hands-free control of your computer or phone. 

Although it may not be as accurate as other hands-free options, such as wearable sensors, with this approach, you don’t have to wear a sensor or reflective dot.  As you move your head, the motion is translated to mouse cursor movement by the webcam.  However, you do have to maintain a direct line-of-sight to the computer, and the performance is dependent on lighting conditions.

Basic pointing device support on an Android tablet or phone is possible with EVA Facial Mouse.  This is available through Google Play.  It will allow access to functions of the mobile device by means of tracking the user’s face, captured through the frontal camera. 

At the time of writing, a webcam face tracker is not available on iOS devices.  However, it is possible to use Switch Control with head gestures to act as switches.  For example look left for select, look right for home.

All 5 Webcam Face Trackers listed below have options for mouse dwell, click and drag lock.

There are two free windows webcam face trackers – Camera mouse and Enable Viacam.  Both work quite well.  For the paid options, SmyleMouse also tracks facial expressions and has the option of clicking with a smile.  ViVo offers integration with leading speech recognition programs.

As there are trial versions for most of these options below, its best to try them all to really get a feel for it and see which one works best for you.

Wearable hands-free mice options to consider are:

SmyleMouse $499


ViVo Mouse $430


Camera Mouse free


Enable Viacam free

iTracker for Mac $35

The good:  You don’t have to wear a sensor or reflective dot and they are battery-free.

The not so good: They are not as accurate as other methods of hands-free options.

The verdict:  If you don’t need very fine cursor control and don’t want to wear a sensor on your head, then webcam face trackers are a good option for hands-free control.

Wearable hands-free mice

Wearable hands-free mice allow full control of mouse functions without the use of hands. They can be used to access a computer (Windows, Mac, etc.), as well as a tablet or smartphone (Android, iOS)

Primary users of these technologies are or cervical spinal cord injury.

There are various options for hands-free control of your mouse on a computer screen such as reflective dot trackers, lip and chin joysticks, speech recognition or even eye trackers.  One other possible group of devices are wearable hands-free mice.  With this approach, you wear a sensor (usually on your head but can be worn elsewhere if that works better for you) and as you move, the motion of that sensor controls the mouse cursor.

There is no camera or other optical unit involved, so you do not have to maintain a direct line-of-sight to the computer, and the performance is independent of lighting conditions.

The GlassOuse and the Zono are wireless, requiring no physical connection between the sensor unit that you wear and the computer that you are controlling. They both have perhaps the most thorough and refined designs in this family.

 The GlassOuse package is worn like eyeglasses (but without anything in front of the eye).  It weighs about 50g. GlassOuse also supplies a range of switches that can be used to perform the mouse click such as bite, puff or a proximity switch. 

The Zono is more of a headphone-style mount for its sensor, and also has several alternative ways to wear the sensor, such as an eyeglass clip.  The Zono can be used with a breath or puff switch so you can click by blowing lightly on the switch sensor.

The EnPathia and eeZee sensors require that the mouse must be tilted, not rotated, to move the cursor. So the motion used will be quite different in the head-controlled case; to move right, you would tilt your right ear toward your right shoulder, instead of rotating your head to the right. This is a less intuitive and more difficult movement for many people.  Finally, an open-source option is the Headmouse by Millmore with build instructions available on instructables.com

Some wearable hands-free mice options to consider are

GlassOuse V1.2 €499

GlassOuse V1.2 mouse with bite switch
User with GlassOuse V1.2 mouse with bite switch

Quha Zono £550

EnPathia €227

EnPathia mouse worn on the users head
EnPathia mouse worn on the users head

eeZee Switch  $599

eeZee Switch on frame of glasses
eeZee Switch on frame of glasses

ED Air Mouse

ED Air Mouse with switches
ED Air Mouse with switches

Head Mouse by Millmore <€50

Millmore testing his Head Mouse
Millmore testing his Head Mouse

Video of user using a wearable mouse

The good:  These hands-free options can potentially have precise control and are not affected by lighting or sound.

The not so good: Commercial options are expensive.

The verdict: If you need or want the ability to make very fine cursor control, and you are happy to wear a sensor, then these wearable mice are a good option for hands-free control.

Handsfree Lip and chin Joysticks

Person using a BJOY chin joystick to control a computer
User using the BJOY chin joystick

A hands-free mouse allows you to perform computer mouse functions without using your hands. There are various options for hands free control of your mouse on a computer screen such as reflective dot trackers, wearable sensors, speech recognition or even eye trackers.  One other possible group of devices are Lip and chin Joysticks. 

These products are designed specifically for users with physical disabilities. They are typically USB Plug and Play, which means they will work with any computer platform that supports USB mice, including Windows, Mac OS X, Linux, and Android. All can be customized using the built-in mouse settings in the operating system, while some will also include setup software for further customization.

To activate the mouse buttons. The IntegraMouse+, Jouse3, and QuadJoy incorporate a sip/puff switch into their joystick, so that a sip action clicks one mouse button, and a puff action clicks the other. Other options are switches, the BJOY Chin has two circular switch pads, one on either side of the joystick, which can be pressed using the chin or cheek. And the TetraMouse has a second joystick that is devoted to button actions, right next to the joystick for cursor control. Low cost options are the Tobias’ mouse and the Flipmouse.  This are open source hardware and software projects with documented instruction on how to build and 3D Print.  The user moves the cursor by using a mouthpiece. The right mouse button is operated by pushing the mouthpiece towards the case. The left mouse button is emulated by a sensor that recognizes if the user sucks air through it.

Some Lip and chin Joysticks options to consider are

IntegraMouse+  €2000

Person using a IntegraMouse for mouse control

Jouse3 $1,495

Person using a Jouce3 for mouse control

QuadJoy 3 $1,398.60

Person using a QuadJoy 3 for mouse control

BJOY Chin €445

TetraMouse XA2 $449

TetraMouse XA2 for mouse control

Tobias’ mouse  <€50 for parts

Tobias’ mouse low cost opensource mouse

FLipMouse €179

FLipMouse low cost opensource mouse

Video

  • The good:  These hand free options can potentially have precise control and are not effected by lighting or sound.

The not so good: do require a line-of-sight to the computer, and commercial options are expensive.

The verdict:  If you need or want the ability to make very fine cursor control, and don’t want to wear a sensor or reflective dot then these joysticks are a good option for hands free control.