iPad, Meet Apple TV

6 November 2018

Mice and trackpads are second nature to most people, even more so than touchscreens. Nowadays, we expect every display around us to respond to a tap of the finger, but we get genuinely disconcerted when there isn’t a cursor where we think a cursor should be. This is one of the reasons the iPad, a touch-and-keyboard-only device, has faced opposition as the new form of all-purpose computer. Many people just can’t get past the idea of using a laptop without a mouse or trackpad.

Some think Apple is already solving this problem. Ben Bajarin says the Apple Pencil is the iPad’s mouse killer:

I’d argue that what can be enabled by pencil, gestures, and software, will take precision input to a level not possible with a trackpad/mouse.

But this is only true for specific use cases such as drawing and handwriting. Try using an Apple Pencil to select text or move the insertion point while typing on the Smart Keyboard Folio. It just doesn’t work. What makes a mouse or trackpad so powerful is that it’s on the same flat surface as the keyboard — the user doesn’t have to reach up off that two-dimensional plane. The Apple Pencil cannot and never will match that kind of convenience.

There’s a good reason why a traditional mouse would be terrible on an iPad. iOS’s touch targets are huge, and selecting one of them with a cursor would just be awkward. This is why Marzipan apps in macOS Mojave — take News for example — look so weird: a mouse is designed for more densely organized information. Put simply, iOS is more spread out.

This is why Uluroo had always thought trackpads on iOS would make no sense. But why assume that the iPad would get a normal cursor? What if the iPad’s mouse acted like the Apple TV remote?

Halfway through writing this article, Uluroo finally figured out where he had seen this idea brought up before. John Gruber suggested this eighteen months ago, based on a tidbit Steve Troughton-Smith found in iOS 9 back in 2015. Apple hasn’t done anything since then, so Uluroo is just going to make the suggestion a second time.

If you recall, tvOS does not have a cursor, per se. Interface elements expand slightly to indicate that they are selected. The user swipes on a trackpad-like surface, and tvOS selects the next item in the direction that was swiped. There is no freeform cursor; the “focus point,” if you will, only moves between designated elements.

Imagine if the iPad behaved that way. The cursor wouldn’t have to move through a bunch of dead space to get from one interface element to another; rather, it would simply jump from target to target. This system would have the convenience of a trackpad that remains on the same plane as the keyboard, but it would get around the annoyances that come with a traditional cursor being jammed into a touch interface.

In case none of that makes sense, here’s how it would work in some common scenarios.

Opening an app from the Home screen. Swipe up, down, left, or right on the trackpad to navigate between app icons. If an icon is selected, it looks slightly bigger than the others. Click the trackpad to open the app that is selected. (Maybe a trackpad would even support 3D Touch.) Picture the way this looks on the tvOS Home screen, just with iOS’s icon shapes.

Navigating an app like Mail. Swipe left on the trackpad to move into the column that contains all messages. From there, swipe up and down to choose a new message. The message that is selected looks bigger than the others — this could have a kind of three-dimensional effect, as though the message is hovering slightly. Click on a message to enter it.

Selecting text or moving the insertion point in any app. Use the trackpad as you would on a laptop; instead of a traditional arrow-shaped cursor, it will just be the vertical line that moves around. This is one of the only places where freeform cursor-ing would work.

Scrolling. Just swipe up and down with two fingers.

Dock, Control Center, and Cover Sheet. For the Dock, swipe up with two fingers from the bottom edge of the trackpad surface. For Control Center, swipe down with two fingers from the upper-right corner of the trackpad. For Cover Sheet, swipe down with two fingers from anywhere else on the top edge of the trackpad.

Split View and Slide Over. After the Dock is summoned, use the trackpad to drag an app into place on the screen. After dismissing a Slide Over app, swipe with two fingers from the right edge of the screen to restore it. To change which Split View app has focus, swipe sideways with two fingers.

As you can see, navigation would be very similar to the way it is on tvOS, with a few modifications for the iPad’s multitasking features. If Apple offered anything remotely like this, Uluroo would take it in a heartbeat. It would be immensely more convenient to be able to control iOS with some kind of pointing device, even one that’s more limited than macOS’s cursor.

Are there any glaring problems with this idea? Is it still a regression from a future that appears based around touch? Uluroo doesn’t think so, but please let him know.