Tags: ipad

Apple previews Live Speech, Personal Voice, and more new accessibility features - Apple

This is the kind of press release I like.

I documented a weird bug with web audio on iOS a while back:

On some pages of The Session, as well as the audio player for tunes (using the Web Audio API) there are also embedded YouTube videos (using the video element). Press play on the audio player; no sound. Press play on the YouTube video; you get sound. Now go back to the audio player and suddenly you do get sound!

It’s almost like playing a video or audio element “kicks” the browser into realising it should be playing the sound from the Web Audio API too.

This was happening on iOS devices set to mute, but I was also getting reports of it happening on devices with the sound on. But it’s that annoyingly intermittent kind of bug that’s really hard to reproduce consistently. Sometimes the sound doesn’t play. Sometimes it does.

I found a workaround but it was really hacky. By playing a one-second long silent mp3 file using audio, you could “kick” the sound into behaving. Then you can use the Web Audio API and it would play consistently.

Well, that’s all changed with the latest release of Mobile Safari. Now what happens is that the Web Audio stuff plays …for one second. And then stops.

I removed the hacky workaround and the Web Audio API started behaving itself again …but your device can’t be set to silent.

The good news is that the Web Audio behaviour seems to be consistent now. It only plays if the device isn’t muted. This restriction doesn’t apply to video and audio elements; they will still play even if your device is set to silent.

This descrepancy between the two different ways of playing audio is kind of odd, but at least now the Web Audio behaviour is predictable.

You can hear the Web Audio API in action by going to any tune on The Session and pressing the “play audio” button.

Enable/unmute WebAudio on iOS, even while mute switch is on

Remember when I wrote about Web Audio weirdness on iOS? Well, this is a nice little library that wraps up the same hacky solution that I ended up using.

It’s always gratifying when something you do—especially something that feels so hacky—turns out to be independently invented elsewhere.

I told you about how I’m using the Web Audio API on The Session to generate synthesised audio of each tune setting. I also said:

Except for some weirdness on iOS that I had to fix.

Here’s that weirdness…

Let me start by saying that this isn’t anything to do with requiring a user interaction (the Web Audio API insists on some kind of user interaction to prevent developers from having auto-playing sound on websites). All of my code related to the Web Audio API is inside a click event handler. This is a different kind of weirdness.

First of all, I noticed that if you pressed play on the audio player when your iOS device is on mute, then you don’t hear any audio. Seems logical, right? Except if using the same device, still set to mute, you press play on a video or audio element, the sound plays just fine. You can confirm this by going to Huffduffer and pressing play on any of the audio elements there, even when your iOS device is set on mute.

So it seems that iOS has different criteria for the Web Audio API than it does for audio or video. Except it isn’t quite that straightforward.

On some pages of The Session, as well as the audio player for tunes (using the Web Audio API) there are also embedded YouTube videos (using the video element). Press play on the audio player; no sound. Press play on the YouTube video; you get sound. Now go back to the audio player and suddenly you do get sound!

It’s almost like playing a video or audio element “kicks” the browser into realising it should be playing the sound from the Web Audio API too.

This was happening on iOS devices set to mute, but I was also getting reports of it happening on devices with the sound on. But it’s that annoyingly intermittent kind of bug that’s really hard to reproduce consistently. Sometimes the sound doesn’t play. Sometimes it does.

Following my theory that the browser needs a “kick” to get into the right frame of mind for the Web Audio API, I resorted to a messy little hack.

In the event handler for the audio player, I generate the “kick” by playing a second of silence using the JavaScript equivalent of the audio element:

var audio = new Audio('1-second-of-silence.mp3');
audio.play();

I’m not proud of that. It’s so hacky that I’ve even wrapped the code in some user-agent sniffing on the server, and I never do user-agent sniffing!

Still, if you ever find yourself getting weird but inconsistent behaviour on iOS using the Web Audio API, this nasty little hack could help.

Update: Time to remove this workaround. Mobile Safari has been updated.

I started getting some emails recently from people having issues using The Session. The issues sounded similar—an interactive component that wasn’t, well …interacting.

When I asked what device or browser they were using, the answer came back the same: Safari on iPad. But not a new iPad. These were older iPads running older operating systems.

Now, remember, even if I wanted to recommend that they use a different browser, that’s not an option:

Safari is the only browser on iOS devices.

I don’t mean it’s the only browser that ships with iOS devices. I mean it’s the only browser that can be installed on iOS devices.

You can install something called Chrome. You can install something called Firefox. Those aren’t different web browsers. Under the hood they’re using Safari’s rendering engine. They have to.

It gets worse. Not only is there no choice when it comes to rendering engines on iOS, but the rendering engine is also tied to the operating system.

If you’re on an old Apple laptop, you can at least install an up-to-date version of Firefox or Chrome. But you can’t install an up-to-date version of Safari. An up-to-date version of Safari requires an up-to-date version of the operating system.

It’s the same on iOS devices—you can’t install a newer version of Safari without installing a newer version of iOS. But unlike the laptop scenario, you can’t install any version of Firefox of Chrome.

It’s disgraceful.

It’s particularly frustrating when an older device can’t upgrade its operating system. Upgrades for Operating system generally have some hardware requirements. If your device doesn’t meet those requirements, you can’t upgrade your operating system. That wouldn’t matter so much except for the Safari issue. Without an upgraded operating system, your web browsing experience stagnates unnecessarily.

For want of a nail…

• A website feature isn’t working so
• you need to upgrade your browser which means
• you need to upgrade your operating sytem but
• you can’t upgrade your operating system so
• you need to buy a new device.

Apple doesn’t allow other browsers to be installed on iOS devices so people have to buy new devices if they want to use the web. Handy for Apple. Bad for users. Really bad for the planet.

It’s particularly galling when it comes to iPads. Those are exactly the kind of casual-use devices that shouldn’t need to be caught in the wasteful cycle of being used for a while before getting thrown away. I mean, I get why you might want to have a relatively modern phone—a device that’s constantly with you that you use all the time—but an iPad is the perfect device to just have lying around. You shouldn’t feel pressured to have the latest model if the older version still does the job:

An older tablet makes a great tableside companion in your living room, an effective e-book reader, or a light-duty device for reading mail or checking your favorite websites.

Hang on, though. There’s another angle to this. Why should a website demand an up-to-date browser? If the website has been built using the tried and tested approach of progressive enhancement, then everyone should be able to achieve their goals regardless of what browser or device or operating system they’re using.

On The Session, I’m using progressive enhancement and feature detection everywhere I can. If, for example, I’ve got some JavaScript that’s going to use querySelectorAll and addEventListener, I’ll first test that those methods are available.

if (!document.querySelectorAll || !window.addEventListener) {
  // doesn't cut the mustard.
  return;
}

I try not to assume that anything is supported. So why was I getting emails from people with older iPads describing an interaction that wasn’t working? A JavaScript error was being thrown somewhere and—because of JavaScript’s brittle error-handling—that was causing all the subsequent JavaScript to fail.

I tracked the problem down to a function that was using some DOM methods—matches and closest—as well as the relatively recent JavaScript forEach method. But I had polyfills in place for all of those. Here’s the polyfill I’m using for matches and closest. And here’s the polyfill I’m using for forEach.

Then I spotted the problem. I was using forEach to loop through the results of querySelectorAll. But the polyfill works on arrays. Technically, the output of querySelectorAll isn’t an array. It looks like an array, it quacks like an array, but it’s actually a node list.

So I added this polyfill from Chris Ferdinandi.

That did the trick. I checked with the people with those older iPads and everything is now working just fine.

For the record, here’s the small collection of polyfills I’m using. Polyfills are supposed to be temporary. At some stage, as everyone upgrades their browsers, I should be able to remove them. But as long as some people are stuck with using an older browser, I have to keep those polyfills around.

I wish that Apple would allow other rendering engines to be installed on iOS devices. But if that’s a hell-freezing-over prospect, I wish that Safari updates weren’t tied to operating system updates.

Apple may argue that their browser rendering engine and their operating system are deeply intertwingled. That line of defence worked out great for Microsoft in the ‘90s.

Robin Rendle ･ A Rocket-Powered Jumbo Jet

Before the hagiographical praise for working with an iPad Pro, Robin nails the fundamental shape of the design process:

I had forgotten that there are two modes of design, just as there is in writing.

The first mode is understanding the problem, getting a ten-thousand foot view of the land. It’s getting people to acknowledge that this really is the problem we need to agree upon. This work needs to happen in a sketchbook in the form of messy, back-of-the-napkin drawings or in writing. All this helps you to form a proper argument and focus your thoughts.

The second mode of design is taking that ten-thousand foot view and zooming all the way in to the hairs on the back of the rabbit; figuring out the precise UI and components, the copywriting, the animations, the everything else. This should be done in a design tool like Figma or Sketch. And this is when we should be talking about color palettes, icons, design systems, and consistency.

The problem with almost all design work is that first phase never really happens. People don’t take that ten thousand foot view of the problem and are focusing instead on the pixels; they’re trapped by the system they know too well.

Yes, yes, yes! Spot on:

I think people get stuck in that second mode because productivity in design is often tied to “how many pages or frames did I design today?” when productivity should instead be thought of as “how did my understanding of the problem change?

The ‘Future Book’ Is Here, but It’s Not What We Expected | WIRED

Craig writes about reading and publishing, from the memex and the dynabook to the Kindle, the iPhone, and the iPad, all the way back around to plain ol’ email and good old-fashioned physical books.

We were looking for the Future Book in the wrong place. It’s not the form, necessarily, that needed to evolve—I think we can agree that, in an age of infinite distraction, one of the strongest assets of a “book” as a book is its singular, sustained, distraction-free, blissfully immutable voice. Instead, technology changed everything that enables a book, fomenting a quiet revolution. Funding, printing, fulfillment, community-building—everything leading up to and supporting a book has shifted meaningfully, even if the containers haven’t. Perhaps the form and interactivity of what we consider a “standard book” will change in the future, as screens become as cheap and durable as paper. But the books made today, held in our hands, digital or print, are Future Books, unfuturistic and inert may they seem.

Apple launched the iPad five years ago. A few months after its release, I bought one of those first-generation iPads. I used it for a while before concluding that, much as I had suspected, it wasn’t the right device for me. But it was the perfect device for my mother. So I gave the iPad to my mother.

Sure enough, it worked out great for her. But now it’s getting quite long in the tooth. So now I’ve given her a new iPad for her birthday; one that’s lighter, faster, and—crucially—comes with a camera. It’s a Facetime device.

Unmoored

James re-imagines the Barbican as an airship drifting free of central London.

In Flux | Trent Walton

Trent offers some excellent advice for dealing with the effects of the iPad’s retina display on your websites. That advice is: don’t panic.

From Print to iPad: Designing a Reading Experience

Harry’s 15 minute case-study presentation at UX London was excellent. He says the lesson is that we shouldn’t be afraid to make mistakes, but there’s another lesson here too: testing with users will save your ass.

Newsstand Is Promising, Yay! But Enough with Issue-Based Publishing (Global Moxie)

Josh nails it: publishers need to stop thinking in terms of issues:

Publishers and designers have to start thinking about content at a more atomic level, not in aggregated issues. That’s how we already understand news as consumers, and we have to start thinking that way as publishers, too. This is why Flipboard, Instapaper, and other aggregators are so interesting: they give you one container for the whole universe of content, unbound to any one publisher.

OLO

This is a thoroughly enjoyable, frustratingly addictive two-player game for the iPad.

Oh, and it just happens to be made with HTML, CSS and JavaScript.

McSweeney’s Internet Tendency: TripAdvisor.com Reviews for the Overlook Hotel.

I look forward to seeing Eyes Wide Shut as a series of Foursquare check-ins.

Push Pop Press: Al Gore’s Our Choice

This looks like a beautiful way to present information, although it seems a real shame that the information is locked to just one class of device.

Paul Irish, Divya Manian and Shi Chuan launched Mobile Boilerplate recently—a mobile companion site to HTML5 Boilerplate.

There’s some good stuff in there but I was a little surprised to see that the meta viewport element included values for minimum-scale=1.0, maximum-scale=1.0, user-scalable=no:

<meta name="viewport" content="width=device-width, target-densitydpi=160dpi, minimum-scale=1.0, maximum-scale=1.0, user-scalable=no">

Setting user-scalable=no is pretty much the same as setting minimum-scale=1.0, maximum-scale=1.0. In any case, I’m not keen on it. Like Roger, I don’t think we should take away the user’s right to pinch and zoom to make content larger. That’s why my usual viewport declaration is:

<meta name="viewport" content="width=device-width, initial-scale=1">

Yes, I know that most native apps don’t allow you to zoom but I see no reason to replicate that failing on the web.

But there’s a problem. Allowing users to scale content for comfort would be fine if it weren’t for a bug in Mobile Safari:

When the meta viewport tag is set to content="width=device-width,initial-scale=1", or any value that allows user-scaling, changing the device to landscape orientation causes the page to scale larger than 1.0. As a result, a portion of the page is cropped off the right, and the user must double-tap (sometimes more than once) to get the page to zoom properly into view.

This is really annoying so Shi Chuan set about fixing the problem.

His initial solution was to keep minimum-scale=1.0, maximum-scale=1.0 in the meta viewport element but then to change it using JavaScript once the user initiates a gesture (the gesturestart event is triggered as soon as two fingers are on the screen). At the point, the content attribute of the meta viewport element gets updated to read minimum-scale=0.25, maximum-scale=1.6, the default values:

var metas = document.getElementsByTagName('meta');
var i;
if (navigator.userAgent.match(/iPhone/i)) {
  document.addEventListener("gesturestart", gestureStart, false);
  function gestureStart() {
    for (i=0; i<metas.length; i++) {
      if (metas[i].name == "viewport") {
        metas[i].content = "width=device-width, minimum-scale=0.25, maximum-scale=1.6";
      }
    }
  }
}

That works nicely but I wasn’t too keen on the dependency between the markup and the script. If, for whatever reason, the script doesn’t get executed, users are stuck with an unzoomable page.

I suggested that the script should also set the initial value to minimum-scale=1.0, maximum-scale=1.0:

var metas = document.getElementsByTagName('meta');
var i;
if (navigator.userAgent.match(/iPhone/i)) {
  for (i=0; i<metas.length; i++) {
    if (metas[i].name == "viewport") {
      metas[i].content = "width=device-width, minimum-scale=1.0, maximum-scale=1.0";
    }
  }
  document.addEventListener("gesturestart", gestureStart, false);
}
function gestureStart() {
  for (i=0; i<metas.length; i++) {
    if (metas[i].name == "viewport") {
      metas[i].content = "width=device-width, minimum-scale=0.25, maximum-scale=1.6";
    }
  }
}

Now the markup still contains the robust accessible default:

<meta name="viewport" content="width=device-width, initial-scale=1">

…while the script takes care of initially setting the scale values and also updating them when a gesture is detected. Here’s what’s happening:

1. By default, the page is scaleable because the initial meta viewport declaration doesn’t set a minimum-scale or maximum-scale.
2. Once the script loads, the page is no longer scalable because both minimum-scale and maximum-scale have been set to 1.0. If the device is switched from portrait to landscape, the resizing bug won’t be triggered because scaling is disabled.
3. When the gesturestart event is detected—indicating that the user might be trying to scale the page—the minimum-scale and maximum-scale values are updated to allow scaling. At this point, if the device is switched from portrait to landscape, the resizing bug will occur because the page is now scaleable.

Jason Weaver points out that you should probably detect for iPad too. That’s a pretty straightforward update:

if (navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPad/i))

Mathias Bynens updated the code to use querySelectorAll which is supported in Mobile Safari. Here’s the code I’m currently using:

if (navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPad/i)) {
  var viewportmeta = document.querySelector('meta[name="viewport"]');
  if (viewportmeta) {
    viewportmeta.content = 'width=device-width, minimum-scale=1.0, maximum-scale=1.0';
    document.body.addEventListener('gesturestart', function() {
      viewportmeta.content = 'width=device-width, minimum-scale=0.25, maximum-scale=1.6';
    }, false);
  }
}

You can try it out on Huffduffer, Salter Cane, Principia Gastronomica and right here on Adactio.

Right now there’s still a little sluggishness between the initial pinch-zoom gesture and the scaling; the scale values (0.25 - 1.6) don’t seem to take effect immediately. A second pinch-zoom gesture is often required. If you have any ideas for improving the event capturing and propagation, dive in there.

Update: the bug has been fixed in iOS 6.

iPad: The Microwave Oven of Computing | Techinch

Y’know, I think this comparison actually makes a lot of sense.

A Starry Night on Vimeo

A very cute Christmas message from Torchbox.

iOS Fonts

A handy list of installed fonts on the iPhone and iPad.

When I was in Boston for An Event Apart back in May, I bought an iPad.

That makes it sound like a very casual act. Back then, iPads were in very short supply. When I asked an employee at the Apple Store in Boston if they had any iPads in stock, he didn’t answer Hmm… let me check, or Not at the moment. He answered Oh, God no!

Fortunately, Malarkey was far more persistent than I and managed, through means fair or foul, to track down the only two iPads for sale in all of Beantown.

I used the device for two months and then, as predicted, I gave it to my mother. It was definitely the right decision. She loves it. She’s using it all the time, surfing the web wherever she pleases.

I liked using the iPad, but I didn’t love it. Once the novelty wore off, I found I had few good reasons to use it rather than use my Macbook.

Some observations from a Summer spent in the company of the iPad…

The iPad is almost exactly the wrong weight. It’s just that little bit too heavy to comfortably hold in one hand. It’s no coincidence that all of iPad demonstrations show it resting on a raised leg. I found that if I was using the iPad for any length of time, I would adopt a more and more relaxed, nay… slouchy position. Before I knew it, I was supine on the sofa, one leg raised with the iPad resting against it, like a Roman emperor waiting to be fed individually-peeled grapes.

The iPad is a great device for reading. I rediscovered the power of RSS. The iPad is also a great device for browsing the web. The problem is, I don’t really browse the web that much. Instead, I browse, I bookmark, I quote, I huffduff, I copy, I paste, I tweet, I share. Using an iPad made me realise that the web is very much a read/write medium for me. While it’s possible to do all those things on the iPad, it isn’t easy.

The tell-tale moment came when I was reading something on the iPad—supine on the sofa, of course—that I wanted to post to Delicious. Rather than fill in a form using the iPad’s on-screen keyboard, I found myself putting the iPad down and reaching for my Macbook just to accomplish that one little task. That’s when I knew it wasn’t the ideal device for me.

Don’t get me wrong: I’m not joining in the chorus of those silly enough to say that the iPad can’t be used for creating, editing and updating. That conclusion would be far too absolute and Boolean. But I think the iPad favours consumption more than creation. And that’s okay. That still makes it a great device for many people—such as my mother.

There’s one big fly in the ointment of the iPad as a device for non-geeks—such as my mother—and that’s the out-of-the-box experience. John Gruber pointed this out in his review of the iPad:

One thing that is very iPhone-like about iPad is that when you first take it out of the box, it wants to be plugged into your Mac or PC via USB and sync with iTunes. … It creates an impression that the iPad does not stand on its own. It’s a child that still needs a parent. But it’s not a young child. It’s more like a teenager. It’s close. So close that it feels like it ought to be able to stand on its own.

My mother has an old G3 Ruby iMac which doesn’t have USB 2, so I had to set up the iPad on my machine before giving it to her. It’s such a shame that this step is even necessary. I love the idea of a portable touchscreen device that you can simply turn on for the first time, connect to a WiFi network, and start surfing the web.

Perhaps later versions of the iPad will support account synching via MobileMe. Perhaps later versions of the iPad will support multiple accounts, which would make it a great family household device. Perhaps later versions of the iPad will have a front-facing camera and support FaceTime. Perhaps later versions of the iPad will be slimmer and lighter.

Perhaps I will get a later version of the iPad.