Google pilots Duplex in New Zealand, brings booking to Maps and Search

Google quietly announced this evening that Google Duplex, its AI chat agent that can arrange appointments over the phone, will launch in New Zealand as part of a limited pilot to confirm holiday hours for a small group of local businesses. It marks the first time Duplex has expanded outside of the U.S., and it coincides with the wider rollout of a visual Duplex-powered booking flow in Google Maps and Google Search.

Concerning the New Zealand test, Google says that businesses will receive an automated call from Google asking to confirm their hours for the upcoming Labor Day public holiday on October 28. Once confirmed, the times will automatically be updated on Maps and Search.

“This not only helps people save time during busy holidays, but also ensures that businesses maximize foot traffic at their stores,” wrote Google’s VP of engineering Scott Huffman. “As always, we’ll disclose that the business is speaking with an automated system when making a call.”

As for the above-mentioned booking flow, Google writes on a support page that Duplex is now available in the U.S. for select “devices that can access Search or Maps.” In mobile Search and the latest version of Maps, tapping on the Reserve shortcut from an eligible businesses’ listing pops up a UI that requests you specify a date and your party’s size. You’re then prompted to choose a preferred time (and whether you’d accept an alternative time within a 30-minute or 60-minute window), after which you’re instructed to confirm your contact information, including your name, phone number, and email address.

Earlier this year, Duplex rolled out to all Pixel phones including the Pixel, Pixel 2, and Pixel 3 plus unspecified iOS devices and third-party Android devices via Google Assistant starting in 48 U.S. states. (The exceptions are Kentucky and Louisiana.) That’s up from the pilot metros of New York, Atlanta, Phoenix, and San Francisco where it launched last November.

To be clear, it’s not quite the Duplex experience Google demoed at its I/O 2018 developers conference in May — Google Assistant isn’t booking haircut appointments just yet. But importantly, it’s no longer limited to businesses with which Google has explicitly partnered. And for restaurants which use an online booking service that partners with Google (like Bookatable, Chef’s Club, Reserve, Resy, Seatninja, Dinetime, or Easydiner), the Assistant works directly with Reserve with Google, the company’s cross-platform service that makes it easier to manage bookings through Google Search, Maps, and other apps and portals.

Booking a table with Duplex

The most reliable way to book a table using the Google Assistant with Duplex is by invoking Google Assistant on a Pixel phone and saying “Book a table for four people at [restaurant name] tomorrow night,” “Hey Google, make a restaurant reservation,” or “Hey Google, reserve a table.” For example: “Make a restaurant reservation in Manhattan.” Those utterances spring up a list of options that can be refined by cuisine (e.g., Thai, Chinese, barbecue, American).

Selecting a restaurant either by saying its name or by tapping on it brings you to the next step in the Duplex-powered reservation flow. Alternatively, you can look up a specific restaurant first, and if the option shows up, tap “Request a table there.”

Duplex doesn’t yet work with every restaurant Google says that businesses will be able to opt out from receiving calls by toggling a setting in their Google My Business account. But for restaurants that haven’t been delisted, kicking off a reservation is a relatively straightforward process: Google Assistant first prompts you to specify the size of your party, suggesting sizes like two, three, or four people. Confirming a date comes next. Then, you’ll be asked to supply a preferred time within the restaurant’s hours of operation, in addition to a backup time window (for instance, 6:00 p.m to 7:00 p.m.) in the event the requested time is not available.

At this stage, Google Assistant asks for a booking phone number, which by default is your phone number. You can supply a new one, or alternatively tap the Manage Phone Number button at the bottom of the screen to pull up the Phone settings page on the Google Account dashboard. It’s populated with numbers you’ve provided and verified for Google services like Hangouts and Google Voice, which you can edit and delete at your leisure.

Finally, after you’ve given a phone number, Google Assistant asks you to confirm the details the place, date, and time. Then you’ll get a notification on your phone, an email update, and a calendar appointment.

It’s at this final screen where you’ll find the My Reservations button. Select it, and it will surface an inline list of reservations automatically imported from Gmail, manually added to Google Calendar, or placed with Duplex. Duplex-facilitated reservations that haven’t been confirmed bear a blue “Pending” tag underneath the name of the restaurant and a “View details” link. Select that link, and the Google Assistant serves a webpage with a confirmation number and a chronological list of attempts that have been made to book your reservation; a reservation summary with the date, time, party size, restaurant name, and restaurant address; and the name and phone number associated with the reservation.

What if you have to cancel a reservation? Easy tap the Cancel button at the bottom of the screen, and Google Assistant places a call to inform the restaurant.

Duplex calls for all

Booking restaurants with a robot would be fraught with peril, you’d think, considering the number of things that might go wrong. What if the human on the other end has a thick accent that throws off Duplex’s speech recognition? What if they ask an obscure question the system hasn’t been trained to handle? Or what if the restaurant’s listed number is no longer in service?

Google received a ton of criticism after its initial Duplex demo in May many were not amused that Google Assistant mimicked a human so well. In June, the company promised that Google Assistant using Duplex would first introduce itself.

Part of the reason Duplex sounds so natural is because it taps Google’s sophisticated WaveNet audio processing neural network, and because it intelligently inserts “speech disfluencies” the “ums” and “ahs” people make involuntarily in the course of conversation. They come from a branch of linguistics known as pragmatics, which deals with language in use and the contexts in which it is used including such things as taking turns in conversation, text organization, and presupposition.

When we tested Duplex late last year, Google said that although a majority of Duplex calls are made using the automated system, others are conducted with human operators. It declined to say whether that would be the case going forward, but a New York Times investigation in May found that about 25% of calls were fielded by humans and that a human had to intervene in 15% of the time.

Vice president of engineering for the Google Assistant Scott Huffman revealed in interviews this summer that the disfluencies turned out to be the key to advancing talks in tests of Duplex. Without them, he said, people were more likely to hang up as the exchange starts to feel overly artificial.

They have another, more practical purpose: buying time. When Duplex is confronted with an uncertain response from a conversation partner after, say, requesting a table for three, it’ll reiterate with a disfluency “ah, for three.”

Of course, restaurant owners don’t have to speak with the Google Assistant if they decide they’d rather not. At the beginning of exchanges, Duplex makes clear that the call is automated. In all states, it informs the person on the other end that they’re being recorded. If they respond with “I don’t want to be recorded” or some variation of the phrase, the call is handed off to a human operator on an unrecorded line. (Those operators also annotate the call transcripts used to train Duplex’s algorithms.)

Those precautions are unlikely to put to rest the ethical and legal concerns raised by Duplex’s debut. But as Google steadily marches toward its goal of launching Duplex across the U.S., they’ll be a small comfort for businesses that opt to field its autonomous phone calls.