Speaker: Stefan Weitz Bing Director of Search
How do create the ability for folks to use whatever modality they want in search (not just words). How do they help the systems understand natural human language (written – text from a keyboard/pen; spoken – voice; body – gestures). Context is also a big thing (physical/location; social). Understand the intent of the user – what are they trying to do. 2 ½ words is still the average query length. These queries could mean many things. But, these queries are still what they see the most of. Because of all of the new signals, they can better predict/model the intent.
Once you understand the intent, now we need to work on predicting. Based on historical search queries, and everything else, understand what they’re looking for.
How do they get all of this information to create the information fabric?
Consumer side – Photos taken, MS Account, Profile info, Saved items, Notes taken, media purchased, One Drive, One Note – helps to create a model of a person and aids to predictive searching.
Enterprise-side – Have one of the largest repositories of corporate data. Personal and Corporate often blend. How do we stop thinking of people as two distinct objects? You have one life. All things from Enterprise side can help with building the information fabric to know who you are as a person.
Language provides a “low resolution” description. Things have moved away from “text” to “everything captured in digital in real-time; machines can read this”. We’re capturing meta data, etc.
The world is more than a collection of pages. It’s a set of people, places and things. Services, Devices, Things, Events, Places, Payment, Images, Video, Social, and Personal. Search systems now have the raw data to start making sense of the world in which they live.
Task completion is the big thing.
How do they create these entities?
“Movies” as an entity
“Restaurant” as an entity
Patterns: what are people trying to do?
Buy – 9%
Locate – 6%
Watch video – 3%
Read – 2%
Download – 2%
They can’t just look at what people are doing on the web, today. They need to expand upon “what is possible”. With this, the universe of possible tasks expands (ie: Uber).
Need to look at people’s query strings NOT in isolation but rather in aggregate. Sometimes folks will extend a search “hunt” for days or even weeks.
What does this look like?
Task completion experiences:
Bing.com/IE – Snapshot and Bing Knows
Shells (service enabled) – Windows Search Charm; Windows Phone – Cortana; Xbox – CU; Office: Search;
1st Party Applications (Bing Enabled) – Extend across Form Factors – Phone/Xbox
Speech Recognition – looks at the entire string of “what you’re saying” and making sense of it (not just turning speech into text). Not having to rely on particular nouns/keywords.
There are 15 billion sensors right now. They need to make sense of all of this data. They then create an Inference System. They can then present stuff that is likely important to you.
Proactive search experiences extend from desktop to mobile (Cortana).
Example of pushing stuff to someone without them searching/asking for it:
If Xbox knows you listen to a particular artist, and Bing understands that the artist is coming to your town, it might let you know about this, without you asking.
Photosynth – allows anyone to take a bunch of photos and upload them to create a 3-dimensional product. Create the entire physical world by stitching all photos together (230 photos; Coliseum in Rome)
Example: “What is the President of China” – then “who is his wife?”; “who are his kids”; “where did he go to school”. Being able to remember and not force someone to redo searches from the beginning, every single time, is a big deal in search.
Bing is now understanding the entire physical world. What makes sense to show? Search “physics” and the system will show Courses. Again, this helps people complete a task. Need to respond intelligently.
Take all the data that we’re talking about (tweets, news articles, videos, etc.) and begins to predict the real world.
US House of Representatives (Missouri) predictions
How are you going to grow search share?
A: Be where they are. Spotlight on IOS8 (Bing powers search there)
Example – translates: Chinese chat via mobile device; translates in real time. Robot test – had people communicating with a bot in different languages. Natural language conversation. No one knew it was a bot. Nothing was programmed. It was completely algorithmic. This is where Bing is today. Imagine where things will be in the future?
Stefan thinks the predictive stuff will make our society more transparent. But, it’s a good question to ask. Could search engines, for example, swing an election?
Prediction – have been using text and numeric data for this. Stefan doesn’t know if other inputs are considered (video engagement, etc.).