Speaker: Stefan Weitz Bing Director of Search
How do create the ability for folks to use whatever modality they want in search (not just words). How do they help the systems understand natural human language (written text from a keyboard/pen; spoken voice; body gestures). Context is also a big thing (physical/location; social). Understand the intent of the user what are they trying to do. 2 ½ words is still the average query length. These queries could mean many things. But, these queries are still what they see the most of. Because of all of the new signals, they can better predict/model the intent.
Once you understand the intent, now we need to work on predicting. Based on historical search queries, and everything else, understand what theyre looking for.
How do they get all of this information to create the information fabric?
Consumer side Photos taken, MS Account, Profile info, Saved items, Notes taken, media purchased, One Drive, One Note helps to create a model of a person and aids to predictive searching.
Enterprise-side Have one of the largest repositories of corporate data. Personal and Corporate often blend. How do we stop thinking of people as two distinct objects? You have one life. All things from Enterprise side can help with building the information fabric to know who you are as a person.
Language provides a low resolution description. Things have moved away from text to everything captured in digital in real-time; machines can read this. Were capturing meta data, etc.
The world is more than a collection of pages. Its a set of people, places and things. Services, Devices, Things, Events, Places, Payment, Images, Video, Social, and Personal. Search systems now have the raw data to start making sense of the world in which they live.
Task completion is the big thing.
How do they create these entities?
Movies as an entity
Restaurant as an entity
Patterns: what are people trying to do?
Watch video 3%
They cant just look at what people are doing on the web, today. They need to expand upon what is possible. With this, the universe of possible tasks expands (ie: Uber).
Need to look at peoples query strings NOT in isolation but rather in aggregate. Sometimes folks will extend a search hunt for days or even weeks.
What does this look like?
Task completion experiences:
Bing.com/IE Snapshot and Bing Knows
Shells (service enabled) Windows Search Charm; Windows Phone Cortana; Xbox CU; Office: Search;
1st Party Applications (Bing Enabled) Extend across Form Factors Phone/Xbox
Speech Recognition looks at the entire string of what youre saying and making sense of it (not just turning speech into text). Not having to rely on particular nouns/keywords.
There are 15 billion sensors right now. They need to make sense of all of this data. They then create an Inference System. They can then present stuff that is likely important to you.
Proactive search experiences extend from desktop to mobile (Cortana).
Example of pushing stuff to someone without them searching/asking for it:
If Xbox knows you listen to a particular artist, and Bing understands that the artist is coming to your town, it might let you know about this, without you asking.
Photosynth allows anyone to take a bunch of photos and upload them to create a 3-dimensional product. Create the entire physical world by stitching all photos together (230 photos; Coliseum in Rome)
Example: What is the President of China then who is his wife?; who are his kids; where did he go to school. Being able to remember and not force someone to redo searches from the beginning, every single time, is a big deal in search.
Bing is now understanding the entire physical world. What makes sense to show? Search physics and the system will show Courses. Again, this helps people complete a task. Need to respond intelligently.
Take all the data that were talking about (tweets, news articles, videos, etc.) and begins to predict the real world.
US House of Representatives (Missouri) predictions
How are you going to grow search share?
A: Be where they are. Spotlight on IOS8 (Bing powers search there)
Example – translates: Chinese chat via mobile device; translates in real time. Robot test had people communicating with a bot in different languages. Natural language conversation. No one knew it was a bot. Nothing was programmed. It was completely algorithmic. This is where Bing is today. Imagine where things will be in the future?
Stefan thinks the predictive stuff will make our society more transparent. But, its a good question to ask. Could search engines, for example, swing an election?
Prediction have been using text and numeric data for this. Stefan doesnt know if other inputs are considered (video engagement, etc.).