Windows 10: Windows 10 search bar now with Bing Visual Search

Brink · Dec 17, 2019

Every day, we have questions. Search engines, apps, digital voice assistants, or phones are usually our go-to places to find answers.

You may already know that you can use the search bar in Windows – found on your PC taskbar – to find things like an app, setting, or a file you’ve been working on.

But did you know it can also find answers to questions like…

What’s the weather like in Paris? What’s the score of the Leicester City match? How do you unzip a file?

Windows brings all these searches together in one place. It’s easy to find and easy to use.

You don’t have to scroll through a long list of results, you don’t have to open a browser, and you don’t have to leave whatever application you’re in. With a couple of keystrokes, you can find your answers.

“The goal was to enable search in Windows to bring the web to where you are,” says Alexander Campbell, a senior program manager working on search. “People spend a lot of time on the web and in browsers. We saw an opportunity to help people save time in Windows by bringing convenient, fast web results directly to the search bar in Windows. Stay on task and in your flow, just by hitting the Windows key or clicking on the search box.”

It’s also a convenient place to start a search that leads to a website. From the search bar in Windows, you can quickly type and navigate to a URL, without having to open a browser first.

“We have a search experience that gives you quick answers, web results and helps you start your web task. It’s amazing what you can do with it,” he adds.

Every day it gets more useful: a full web search experience powered by the intelligence of Bing. You can search for quick calculations, music videos, movie trailers, sports scores, stock prices, movie times, weather forecasts and much more. Results turn up right in the taskbar. If you want to see more, you can open the results in your browser.

“The search bar in Windows gives you the quick info you need with none of the distraction,” Campbell says. “In the middle of an essay? It’s easy to jump in, check a fact and then get back to your work.”

The Windows and Bing teams worked together to bring a true web search experience into Windows, just like what you’d expect in a browser, into the search bar. And the teams continue to improve the search capability thanks to your feedback and Bing innovation.

The search bar is also the best place to go when you need to get things done on your Windows 10 device. Let’s say you need to download an app, Search can find it for you fast. For example, type “download microsoft to-do” or “download solitaire” and get an install link in the results preview. It’s also an easy way to get Tech Help answers to common questions, like “how do I connect a printer” or “how do I unzip files.”

And now, in addition to entering a text search, you can search with an image by taking a screenshot. This feature is rolling out to users in the U.S. first with international markets to follow shortly after. (You need the Windows 10 May 2019 Update or newer to see this.)

Simply click the Bing Visual Search button in the bottom right corner and snip any part of your screen to search the web using the image.

“Visual search is a new way of searching,” says Ravi Yada, product lead for Bing Visual Search. “There are a lot of things you can’t describe in words to get good enough results. By using an image as input for search, Bing can help you search what you see.”

Bing added camera-based searches in 2017, and made visual search features available in the Windows Photos App, Microsoft Edge, the Bing App and on Android phones through Microsoft Launcher. With the latest releases of Windows, you can use Windows Search to snip any part of your screen and search.

“We found people also wanted to search with screenshots, so we brought searching with screen snips to the Windows search bar,” says Nektarios Ioannides, Bing Image Search lead.

For instance, if you’re planning your wardrobe for winter and you want inspiration from what your favorite celebs are wearing? You’ll find similar results. Or you might be shopping for furniture and see a sofa you like. Snip it and use it to comparison shop with Visual Search results [1].

Aside from finding similar products, Visual Search can recognize landmarks, flowers, celebrities, animals. It also recognizes text in images, so you can copy or search it [2].

“Search is evolving. We’re still expanding the breadth of capabilities,” Ioannides says. “It’s cool to see the growth and excitement it generates, especially as we move into a world where using our eyes to search becomes more common.”

For developers, there’s an API to incorporate Visual Search within products and apps they create.

“Search in Windows is now more intelligent than ever at getting you where you want to go, saving steps and time,” Campbell says. “We can’t wait to see how you use it.”

Let us know your feedback and what you want to see next by clicking the “Feedback” button in the top right corner of the search experience.

Eager to try this yourself? If you’re reading this on a device running Windows 10 May 2019 Update or newer, click on the following examples:

Open Search home

“How to zip files”

“5 pounds to euro”

“Boston to Seattle flight status”

[1] Shopping results are only available to users in the United States and United Kingdom.
[2] This is available in English for now.
Click to expand...

Source: https://blogs.windows.com/windowsexp...visual-search/

Tweet

— Twitter API (@user) View on Twitter
Click to expand...

:)

Brink · Dec 17, 2019

Introducing Bing Visual Search

Over the years people have come to expect search engines to automatically detect intent and provide great search results for text queries typed into a single search box. Now Bing takes the first step to achieve the same for images. The Bing Team sets out to connect your camera to a deep search experience.

We started several years ago by introducing ‘Search By’ Image capability. This allowed a user to specify an entire image to be used as a search query. But what if you only want to search for a certain object you saw in an internet image – or one you photographed? So far you'd have to jump through hoops to accomplish this - but not anymore. In this post we will share some recent work we’ve done to solve this challenge and some of the technical wizardry that made it possible.

Introducing Bing Visual Search

Let's say you are looking for kitchen decoration inspiration, and an image attracted your attention. You click on a thumbnail result to get to the ‘Detail View’. You really like the overall décor, but you are particularly interested in that nice-looking chandelier. Would it be possible to see where you can get one just like that? With Bing Visual Search, now you can!

In the Detail View, you will now see a magnifying glass symbol in the top left of the image. It is called the visual search button.

Clicking the visual search button displays a visual search box on the image. You can click and drag this box to adjust it to cover just the object of your interest. You can also simply draw a box around the chandelier if that’s more convenient.

Every time you adjust the visual search box, Bing instantly runs a visual search using the selected portion of the image as the query.

We realize that many Bing image search users may be shopping for items they see in the image or a similar product. We automatically detect the shopping intent and, in addition to regular image search, we also run a product search to find matching products:

Now you can simply click on the chandelier that is right for you, pick the best merchant on the detail page and finalize your purchase!

If you're not in a shopping mood after all, you can still click on “Related Images” to continue exploration of similar images. We are continuously working to detect more intents and bring the best information to the results to satisfy your search needs.

After you’ve found the perfect chandelier options for your project, it’s easy to conduct a visual search for other items. For example, in our example image you can select that beautiful bowl and find a similar one for your kitchen. You can search for any object you see in the image.

Visual search is in its infancy, and we are aware of cases where there is still room for improvement. For example, you may need to tweak your visual search box to fully capture the object of interest to get the best results.

All this goodness is available on your PC or mobile device by visiting Bing.com or in the Bing mobile app. Developers can build visual search into their app using Bing APIs as described here.

Please try out our visual search - just be careful as it can get quite addictive! Let us know what you think using Bing Listens or the Feedback button on Bing.

Under the hood

So how does it all work?

In text based search, the first step is query understanding. Similarly, in visual search, we first need to understand the query image. The major difference is that now instead of query terms we need another way to represent the query-image. Once the query understanding phase is complete, the subsequent steps in the execution of the query are similar:

As a first step in the query-image understanding process, we run Image Processing Service to perform object detection, extraction of various image features including DNN features, recognition features, and additional features used for duplicate detection.

Next comes text query inference step. Here we try to generate the best text query to represent the input image. This is based on a pipeline widely used in Bing image search to identify so called BRQs (for 'best representative query'). You can learn more about how this process works in our earlier post at: The Image Graph - Powering the Next Generation of Bing Image Search).

Subsequently we run a triggering model to identify different scenarios for search by image. Here we leverage the expertise already used in Bing answers triggering. For instance, if we detect that the query-image has the shopping intent, then we show rich segment specific experience.

Once the image understanding phase is complete, we enter the next step: matching. In order to implement search by image inside of existing Bing index serve stack, designed mostly for text search, we need to get text-like representation for the image feature vector. To accomplish this we employ a technique known in vision area as Visual Words. This technique allows us to quantize a dense feature vector into a set of discrete visual words, which are essentially a clustering of similar feature vectors into clusters, using the joint k-means algorithm. The visual words are then used to narrow down a set of candidates from billions to several millions.

After the matching step, we enter the stage of multilevel ranking. We need to rank millions of image candidates, and we’ll do that based on feature vector distances. To speed up the calculations, we use an innovative algorithm developed by Microsoft Research in collaboration with University of Science and Technology of China called Optimized Product Quantization (for more information see the paper: Optimized Product Quantization for Approximate Nearest Neighbor Search). The main idea is to decompose the original high-dimensional vector into many low-dimensional sub-vectors that are then quantized separately as pictured below.

After quantization is complete we need to calculate distances between the query-image and result-image vectors. Instead of using the usual Euclidean distance calculation, we perform a table lookup against a set of pre-calculated values to speed things up even further.

As a result of Optimized Product Quantization, we have reduced candidate set from millions to thousands. At this point we can perform a more expensive operation to rank the images more accurately. After multiple levels of ranking the de-duplication step is executed to remove any duplicate images from the results. The set of images we end up with after this step is the final result set that will be returned to the user.

What’s Next?

We’re constantly working to make the experience better. For example, you may soon notice that we will automatically help you pick objects without needing to draw a box, and provide other tools to help refine your search. We’re also continually focused on bringing the most comprehensive and highest quality visual search results.

Let us know what you think and stay tuned for more improvements!

- The Bing Team
Click to expand...

Source: Beyond text queries: Searching with Bing Visual Search | Search Quality Insights

Deborah1953_709 · Dec 17, 2019

Searching with the Bing Search bar

When I enter a search in the Bing search bar on the screen of my desk top computer, the search comes on the minimized window, but will not open in Maximized view. What caused this and how can it be fixed to open the window on the screen instead of the
bottom task bar on the computer?

Amit(Kumar) · Dec 17, 2019

Bing Search Bar

Hello,

Please answer the questions below:

Are you mentioning about the Bing search bar in the Internet explorer or Microsoft Edge?

Did you make any changes prior to this issue?

Please get back to us with the information and also with a screenshot that will help us to understand the issue better.

I would suggest you to check this link and see if it helps.

How to include a screenshot in your post

If you have any further queries, feel free to contact us. We will be happy to help you.

Thank you.

Windows 10: Windows 10 search bar now with Bing Visual Search

Windows 10 search bar now with Bing Visual Search

Windows 10 search bar now with Bing Visual Search

Windows 10 search bar now with Bing Visual Search

Windows 10 search bar now with Bing Visual Search - Similar Threads - search bar Bing

Annoying search suggestions in Bing search bar

Annoying search suggestions in Bing search bar

bing search bar

bing search bar

bing search bar

Searching with the Bing Search bar

Visual Search from Bing now lets you search what you see

Introducing Bing Visual Search

Object Detection for Visual Search in Bing

Users found this page by searching for:

do not use bing anymore

speak bar search bing