Some Helpful Links:
Pre-Built Search Demo
Hello there! If you're interested in doing visual similarity searches and want to get your feet wet with our user interface, you've come to the right place! We're going to simulate an actual application and go through all of the VS functionality in this article, so let's get searchin'.
Note that you can also click on the "Pre Built Demo" link above if you want to skip steps 1-3 and simply view a demo with some images that we've uploaded already. All of the things you can do with that demo are described in step 4.
And another note - if any of the videos in this guide are a bit tough to see, feel free to click on them in the bottom right corner to make them larger.
Step 1: Create an account at https://clarifai.com/signup
The first thing that you need to do is sign up for an account, and when you verify your email address you'll get to perform 5,000 searches and store 10,000 images for free each month. Pretty cool huh?
Step 2: Edit your Application
All search operations are tied to an account and application, and any images that you add to your search index will be contained within a specific application.
Head on over to our new user interface if you're not there yet. By default you'll have an application in your account already so let's change that one's name to whatever you'd like. I'm going to name mine "Super Cool Search Demo" just for kicks.
Step 3: Explore the UI and add images to your application
Search indexes are built by uploading your own data to our cloud. To do this you can either:
- Drag + drop image files from your computer, OR
- Paste in a list of URLs
This video showcases the drag-and-drop option, but if you want to use URLs you can just copy and paste them into the applicable text box:
Step 4: Do some searches!
Now that we have some data in our application we can search through it in a variety of ways. Let's go through each of them:
Search by Concept(s)
One of the cool things about our Search functionality is that it enables you to filter by any of our 11,000 general model concepts, and you can even stack them on top of each other too! In my collection of Emoji images below, I first filter by "person", and then by "sketch", giving me the only two images that are essentially both. Then I remove "person" and I'm left with all of the images that our general model considers sketches.
Give it a shot in your application as well! Our search field offers predictive text so even if you don't know what tags to choose from, it'll try to help you out.
Search by Image(s)
Another search that you can perform is a similarity search by an external image (or multiple images at once, if you'd like). To do this you can either copy and paste a URL into the search field OR you can actually copy a file from your computer and just paste it there:
In the video above, I'm first pasting a URL of the Yankees' logo from the web and then I'm pasting a local file of it from my hard drive. You can see that each search brings up the Yankee logo that I have in my collection already as the best match.
Search by Metadata
In this particular demo we don't have any metadata appended to our images since this needs to be done via the API, but if we did have metadata we could search for it by typing this directly into the search field:
In your use case, "key" might be "brand" and "value" might be "Doritos" for any applicable images, though you can use your imagination beyond tasty snack foods. This is valuable info because you can filter for it ahead of time, AND you can also return it to the end user because we return metadata in the API results.
Search by Image + Concept
Similar to the previous searches, the image + concept one can be used to filter your pool within a narrow range of scope. Searching by image with a concept will first find all images in your app that the General Model has identified as the respective concept ("sketch" here) and then find the most visually similar images/products within that subset. This is often helpful if the given image is a visually distinct pattern but not necessarily an image of a whole product.
With the filter above of a picture of lettuce (presumably from the web) and "sketch", we get the granularity that we're looking for here...a sad lettuce emoji. You could also try to do "sketch" + "lettuce" as concepts but since our General Model wasn't trained on animated pictures of lettuce, the picture definitely works better.
Search by NOT a Concept
This one is essentially like the concept search, except you are saying "Not" this. All you have to do for this is add a concept in the search field, and then click on it until it turns from green to red.
In the above example, I'm searching for everything that is not a sketch.
Search by Crop
Moving over to a different application for a bit, with sneakers instead of emojis - let's say you wanted to perform a search on only PART of an image and not all of it. If you click on any picture in your gallery and then the little crop button on the bottom left of the ensuing page, you will narrow down the scope of the search to just that crop, which you can see on the right-hand side of the page.
Search by Geolocation
Another tidbit of data that you can add to your images before uploading them is their latitude and longitude coordinates, if that fits in with your use case. This also needs to be done via the API, and if you do have that data handy you can then search for it in our UI via the following syntax:
Step 5: Add more inputs or delete some (optional)
Now that you've done a bunch of searches in your application you can determine if you need to add more images or maybe delete some. Doing either is pretty simple.
Adding More Inputs
To add more you can always click on the "Add Inputs" button on the top right of the gallery page. This will bring you back to that initial screen that we saw, and you'll also see the total # of images that you have at the bottom of it:
To remove inputs from your application, all you'll need to do is click on the little checkmark on the bottom left of one and then click on the "Delete Inputs" button at the bottom of the page. To select multiple images at once you can hold down the Shift key, which you'll see me doing in the example below:
And that's pretty much it!
If you have any questions, comments or suggestions we'd love to hear them! Feel free to email us at firstname.lastname@example.org!