Image search

Image search uses an image as a search input to perform vector similarity search.

Additional information

Configure image search

To use images as search inputs, configure an image vectorizer integration for your collection. See the model provider integrations page for a list of available integrations.

By local image path

Use the Near Image operator to execute image search.

If your query image is stored in a file, you can use the client library to search by its filename.

Python

from pathlib import Pathdogs = client.collections.use("Dog")response = dogs.query.near_image(    near_image=Path("./images/search-image.jpg"),  # Provide a `Path` object    return_properties=["breed"],    limit=1,    # targetVector: "vector_name" # required when using multiple named vectors)print(response.objects[0])

JavaScript/TypeScript

const myCollection = client.collections.use('Dog');

// Query based on the image content
const result = await myCollection.query.nearImage('./images/search-image.jpg', {
  returnProperties: ['breed'],
  limit: 1,
  // targetVector: 'vector_name' // required when using multiple named vectors
})

console.log(JSON.stringify(result.objects, null, 2));

response, err := client.GraphQL().Get().
  WithClassName("Dog").
  WithFields(graphql.Field{Name: "breed"}).
  WithNearImage((&graphql.NearImageArgumentBuilder{}).WithImage("image.jpg")).
  WithLimit(1).
  Do(ctx)

Java

CollectionHandle<Map<String, Object>> dogs = client.collections.use("Dog");var response = dogs.query.nearImage(    QUERY_IMAGE_PATH,    q -> q.returnProperties("breed").limit(1)// targetVector: "vector_name" // required when using multiple named vectors);if (!response.objects().isEmpty()) {  System.out.println(response.objects().get(0));}

// Coming soon

Example response

JSON

{
  "data": {
    "Get": {
      "Dog": [
        {
          "breed": "Corgi"
        }
      ]
    }
  }
}

By the base64 representation

You can search by a base64 representation of an image:

Python

base64_string="SOME_BASE_64_REPRESENTATION"# Get the collection containing imagesdogs = client.collections.use("Dog")# Perform queryresponse = dogs.query.near_image(    near_image=base64_string,    return_properties=["breed"],    limit=1,    # targetVector: "vector_name" # required when using multiple named vectors)print(response.objects[0])

JavaScript/TypeScript

import { toBase64FromMedia } from 'weaviate-client';const myCollection = client.collections.use('Dog');const filePath = './images/search-image.jpg'const base64String = await toBase64FromMedia(file.path)// Perform queryconst result = await myCollection.query.nearImage(base64String, {  returnProperties: ['breed'],  limit: 1,  // targetVector: 'vector_name' // required when using multiple named vectors})console.log(JSON.stringify(result.objects, null, 2));

response, err := client.GraphQL().Get().
  WithClassName("Dog").
  WithFields(graphql.Field{Name: "breed"}).
  WithNearImage((&graphql.NearImageArgumentBuilder{}).WithImage(base64String)).
  WithLimit(1).
  Do(ctx)

Java

String base64String = fileToBase64(QUERY_IMAGE_PATH); // This would be a real base64 string// Get the collection containing imagesCollectionHandle<Map<String, Object>> dogs = client.collections.use("Dog");// Perform queryvar response = dogs.query.nearImage(base64String,    q -> q.returnProperties("breed").limit(1)// targetVector: "vector_name" // required when using multiple named vectors);if (!response.objects().isEmpty()) {  System.out.println(response.objects().get(0));}

// The C# client's NearImage method takes a byte array directly.var imageBytes = await FileToByteArray(QUERY_IMAGE_PATH);// Get the collection containing imagesvar dogs = client.Collections.Use("Dog");// Perform queryvar response = await dogs.Query.NearMedia(    query => query.Image(imageBytes).Build(),    returnProperties: ["breed"],    limit: 1);if (response.Objects.Any()){    Console.WriteLine(JsonSerializer.Serialize(response.Objects.First()));}

Example response

JSON

{
  "data": {
    "Get": {
      "Dog": [
        {
          "breed": "Corgi"
        }
      ]
    }
  }
}

Create a base64 representation of an online image

You can create a base64 representation of an online image, and use it as input for similarity search as shown above.

Python

import base64, requests

def url_to_base64(url):
    image_response = requests.get(url)
    content = image_response.content
    return base64.b64encode(content).decode("utf-8")

base64_img = url_to_base64("https://upload.wikimedia.org/wikipedia/commons/thumb/1/14/Deutsches_Museum_Portrait_4.jpg/500px-Deutsches_Museum_Portrait_4.jpg")

JavaScript/TypeScript

const imageURL = 'https://upload.wikimedia.org/wikipedia/commons/thumb/1/14/Deutsches_Museum_Portrait_4.jpg/500px-Deutsches_Museum_Portrait_4.jpg'

async function urlToBase64(imageUrl: string) {
  const response = await fetch(imageUrl);
  const content = await response.buffer();
  return content.toString('base64');
}

const base64 = await urlToBase64(imageURL)
console.log(base64)

resp, err := http.Get(url)
if err != nil {
  return "", err
}
defer resp.Body.Close()

content, err := ioutil.ReadAll(resp.Body)
if err != nil {
  return "", err
}

base64string := base64.StdEncoding.EncodeToString(content)

Java

private static String urlToBase64(String url)
    throws IOException, InterruptedException {
  HttpClient httpClient = HttpClient.newHttpClient();
  HttpRequest request = HttpRequest.newBuilder().uri(URI.create(url)).build();
  HttpResponse<byte[]> response =
      httpClient.send(request, HttpResponse.BodyHandlers.ofByteArray());
  byte[] content = response.body();
  return Base64.getEncoder().encodeToString(content);
}

private static String fileToBase64(String path) throws IOException {
  byte[] content = Files.readAllBytes(Paths.get(path));
  return Base64.getEncoder().encodeToString(content);
}

private static async Task<string> UrlToBase64(string url)
{
    using var httpClient = new HttpClient();
    var imageBytes = await httpClient.GetByteArrayAsync(url);
    return Convert.ToBase64String(imageBytes);
}

private static async Task<byte[]> FileToByteArray(string path)
{
    return await File.ReadAllBytesAsync(path);
}

Combination with other operators

A Near Image search can be combined with any other operators (like filter, limit, etc.), just as other similarity search operators.

See the similarity search page for more details.

Connect to Weaviate

Questions and feedback

Have a question or feedback? Here's how to reach us.

Community Forum

Ask questions and connect with other developers on our Community forum.

Support

Weaviate Cloud user or customer? Find the right channel on the Support page.

By local image path

By the base64 representation

Create a base64 representation of an online image

Combination with other operators

Related pages

Questions and feedback