2

Could anyone suggest a script/tool/software, that would:

  1. recognize and
  2. quantify the similarity/dissimilarity between the four attached images?

Meaning, it would show that there is a greater similarity between the top two images (images B1 and B2) and between the bottom two images (images A1 and A2) compared to the similarity between either of the top two images and either of the bottom two images. Meaning, for example, that B1 is more similar to B2 than it is to either A1 or A2. The similarity that is referred to here is the overall visual similarity between the images. When I look at those images, it is obvious to me that the top two are more similar between themselves than they are to the bottom two images and I would assume that the tool that would recognize similarities would deal with the spectral, texture, pattern, etc., characteristics of the images. Presumably, it wouldn't work with entire images at once but would allow for a window of a certain size in pixels to be passed over the images. For example, depending on the pixel size, the window in these examples (these are trees, black spruce in the top two images and mostly poplar in the bottom two images), would be large enough to capture the 'minimal unit' of difference, i.e., several trees together. Is there an approach/tool that can quantitatively express such a similarity?

I can use Python and I have tried SSIM (Structural Similarity Index Measurement) but that didn't produce satisfactory results.

B2

B1

A2

A1

  • 2
    This seems like a very poorly constrained problem. Similarity can be assessed in too many ways. Do you mean the size of the image? The arrangement and count of the "objects" in the image? The average pixel intensity? The list could go on forever. You need to determine what properties of the images are best for evaluating their similarity, then try to implement something that focuses on those properties. – Jon May 30 '18 at 19:35
  • The similarity that I meant was the overall visual similarity between the images and that would assume an approach that deals with a combination of spectral, texture, pattern, etc., characteristics. When I look at those images, it is obvious to me that the top two are more similar between themselves than they are to the bottom two images. Is there an approach/tool that quantitatively leads to the same conclusion? Thank you. – Tom May 30 '18 at 19:54
  • 1
    I don't know of any such magic tool, but I'm sure the pieces to build it are all floating around out there. If I were you, I would choose three characteristics that you think are most important toward differentiating the images, then compute a difference metric for each one. Combine the difference metric with weights as you see fit. You can use pdfs of intensity, characterizations of the modes of the pdfs (mean, mode, range, etc.), fourier transforms of the images, segmentation/object counting. If you have many examples, you could look into machine learning. – Jon May 30 '18 at 20:09
  • Thank you for your help, Jon. At first I would like to establish that no such tool exists or at least not as a routinely used tool, and then I will see if I will try to tackle the problem by myself, from scratch. You are offering some very valuable ideas. Thank you. – Tom May 30 '18 at 20:19
2

I would recommend calculating texture metrics using the scikit-image Python package. The following is a scikit-image example that uses GLCM Texture Features:

This example illustrates texture classification using grey level co-occurrence matrices (GLCMs). A GLCM is a histogram of co-occurring greyscale values at a given offset over an image.

In this example, samples of two different textures are extracted from an image: grassy areas and sky areas. For each patch, a GLCM with a horizontal offset of 5 is computed. Next, two features of the GLCM matrices are computed: dissimilarity and correlation. These are plotted to illustrate that the classes form clusters in feature space.

In a typical classification problem, the final step (not included in this example) would be to train a classifier, such as logistic regression, to label image patches from new images.


enter image description here


import matplotlib.pyplot as plt

from skimage.feature import greycomatrix, greycoprops
from skimage import data


PATCH_SIZE = 21

# open the camera image
image = data.camera()

# select some patches from grassy areas of the image
grass_locations = [(474, 291), (440, 433), (466, 18), (462, 236)]
grass_patches = []
for loc in grass_locations:
    grass_patches.append(image[loc[0]:loc[0] + PATCH_SIZE,
                               loc[1]:loc[1] + PATCH_SIZE])

# select some patches from sky areas of the image
sky_locations = [(54, 48), (21, 233), (90, 380), (195, 330)]
sky_patches = []
for loc in sky_locations:
    sky_patches.append(image[loc[0]:loc[0] + PATCH_SIZE,
                             loc[1]:loc[1] + PATCH_SIZE])

# compute some GLCM properties each patch
xs = []
ys = []
for patch in (grass_patches + sky_patches):
    glcm = greycomatrix(patch, [5], [0], 256, symmetric=True, normed=True)
    xs.append(greycoprops(glcm, 'dissimilarity')[0, 0])
    ys.append(greycoprops(glcm, 'correlation')[0, 0])

# create the figure
fig = plt.figure(figsize=(8, 8))

# display original image with locations of patches
ax = fig.add_subplot(3, 2, 1)
ax.imshow(image, cmap=plt.cm.gray, interpolation='nearest',
          vmin=0, vmax=255)
for (y, x) in grass_locations:
    ax.plot(x + PATCH_SIZE / 2, y + PATCH_SIZE / 2, 'gs')
for (y, x) in sky_locations:
    ax.plot(x + PATCH_SIZE / 2, y + PATCH_SIZE / 2, 'bs')
ax.set_xlabel('Original Image')
ax.set_xticks([])
ax.set_yticks([])
ax.axis('image')

# for each patch, plot (dissimilarity, correlation)
ax = fig.add_subplot(3, 2, 2)
ax.plot(xs[:len(grass_patches)], ys[:len(grass_patches)], 'go',
        label='Grass')
ax.plot(xs[len(grass_patches):], ys[len(grass_patches):], 'bo',
        label='Sky')
ax.set_xlabel('GLCM Dissimilarity')
ax.set_ylabel('GLCM Correlation')
ax.legend()

# display the image patches
for i, patch in enumerate(grass_patches):
    ax = fig.add_subplot(3, len(grass_patches), len(grass_patches)*1 + i + 1)
    ax.imshow(patch, cmap=plt.cm.gray, interpolation='nearest',
              vmin=0, vmax=255)
    ax.set_xlabel('Grass %d' % (i + 1))

for i, patch in enumerate(sky_patches):
    ax = fig.add_subplot(3, len(sky_patches), len(sky_patches)*2 + i + 1)
    ax.imshow(patch, cmap=plt.cm.gray, interpolation='nearest',
              vmin=0, vmax=255)
    ax.set_xlabel('Sky %d' % (i + 1))


# display the patches and plot
fig.suptitle('Grey level co-occurrence matrix features', fontsize=14)
plt.show()

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.