Image Analysis for Machine Learning

Investigate images to get sensitive/secret data or sensitive information hidden in the images.

In advance, load an image using Pillow (PIL).

import numpy as np
from PIL import Image

img = Image.open("example.png")

Basic Information

Filename

img.filename

Image information

img.info

Image format (PNG, JPG, etc.)

img.format

Color mode (RPG, CMYK, etc.)

img.mode

Image size

img.size

Bytes

img.tobytes()

Pixels

np.array(img.getdata())

Plot Images

import matplotlib.pyplot as plt

plt.imshow(img)
plt.axis('off') # Turn off axis and labels
plt.show()

Hidden Information

Find hidden data in the image by slightly changing.

Resize Image & Get Bytes

img1 = img.resize((128, 128))
print(img1.tobytes())

XOR Image Bytes

Convert image to bytes

bytes = img.tobytes()

key = 2 # specify the XOR key

xored = []
for byte in bytes:
    xored.append(byte ^ key)
xored_np = np.array(xored)
print(xored_np)

Image Manipulation for Machine Learning

We can update each pixel value to change an image.

Swapping Pixels

Reference: https://www.kaggle.com/code/jonbown/ai-ctf-submissions?scriptVersionId=105606691&cellId=102

This example updates pixel values at specified positions.

import numpy as np
from PIL import Image

img = Image.open("example.png")

# Reshape image data to desired size for easy processing
pixels = np.array(img.getdata())
pixels = np.reshape(pixels, (28, 28))

# Update each pixel with desired value for changing image
for i in range(img.size[0]):
    for j in range(img.size[1]):
        # change pixel value at position (8, 19)
        if i == 8 and j == 19:
            pixels[i, j] = 255
        # change pixel value at position 25th row, 20th column onwards
        if i > 25 and j > 20:
            pixels[i, j] = np.random.randint(0, 50)

# Convert numpy array to image
img_updated = Image.fromarray(pixels.astype(np.uint8))

Image Recognition Bypass for Machine Learning

We can trick image recognizer or classifier by adding filters or obfuscating an image.

The following techniques include those that are ineffective currently or in the future..

Blurring

from PIL import Image
from PIL import ImageFilter

img = Image.open("example.png")

# Box blur
img1 = img.filter(ImageFilter.BoxBlur(5))
# Gaussian blur
img2 = img.filter(ImageFilter.GaussianBlur(5))
# Median filter
img3 = img.filter(ImageFilter.MedianFilter(size=5))
# Rank filter
img4 = img.filter(ImageFilter.RankFilter(size=13, rank=5))

Cropping/Rotating

from PIL import Image
from PIL import ImageFilter

img = Image.open("example.png")
img = img.resize((512, 512))

img1 = img.crop((0, 0, 300, 280)).rotate(-60)