Convolution#

In mathematics (in particular, functional analysis), convolution is a mathematical operation on two functions (f and g) that produces a third function \({\displaystyle f*g}\) that expresses how the shape of one is modified by the other. The term convolution refers to both the result function and to the process of computing it. It is defined as the integral of the product of the two functions after one is reversed and shifted. The integral is evaluated for all values of shift, producing the convolution function.

\((f * g)(t) := \int_{-\infty}^\infty f(\tau) g(t - \tau) \, d\tau\)

\((f * g)(t) := \int_{-\infty}^\infty f(t-\tau) g(\tau)\, d\tau\)

\(g(x,y)= \omega *f(x,y)=\sum_{dx=-a}^a{\sum_{dy=-b}^b{ \omega (dx,dy)f(x+dx,y+dy)}}\)

valid : no padding. result shape will be less than input shape.
n x n * f x f = n-f+1 x n-f+1
same : with padding p (number of layers of padding). result shape is same to input shape.
n+p x n+p * f x f = n+2p-f+1 x n+2p-f+1
[1]:
import numpy as np
import pandas as pd
from matplotlib import style
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import seaborn as sns
from scipy import signal



style.use('ggplot')
%matplotlib inline
[2]:
sine_signal1 = np.sin(range(50)) * 0.5
sine_signal2 = np.sin(range(50))
new_sine_singal = np.convolve(sine_signal1,sine_signal2,mode='same')

fig = plt.figure(figsize=(10,10))

ax = fig.add_subplot(2,2,1)
ax.plot(sine_signal1)

ax = fig.add_subplot(2,2,2)
ax.plot(sine_signal2)

ax = fig.add_subplot(2,2,(3,4))
ax.plot(sine_signal1)
ax.plot(sine_signal2)
ax.plot(new_sine_singal,label='convoluted')

plt.legend()
plt.show()
../_images/MathExploration_Convolution_3_0.png
[3]:
signal1 = np.ones(50) * 0.5
signal2 = np.array([i for i in range(0,26)] + [i for i in range(0,25)][::-1]) * 0.1
new_singal = np.convolve(signal1,signal2,mode='same')

fig = plt.figure(figsize=(10,10))

ax = fig.add_subplot(2,2,1)
ax.plot(signal1)

ax = fig.add_subplot(2,2,2)
ax.plot(signal2)

ax = fig.add_subplot(2,2,(3,4))
ax.plot(signal1)
ax.plot(signal2)
ax.plot(new_singal,label='convoluted')

plt.legend()
plt.show()
../_images/MathExploration_Convolution_4_0.png

Signal Convolution#

[4]:
signal_line = np.random.randn(100)
[5]:
plt.plot(signal_line)
[5]:
[<matplotlib.lines.Line2D at 0x7f14dd879310>]
../_images/MathExploration_Convolution_7_1.png
[6]:
window = 5
smoothing_signal = np.ones(window) / float(window)
[7]:
new_signal_line = np.convolve(signal_line,smoothing_signal,mode='same')
[8]:
new_signal_line
[8]:
array([-0.47734088, -0.27687185, -0.29393971, -0.1066033 ,  0.09736556,
        0.45289297,  0.04756477,  0.26822905,  0.40187473,  0.24082647,
        0.41217662,  0.76517268,  0.49688369,  0.46312898,  0.26580692,
        0.03351846, -0.33633262, -0.39381024, -0.55957182, -0.52947594,
       -0.525092  , -0.07016307,  0.22322092,  0.10135858, -0.02783182,
        0.0984603 , -0.00148046, -0.48340608, -0.57376853, -0.39750573,
       -0.5823513 , -0.80086388, -0.45619374, -0.47690976, -0.35580649,
       -0.13950052, -0.04144689,  0.14598446,  0.38731381,  0.35532984,
        0.08925859,  0.02268044, -0.27751046, -0.1413949 , -0.15856734,
       -0.09842096,  0.08271659,  0.34134293,  0.51861305,  0.35953995,
        0.36056003,  0.33462104,  0.02376747, -0.21990036, -0.27285818,
        0.01674449,  0.16364097,  0.02327841, -0.15910063,  0.08299095,
       -0.12733051, -0.3531635 , -0.16128683,  0.28128665,  0.00430502,
        0.10949646,  0.44265985,  0.85052908,  0.33311229,  0.80411702,
        0.32739469,  0.03266699, -0.49257795, -0.49637979, -0.65403141,
       -0.58598044, -0.99550792, -0.59266923, -0.31530281, -0.05183088,
        0.53585522,  0.74286332,  0.54320935,  0.57520181,  0.42163794,
       -0.27952655,  0.14442067,  0.12735241,  0.3131792 ,  0.32660865,
        0.65969175,  0.56178053,  0.70910954,  0.49701814,  0.57190513,
        0.61093803,  0.28358966,  0.00544587, -0.0260022 , -0.29463361])
[9]:
plt.plot(signal_line)
plt.plot(new_signal_line)
[9]:
[<matplotlib.lines.Line2D at 0x7f14dd7e61c0>]
../_images/MathExploration_Convolution_11_1.png

matrix convolution#

| 255 255 255 |    |  1  -1   1 |   | (255 x 1)+(255 x -1)+(255 x 1) +  |
| 255 100 255 |  x | -1   0  -1 | = | (255 x -1)+(100 x 0)+(255 x -1) + |  = -155
| 100 100 100 |    |  1  -1   1 |   | (100 x1 )+(100 x -1)+(100 x 1)    |
[10]:
a =np.array([
    [255, 255, 255],
    [255, 100, 255],
    [100, 100, 100]
])
b = np.array([
    [1, -1, 1],
    [-1, 0, -1],
    [1, -1, 1]
])

(a * b).sum()
[10]:
-155

Image Convolution#

[11]:
img = mpimg.imread("./images/rdj.jpg")
img.shape
[11]:
(550, 550, 3)
[12]:
plt.imshow(img)
[12]:
<matplotlib.image.AxesImage at 0x7f14dd53a100>
../_images/MathExploration_Convolution_16_1.png

Compress image#

array[::a,::a] –> every a index from rows and columns

[13]:
x = np.array([
    [1,2,3,4],
    [2,3,4,5],
    [5,67,7,7]
])
x[::3,::3]
[13]:
array([[1, 4]])

converting to float#

[14]:
a = img[::4,::4] / 255
[15]:
a.shape
[15]:
(138, 138, 3)
[16]:
plt.imshow(a)
[16]:
<matplotlib.image.AxesImage at 0x7f14dd5187f0>
../_images/MathExploration_Convolution_22_1.png
[17]:
def apply_filter(mat, conv_filter):
    mapped_1 = signal.convolve2d(mat[:,:,0],conv_filter,mode='same')
    mapped_2 = signal.convolve2d(mat[:,:,1],conv_filter,mode='same')
    mapped_3 = signal.convolve2d(mat[:,:,2],conv_filter,mode='same')
    new_mat = np.dstack((mapped_1,mapped_2,mapped_3))

    return new_mat

Edge Detection#

[18]:
edge_filter_mat_1 = np.array([
    [1, -1, 1],
    [-1, 0, -1],
    [1, -1, 1]
])

plt.imshow(apply_filter(a,edge_filter_mat_1))
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
[18]:
<matplotlib.image.AxesImage at 0x7f14dd483130>
../_images/MathExploration_Convolution_25_2.png
[19]:
edge_filter_mat_2 = np.array([
    [1, 0, -1],
    [0, 0, 0],
    [-1, 0, 1]
])
plt.imshow(apply_filter(a,edge_filter_mat_2))
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
[19]:
<matplotlib.image.AxesImage at 0x7f14dd8adf40>
../_images/MathExploration_Convolution_26_2.png
[20]:
edge_filter_mat_3 = np.array([
    [-1, -1, -1],
    [-1, 8, -1],
    [-1, -1, -1]
])
plt.imshow(apply_filter(a,edge_filter_mat_3))
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
[20]:
<matplotlib.image.AxesImage at 0x7f14dd81f640>
../_images/MathExploration_Convolution_27_2.png

Sharpen#

[21]:
sharpen_filter_mat = np.array([
    [0, -1, 0],
    [-1, 5, -1],
    [0, -1, 0]
])
plt.imshow(apply_filter(a,sharpen_filter_mat))
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
[21]:
<matplotlib.image.AxesImage at 0x7f14dd764940>
../_images/MathExploration_Convolution_29_2.png

Box Blur#

[22]:
box_blur_filter_mat = np.array([
    [1, 1, 1],
    [1, 1, 1],
    [1, 1, 1]
]) * 1/9
plt.imshow(apply_filter(a,box_blur_filter_mat))
[22]:
<matplotlib.image.AxesImage at 0x7f14dda14c10>
../_images/MathExploration_Convolution_31_1.png

Gaussian Blur 3X3#

[23]:
gaus_blur_filter_mat = np.array([
    [1, 2, 1],
    [2, 4, 2],
    [1, 2, 1]
]) * 1/16
plt.imshow(apply_filter(a,gaus_blur_filter_mat))
[23]:
<matplotlib.image.AxesImage at 0x7f14dd6910d0>
../_images/MathExploration_Convolution_33_1.png

Gaussian Blue 5X5#

[24]:
gaus_blur_5x5_filter_mat = np.array([
    [1, 4, 6, 4, 1],
    [4, 16, 24, 16, 4],
    [6, 24, 36, 24, 6],
    [4, 16, 24, 16, 4],
    [1, 4, 6, 4, 1]
]) * 1/256
plt.imshow(apply_filter(a,gaus_blur_5x5_filter_mat))
[24]:
<matplotlib.image.AxesImage at 0x7f14dd66e940>
../_images/MathExploration_Convolution_35_1.png

Brighten#

[25]:
bright_filter_mat = np.array([
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0]
])
plt.imshow(apply_filter(a,bright_filter_mat))
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
[25]:
<matplotlib.image.AxesImage at 0x7f14dd5d1850>
../_images/MathExploration_Convolution_37_2.png

Costum Function#

[26]:
def custom_conv2D(matrix, kernel):

    k_r, k_c = kernel.shape
    f_r, f_c = matrix.shape

    max_rows = f_r - k_r + 1
    max_cols = f_c - k_c + 1

    new_frame = []

    for r in range(max_rows):

        new_row = []

        for c in range(max_cols):

            res = matrix[r : r+k_r, c : c+k_c ] * kernel
            new_row.append(res.sum())

        new_frame.append(new_row)

    new_mat = np.array(new_frame)

    return new_mat

def custom_image_conv(img,kernel):

    if len(img.shape) == 3:

        layers = img.shape[2]
        layer_mat = []

        for i in range(layers):
            mapped = custom_conv2D(matrix=img[:,:,i], kernel=kernel)
            layer_mat.append(mapped)

        new_mat = np.dstack(tuple(layer_mat))

    elif len(img.shape) == 2:

        new_mat = custom_conv2D(matrix=img, kernel=kernel)

    else:

        raise ValueError("shape is 1-D")

    return new_mat

Vertical Edge#

[27]:
vertical_kernel = np.array([
    [1, 0, -1],
    [1, 0, -1],
    [1, 0, -1]
])
[28]:
feature_mat = np.array([
    [10, 10, 10, 0, 0, 0],
    [10, 10, 10, 0, 0, 0],
    [10, 10, 10, 0, 0, 0],
    [10, 10, 10, 0, 0, 0],
    [10, 10, 10, 0, 0, 0],
    [10, 10, 10, 0, 0, 0]
])


plt.imshow(feature_mat)
plt.show()

plt.imshow(vertical_kernel)
plt.show()


new_mat = custom_conv2D(matrix=feature_mat, kernel=vertical_kernel)

print(new_mat)
plt.imshow(new_mat)
plt.show()
../_images/MathExploration_Convolution_42_0.png
../_images/MathExploration_Convolution_42_1.png
[[ 0 30 30  0]
 [ 0 30 30  0]
 [ 0 30 30  0]
 [ 0 30 30  0]]
../_images/MathExploration_Convolution_42_3.png
[29]:
feature_mat = np.array([
    [0, 0, 0, 10, 10, 10],
    [0, 0, 0, 10, 10, 10],
    [0, 0, 0, 10, 10, 10],
    [0, 0, 0, 10, 10, 10],
    [0, 0, 0, 10, 10, 10],
    [0, 0, 0, 10, 10, 10]
])


plt.imshow(feature_mat)
plt.show()

plt.imshow(vertical_kernel)
plt.show()

new_mat = custom_conv2D(matrix=feature_mat, kernel=vertical_kernel)

print(new_mat)
plt.imshow(new_mat)
plt.show()
../_images/MathExploration_Convolution_43_0.png
../_images/MathExploration_Convolution_43_1.png
[[  0 -30 -30   0]
 [  0 -30 -30   0]
 [  0 -30 -30   0]
 [  0 -30 -30   0]]
../_images/MathExploration_Convolution_43_3.png
[30]:
feature_mat = np.array([
    [10, 10, 10, 0, 0, 0],
    [10, 10, 10, 0, 0, 0],
    [10, 10, 10, 0, 0, 0],
    [0, 0, 0, 10, 10, 10],
    [0, 0, 0, 10, 10, 10],
    [0, 0, 0, 10, 10, 10]
])


plt.imshow(feature_mat)
plt.show()

plt.imshow(vertical_kernel)
plt.show()

new_mat = custom_conv2D(matrix=feature_mat, kernel=vertical_kernel)

print(new_mat)
plt.imshow(new_mat)
plt.show()
../_images/MathExploration_Convolution_44_0.png
../_images/MathExploration_Convolution_44_1.png
[[  0  30  30   0]
 [  0  10  10   0]
 [  0 -10 -10   0]
 [  0 -30 -30   0]]
../_images/MathExploration_Convolution_44_3.png
[35]:
img = mpimg.imread("./images/rdj.jpg")

print(img.shape)
plt.imshow(img)
plt.show()

new_mat = custom_image_conv(img, vertical_kernel)
plt.imshow(new_mat)
plt.show()
(550, 550, 3)
../_images/MathExploration_Convolution_45_1.png
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
../_images/MathExploration_Convolution_45_3.png
[36]:
img = mpimg.imread("./images/rdj.jpg")

print(img.shape)
plt.imshow(img)
plt.show()

new_mat = np.sum(custom_image_conv(img, vertical_kernel),axis=2)
plt.imshow(new_mat,cmap='gray')
plt.show()
(550, 550, 3)
../_images/MathExploration_Convolution_46_1.png
../_images/MathExploration_Convolution_46_2.png

Horizontal Edge#

[38]:
horizontal_kernel = np.array([
    [1, 1, 1],
    [0, 0, 0],
    [-1, -1, -1]
])

[39]:
img = mpimg.imread("./images/rdj.jpg")

print(img.shape)

plt.imshow(img)
plt.show()

new_mat = custom_image_conv(img=img, kernel=horizontal_kernel)
plt.imshow(new_mat)
plt.show()
(550, 550, 3)
../_images/MathExploration_Convolution_49_1.png
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
../_images/MathExploration_Convolution_49_3.png

Sobel Operator#

[40]:
sobel_filter = np.array([
    [1, 0, -1],
    [2, 0, -2],
    [1, 0, -1]
])
[41]:
img = mpimg.imread("./images/rdj.jpg")

print(img.shape)

plt.imshow(img)
plt.show()

new_mat = custom_image_conv(img, sobel_filter)
plt.imshow(new_mat)
plt.show()
(550, 550, 3)
../_images/MathExploration_Convolution_52_1.png
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
../_images/MathExploration_Convolution_52_3.png

Scharr Operator#

[42]:
scharr_filter = np.array([
    [3, 0, -3],
    [10, 0, -10],
    [3, 0, -3]
])

[43]:
img = mpimg.imread("./images/rdj.jpg")

print(img.shape)


plt.imshow(img)
plt.show()

new_mat = custom_image_conv(img, scharr_filter)
plt.imshow(new_mat)
plt.show()
(550, 550, 3)
../_images/MathExploration_Convolution_55_1.png
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
../_images/MathExploration_Convolution_55_3.png
[44]:
img = mpimg.imread("./images/rdj.jpg")

print(img.shape)

plt.imshow(img)
plt.show()

new_mat = custom_image_conv(img, scharr_filter.T)
plt.imshow(new_mat)
plt.show()
(550, 550, 3)
../_images/MathExploration_Convolution_56_1.png
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
../_images/MathExploration_Convolution_56_3.png