人工智能与机器学习·大作业——微笑识别和口罩识别

人工智能与机器学习·大作业——微笑识别和口罩识别

一、人脸图像特征提取的各种方法1. HOG2. 卷积神经网络特征 二、笑脸数据集1. 准备工作（1）下载tensorflow①在 Anaconda 中创建 python3.6 版本的TensorFlow环境②激活 python3.6 的 tensorflow 环境③使用豆瓣镜像安装 tensorflow④激活对应的conda环境⑤安装ipykernel⑥将环境写入notebook的kernel中⑦在系统内切换⑧查看是否安装成功 （2）下载dlib库①查看自己Anaconda中python版本②根据python版本下载相应版本的dlib.whl文件到指定位置③在tensorflow环境下命令下载安装dlib.whl文件④查看是否安装成功 （3）下载Keras①在tensorflow环境下命令下载Keras②查看是否安装成功 2. 划分测试集、训练集以及验证集（1）运行tensorflow环境，导包（2）读取训练集的图片，将训练数据和测试数据放入自己创建的文件夹（3）复制图片到文件夹中（4）在jupyter中将文件夹的路径引入（5）打印文件夹下的图片数量 3. 创建模型4. 归一化处理5. 未进行数据增强时进行训练模型（1）训练模型（2）在培训和验证数据上绘制模型的丢失和准确性 6. 数据增强7. 创建网络8. 训练模型并保存 三、笑脸识别1. 单张图片判别2. 摄像头采集人脸识别 四、口罩数据集1. 划分测试集、训练集以及验证集（1）运行tensorflow环境，导包（2）读取训练集的图片，将训练数据和测试数据放入自己创建的文件夹（3）复制图片到文件夹中（4）在jupyter中将文件夹的路径引入（5）打印文件夹下的图片数量 3. 创建模型4. 归一化处理5. 训练模型6. 数据增强7. 创建网络8. 训练模型并保存 五、口罩识别1. 单张图片判别2. 摄像头采集人脸识别

一、人脸图像特征提取的各种方法

1. HOG

方向梯度直方图（Histogram of Oriented Gradient, HOG）特征是一种在计算机视觉和图像处理中用来进行物体检测的特征描述子。HOG特征通过计算和统计图像局部区域的梯度方向直方图来构成特征。 HOG特征的总结：把样本图像分割为若干个像素的单元，把梯度方向平均划分为多个区间，在每个单元里面对所有像素的梯度方向在各个方向区间进行直方图统计，得到一个多维的特征向量，每相邻的单元构成一个区间，把一个区间内的特征向量联起来得到多维的特征向量，用区间对样本图像进行扫描，扫描步长为一个单元。最后将所有块的特征串联起来，就得到了人体的特征。至今虽然有很多行人检测算法，但基本都是以HOG+SVM的思路为主。

2. 卷积神经网络特征

卷积神经网络（Convolutional Neural Networks, CNN）是一类包含卷积计算且具有深度结构的前馈神经网络，是深度学习的代表算法之一。卷积神经网络具有表征学习能力，能够按其阶层结构对输入信息进行平移不变分类，因此也被称为“平移不变人工神经网络（Shift-Invariant Artificial Neural Networks, SIANN）”。 卷积神经网络中，第一步一般用卷积核去提取特征，这些初始化的卷积核会在反向传播的过程中，在迭代中被一次又一次的更新，无限地逼近我们的真实解。其实本质没有对图像矩阵求解，而是初始化了一个符合某种分布的特征向量集，然后在反向传播中无限更新这个特征集，让它能无限逼近数学中的那个概念上的特征向量，以致于我们能用特征向量的数学方法对矩阵进行特征提取。

二、笑脸数据集

1. 准备工作

*下载tensorflow，dlib，Keras均是在Anaconda Prompt窗口下的TensorFlow环境中进行的。

*后续写代码时jupyter都要切换至tensorflow服务器（下面有写到） 或者

（1）下载tensorflow

①在 Anaconda 中创建 python3.6 版本的TensorFlow环境

在Anaconda Prompt窗口下输入命令：

conda create -n tensorflow python=3.6

②激活 python3.6 的 tensorflow 环境

activate tensorflow

③使用豆瓣镜像安装 tensorflow

pip install tensorflow -i https://pypi.douban.com/simple

④激活对应的conda环境

conda activate tensorflow

⑤安装ipykernel

pip install ipykernel -i https://pypi.douban.com/simple

⑥将环境写入notebook的kernel中

python -m ipykernel install --user --name tensorflow --display-name "Python (tensorflow)"

⑦在系统内切换

*后续写代码时jupyter都要切换服务器

⑧查看是否安装成功

tensorflow的安装（在Anaconda中创建虚拟python3.6环境）参考了以下几个博客： https://www.cnblogs.com/maxiaodoubao/p/9854595.html https://www.cnblogs.com/phoenixash/p/12132197.html

win10中anaconda安装tensorflow时报错Traceback (most recent call last): File "E:\Anaconda3\lib\site-packag 这个问题可能是源的问题，我换了豆瓣镜像源下载就没有出错了，参考下面的博客： https://blog.csdn.net/qq_43211132/article/details/94426458

（2）下载dlib库

①查看自己Anaconda中python版本

②根据python版本下载相应版本的dlib.whl文件到指定位置

下载的dlib.whl文件如下： dlib-19.7.0-cp36-cp36m-win_amd64.whl

③在tensorflow环境下命令下载安装dlib.whl文件

pip install D:\dlib-19.7.0-cp36-cp36m-win_amd64.whl

④查看是否安装成功

（3）下载Keras

①在tensorflow环境下命令下载Keras

pip install keras

②查看是否安装成功

2. 划分测试集、训练集以及验证集

（1）运行tensorflow环境，导包

import keras keras.__version__

（2）读取训练集的图片，将训练数据和测试数据放入自己创建的文件夹

# The path to the directory where the original # dataset was uncompressed riginal_dataset_dir = 'D:\genki4k' # The directory where we will # store our smaller dataset base_dir = 'genki4k' os.mkdir(base_dir) # Directories for our training, # validation and test splits train_dir = os.path.join(base_dir, 'train') os.mkdir(train_dir) validation_dir = os.path.join(base_dir, 'validation') os.mkdir(validation_dir) test_dir = os.path.join(base_dir, 'test') os.mkdir(test_dir) # Directory with our training smile pictures train_smile_dir = os.path.join(train_dir, 'smile') os.mkdir(train_smile_dir) # Directory with our training unsmile pictures train_unsmile_dir = os.path.join(train_dir, 'unsmile') #s.mkdir(train_dogs_dir) # Directory with our validation smile pictures validation_smile_dir = os.path.join(validation_dir, 'smile') os.mkdir(validation_smile_dir) # Directory with our validation unsmile pictures validation_unsmile_dir = os.path.join(validation_dir, 'unsmile') os.mkdir(validation_unsmile_dir) # Directory with our validation smile pictures test_smile_dir = os.path.join(test_dir, 'smile') os.mkdir(test_smile_dir) # Directory with our validation unsmile pictures test_unsmile_dir = os.path.join(test_dir, 'unsmile') os.mkdir(test_unsmile_dir)

（3）复制图片到文件夹中

（4）在jupyter中将文件夹的路径引入

train_smile_dir="genki4k/train/smile/" train_umsmile_dir="genki4k/train/unsmile/" test_smile_dir="genki4k/test/smile/" test_umsmile_dir="genki4k/test/unsmile/" validation_smile_dir="genki4k/validation/smile/" validation_unsmile_dir="genki4k/validation/unsmile/" train_dir="genki4k/train/" test_dir="genki4k/test/" validation_dir="genki4k/validation/"

（5）打印文件夹下的图片数量

print('total training smile images:', len(os.listdir(train_smile_dir))) print('total training unsmile images:', len(os.listdir(train_umsmile_dir))) print('total testing smile images:', len(os.listdir(test_smile_dir))) print('total testing unsmile images:', len(os.listdir(test_umsmile_dir))) print('total validation smile images:', len(os.listdir(validation_smile_dir))) print('total validation unsmile images:', len(os.listdir(validation_unsmile_dir)))

3. 创建模型

#创建模型 from keras import layers from keras import models model = models.Sequential() model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3))) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(128, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(128, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Flatten()) model.add(layers.Dense(512, activation='relu')) model.add(layers.Dense(1, activation='sigmoid'))

查看模型：

model.summary()

4. 归一化处理

from keras import optimizers model.compile(loss='binary_crossentropy', optimizer=optimizers.RMSprop(lr=1e-4), metrics=['acc'])

from keras.preprocessing.image import ImageDataGenerator # All images will be rescaled by 1./255 train_datagen = ImageDataGenerator(rescale=1./255) validation_datagen=ImageDataGenerator(rescale=1./255) test_datagen = ImageDataGenerator(rescale=1./255) train_generator = train_datagen.flow_from_directory( # 目标文件目录 train_dir, #所有图片的size必须是150x150 target_size=(150, 150), batch_size=20, # Since we use binary_crossentropy loss, we need binary labels class_mode='binary') validation_generator = test_datagen.flow_from_directory( validation_dir, target_size=(150, 150), batch_size=20, class_mode='binary') test_generator = test_datagen.flow_from_directory(test_dir, target_size=(150, 150), batch_size=20, class_mode='binary')

for data_batch, labels_batch in train_generator: print('data batch shape:', data_batch.shape) print('labels batch shape:', labels_batch.shape) break

报错，原因是没有安装pillow，因为使用load_img()函数需要pillow， 安装pillow库： 重新运行 ：

train_generator.class_indices

0表示微笑，1表示不笑

5. 未进行数据增强时进行训练模型

（1）训练模型

history = model.fit_generator( train_generator, steps_per_epoch=100, epochs=10, validation_data=validation_generator, validation_steps=50)

保存模型：

#保存模型 model.save('genki4k/smileORunsmile_1.h5')

（2）在培训和验证数据上绘制模型的丢失和准确性

import matplotlib.pyplot as plt acc = history.history['acc'] val_acc = history.history['val_acc'] loss = history.history['loss'] val_loss = history.history['val_loss'] epochs = range(len(acc)) plt.plot(epochs, acc, 'bo', label='Training acc') plt.plot(epochs, val_acc, 'b', label='Validation acc') plt.title('Training and validation accuracy') plt.legend() plt.figure() plt.plot(epochs, loss, 'bo', label='Training loss') plt.plot(epochs, val_loss, 'b', label='Validation loss') plt.title('Training and validation loss') plt.legend() plt.show()

不知道哪一步出了问题，图是空的。。。

6. 数据增强

datagen = ImageDataGenerator( rotation_range=40, width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, zoom_range=0.2, horizontal_flip=True, fill_mode='nearest')

import matplotlib.pyplot as plt # This is module with image preprocessing utilities from keras.preprocessing import image fnames = [os.path.join(train_smile_dir, fname) for fname in os.listdir(train_smile_dir)] img_path = fnames[3] img = image.load_img(img_path, target_size=(150, 150)) x = image.img_to_array(img) x = x.reshape((1,) + x.shape) i = 0 for batch in datagen.flow(x, batch_size=1): plt.figure(i) imgplot = plt.imshow(image.array_to_img(batch[0])) i += 1 if i % 4 == 0: break plt.show()

7. 创建网络

model = models.Sequential() model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3))) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(128, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(128, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Flatten()) model.add(layers.Dropout(0.5)) model.add(layers.Dense(512, activation='relu')) model.add(layers.Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer=optimizers.RMSprop(lr=1e-4), metrics=['acc'])

8. 训练模型并保存

#归一化处理 train_datagen = ImageDataGenerator( rescale=1./255, rotation_range=40, width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, zoom_range=0.2, horizontal_flip=True,) # Note that the validation data should not be augmented! test_datagen = ImageDataGenerator(rescale=1./255) train_generator = train_datagen.flow_from_directory( # This is the target directory train_dir, # All images will be resized to 150x150 target_size=(150, 150), batch_size=32, # Since we use binary_crossentropy loss, we need binary labels class_mode='binary') validation_generator = test_datagen.flow_from_directory( validation_dir, target_size=(150, 150), batch_size=32, class_mode='binary') history = model.fit_generator( train_generator, steps_per_epoch=100, epochs=60, validation_data=validation_generator, validation_steps=50)

好像没跑出来。。。 保存模型：

model.save('genki4k/smileORunsmile_2.h5')

三、笑脸识别

1. 单张图片判别

# 单张图片进行判断 是笑脸还是非笑脸 import cv2 from keras.preprocessing import image from keras.models import load_model import numpy as np #加载模型 model = load_model('genki4k/smileORunsmile_2.h5') #本地图片路径 img_path='genki4k/test/smile/file0901.jpg' img = image.load_img(img_path, target_size=(150, 150)) img_tensor = image.img_to_array(img)/255.0 img_tensor = np.expand_dims(img_tensor, axis=0) prediction =model.predict(img_tensor) print(prediction) if prediction[0][0]>0.5: result='非笑脸' else: result='笑脸' print(result)

原图为： （判断正确）

下面多试几次：

2. 摄像头采集人脸识别

import cv2 from keras.preprocessing import image from keras.models import load_model import numpy as np import dlib from PIL import Image model = load_model('genki4k/smileORunsmile_2.h5') detector = dlib.get_frontal_face_detector() video=cv2.VideoCapture(0) font = cv2.FONT_HERSHEY_SIMPLEX def rec(img): gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) dets=detector(gray,1) if dets is not None: for face in dets: left=face.left() top=face.top() right=face.right() bottom=face.bottom() cv2.rectangle(img,(left,top),(right,bottom),(0,255,0),2) img1=cv2.resize(img[top:bottom,left:right],dsize=(150,150)) img1=cv2.cvtColor(img1,cv2.COLOR_BGR2RGB) img1 = np.array(img1)/255. img_tensor = img1.reshape(-1,150,150,3) prediction =model.predict(img_tensor) print(prediction) if prediction[0][0]>0.5: result='unsmile' else: result='smile' cv2.putText(img, result, (left,top), font, 2, (0, 255, 0), 2, cv2.LINE_AA) cv2.imshow('Video', img) while video.isOpened(): res, img_rd = video.read() if not res: break rec(img_rd) if cv2.waitKey(1) & 0xFF == ord('q'): break video.release() cv2.destroyAllWindows()

四、口罩数据集

1. 划分测试集、训练集以及验证集

（1）运行tensorflow环境，导包

import keras Import os,shutil

（2）读取训练集的图片，将训练数据和测试数据放入自己创建的文件夹

# The path to the directory where the original # dataset was uncompressed riginal_dataset_dir = 'D:\mask' # The directory where we will # store our smaller dataset base_dir = 'mask' os.mkdir(base_dir) # Directories for our training, # validation and test splits train_dir = os.path.join(base_dir, 'train') os.mkdir(train_dir) validation_dir = os.path.join(base_dir, 'validation') os.mkdir(validation_dir) test_dir = os.path.join(base_dir, 'test') os.mkdir(test_dir) # Directory with our training smile pictures train_mask_dir = os.path.join(train_dir, 'mask') os.mkdir(train_mask_dir) # Directory with our training unsmile pictures train_unmask_dir = os.path.join(train_dir, 'unmask') os.mkdir(train_unmask_dir) #s.mkdir(train_dogs_dir) # Directory with our validation smile pictures validation_mask_dir = os.path.join(validation_dir, 'mask') os.mkdir(validation_mask_dir) # Directory with our validation unsmile pictures validation_unmask_dir = os.path.join(validation_dir, 'unmask') os.mkdir(validation_unmask_dir) # Directory with our validation smile pictures test_mask_dir = os.path.join(test_dir, 'mask') os.mkdir(test_mask_dir) # Directory with our validation unsmile pictures test_unmask_dir = os.path.join(test_dir, 'unmask') os.mkdir(test_unmask_dir)

（3）复制图片到文件夹中

（4）在jupyter中将文件夹的路径引入

train_mask_dir="mask/train/mask/" train_unmask_dir="mask/train/unmask/" test_mask_dir="mask/test/mask/" test_unmask_dir="mask/test/unmask/" validation_mask_dir="mask/validation/mask/" validation_unmask_dir="mask/validation/unmask/" train_dir="mask/train/" test_dir="mask/test/" validation_dir="mask/validation/"

（5）打印文件夹下的图片数量

print('total training mask images:', len(os.listdir(train_mask_dir))) print('total training unmask images:', len(os.listdir(train_unmask_dir))) print('total testing mask images:', len(os.listdir(test_mask_dir))) print('total testing unmask images:', len(os.listdir(test_unmask_dir))) print('total validation mask images:', len(os.listdir(validation_mask_dir))) print('total validation unmask images:', len(os.listdir(validation_unmask_dir)))

3. 创建模型

#创建模型 from keras import layers from keras import models model = models.Sequential() model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3))) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(128, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(128, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Flatten()) model.add(layers.Dense(512, activation='relu')) model.add(layers.Dense(1, activation='sigmoid'))

查看模型：

model.summary()

4. 归一化处理

from keras import optimizers model.compile(loss='binary_crossentropy', optimizer=optimizers.RMSprop(lr=1e-4), metrics=['acc'])

from keras.preprocessing.image import ImageDataGenerator # All images will be rescaled by 1./255 train_datagen = ImageDataGenerator(rescale=1./255) validation_datagen=ImageDataGenerator(rescale=1./255) test_datagen = ImageDataGenerator(rescale=1./255) train_generator = train_datagen.flow_from_directory( # 目标文件目录 train_dir, #所有图片的size必须是150x150 target_size=(150, 150), batch_size=20, # Since we use binary_crossentropy loss, we need binary labels class_mode='binary') validation_generator = test_datagen.flow_from_directory( validation_dir, target_size=(150, 150), batch_size=20, class_mode='binary') test_generator = test_datagen.flow_from_directory(test_dir, target_size=(150, 150), batch_size=20, class_mode='binary')

for data_batch, labels_batch in train_generator: print('data batch shape:', data_batch.shape) print('labels batch shape:', labels_batch.shape) break

train_generator.class_indices

0表示不戴口罩，1表示戴口罩

5. 训练模型

训练模型：

#花费时间长 history = model.fit_generator( train_generator, steps_per_epoch=100, epochs=10, validation_data=validation_generator, validation_steps=50)

保存模型：

#保存模型 model.save('mask/maskORunmask_1.h5')

6. 数据增强

datagen = ImageDataGenerator( rotation_range=40, width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, zoom_range=0.2, horizontal_flip=True, fill_mode='nearest')

import matplotlib.pyplot as plt from keras.preprocessing import image fnames = [os.path.join(train_mask_dir, fname) for fname in os.listdir(train_mask_dir)] img_path = fnames[3] img = image.load_img(img_path, target_size=(150, 150)) x = image.img_to_array(img) x = x.reshape((1,) + x.shape) i = 0 for batch in datagen.flow(x, batch_size=1): plt.figure(i) imgplot = plt.imshow(image.array_to_img(batch[0])) i += 1 if i % 4 == 0: break plt.show()

7. 创建网络

model = models.Sequential() model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3))) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(128, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(128, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Flatten()) model.add(layers.Dropout(0.5)) model.add(layers.Dense(512, activation='relu')) model.add(layers.Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer=optimizers.RMSprop(lr=1e-4), metrics=['acc'])

8. 训练模型并保存

#归一化处理 train_datagen = ImageDataGenerator( rescale=1./255, rotation_range=40, width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, zoom_range=0.2, horizontal_flip=True,) # Note that the validation data should not be augmented! test_datagen = ImageDataGenerator(rescale=1./255) train_generator = train_datagen.flow_from_directory( # This is the target directory train_dir, # All images will be resized to 150x150 target_size=(150, 150), batch_size=32, # Since we use binary_crossentropy loss, we need binary labels class_mode='binary') validation_generator = test_datagen.flow_from_directory( validation_dir, target_size=(150, 150), batch_size=32, class_mode='binary') history = model.fit_generator( train_generator, steps_per_epoch=100, epochs=60, validation_data=validation_generator, validation_steps=50)

又没跑出来。。。 保存模型：

model.save('mask/maskORunmask_2.h5')

五、口罩识别

1. 单张图片判别

# 单张图片进行判断 是否戴口罩 import cv2 from keras.preprocessing import image from keras.models import load_model import numpy as np model = load_model('mask/maskORunmask_2.h5') img_path='mask/test/unmask/file0791.jpg' img = image.load_img(img_path, target_size=(150, 150)) #print(img.size) img_tensor = image.img_to_array(img)/255.0 img_tensor = np.expand_dims(img_tensor, axis=0) prediction =model.predict(img_tensor) print(prediction) if prediction[0][0]>0.5: result='未戴口罩' else: result='戴口罩' print(result)

原图为： （判断正确）

下面多试几次：

2. 摄像头采集人脸识别

import cv2 from keras.preprocessing import image from keras.models import load_model import numpy as np import dlib from PIL import Image model = load_model('genki4k/smileORunsmile_2.h5') detector = dlib.get_frontal_face_detector() video=cv2.VideoCapture(0) font = cv2.FONT_HERSHEY_SIMPLEX def rec(img): gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) dets=detector(gray,1) if dets is not None: for face in dets: left=face.left() top=face.top() right=face.right() bottom=face.bottom() cv2.rectangle(img,(left,top),(right,bottom),(0,255,0),2) img1=cv2.resize(img[top:bottom,left:right],dsize=(150,150)) img1=cv2.cvtColor(img1,cv2.COLOR_BGR2RGB) img1 = np.array(img1)/255. img_tensor = img1.reshape(-1,150,150,3) prediction =model.predict(img_tensor) print(prediction) if prediction[0][0]>0.5: result='unsmile' else: result='smile' cv2.putText(img, result, (left,top), font, 2, (0, 255, 0), 2, cv2.LINE_AA) cv2.imshow('Video', img) while video.isOpened(): res, img_rd = video.read() if not res: break rec(img_rd) if cv2.waitKey(1) & 0xFF == ord('q'): break video.release() cv2.destroyAllWindows()

不知道是我电脑性能的原因还是其他配置的原因，我的训练模型一直没有完整的跑出来过，总是中途的时候就自己停掉了，幸好不影响后面的摄像头采集人脸识别，可能精确度会低一些吧，也没有找到解决办法。