黑得五彩斑斓：基于PaddleHub实现系列漫画生成器
黑得五彩斑斓：基于PaddleHub实现系列漫画生成器。 仓促做于飞桨黑客松Coding Party 2021年12月18日，北京场
0.题解
世界上有一种黑，虽然颜色单调，但却映射出五彩斑斓的世界。
那就是——
漫画书！
这次我们就来自己做个用视频生成系列漫画的小玩意。
经过实验，使用stylepro_artistic可以得到比较接近的效果，过程是：先提取视频中发生场景切换的帧（用统计直方图的差异来判断），再使用小人书的图片作为风格集，用stylepro_artistic将视频关键帧转换为黑白漫画风格。
不废话，上代码——
1.准备阶段
1.1. 安装paddleHub和预训练模型
# 安装 paddlehub 和 stylepro_artistic
!pip install paddlehub
!hub install stylepro_artistic==1.0.1
1.2. 准备给预训练模型用的风格图片
风格数据使用“漫画风格数据集-人美-水浒传”
地址是： https://aistudio.baidu.com/aistudio/datasetdetail/93870
其中 index.zip是压缩包，解开后会有26个文件夹，每个文件夹里是百余张png格式文件，2k * 1.5k 的分辨率（尺寸根据每本书稍有差别）。
index.txt里面是文件夹对应的分册书名，用处不大。
# 首先解压：
!rm -dfr /home/aistudio/work/style/93870/;mkdir /home/aistudio/work/style/;mkdir /home/aistudio/work/style/93870/;cd /home/aistudio/work/style/93870/ &&unzip /home/aistudio/data/data93870/index.zip
数据文件解压后，会给每本书建立一个文件夹。
#显示一张看看
import cv2
import matplotlib.pyplot as plt
#用open cv读出来，注意要调整3原色的顺序
t= cv2.cvtColor(cv2.imread('/home/aistudio/work/style/93870/05/0013.png'),cv2.COLOR_BGR2RGB )
plt.imshow( t )
plt.axis('off') #不显示坐标轴
plt.show()
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-IeRaO7YL-1641602244866)(output_6_0.png)]
由于原图带文字和边框，所以用作风格训练前要只抠出图画部分。
import cv2
def load_comic_photo(src=''):
img = cv2.imread(src)
(h,w,_) = img.shape
#img = img[56:912, 386:1453] #y0,y1 x0,x1
img = img[int(h*0.06):int(h*0.95), int(w*0.25):int(w*0.95)]
return img
#显示一张看看
import cv2
import matplotlib.pyplot as plt
t=load_comic_photo('/home/aistudio/work/style/93870/05/0013.png')
t = cv2.cvtColor(t,cv2.COLOR_BGR2RGB)
plt.imshow( t , cmap='gray')
plt.axis('off')
plt.show()
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2aSa05eG-1641602244866)(output_9_0.png)]
1.3. 区分关键帧
基本思路是如果一帧图像与它上一帧发生很大变化，就可以认为切换了场景。那么就需要一个算法来计算2个图片的差异大小，这里简单的统计B通道明暗统计分布。
#定义函数计算单通道的直方图的相似值，越小差异越大
def calculate_delta(image1=False, image2=False):
if type(image1)!=type(image2) :
degree = [1e-10]
else:
# 灰度直方图算法
# 计算单通道的直方图的相似值
hist1 = cv2.calcHist([image1], [0], None, [256], [0.0, 255.0])
hist2 = cv2.calcHist([image2], [0], None, [256], [0.0, 255.0])
# 计算直方图的重合度
degree = 0
for i in range(len(hist1)):
if hist1[i] != hist2[i]:
degree = degree + \
(1 - abs(hist1[i] - hist2[i]) / max(hist1[i], hist2[i]))
else:
degree = degree + 1
degree = degree / len(hist1)
while type(degree)==type([]):
degree=degree[0]
degree=[degree]
return degree
#找2张图试试
img1 = load_comic_photo('/home/aistudio/work/style/93870/06/0012.png')
plt.imshow( cv2.cvtColor( img1 ,cv2.COLOR_BGR2RGB) )
plt.axis('off')
plt.show()
img2 = load_comic_photo('/home/aistudio/work/style/93870/05/0013.png')
plt.imshow( cv2.cvtColor( img2 ,cv2.COLOR_BGR2RGB) )
plt.axis('off')
plt.show()
print( calculate_delta(img1, img2 ) )
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-e6OpnMEH-1641602244867)(output_11_0.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-5KecAbN8-1641602244867)(output_11_1.png)]
[array([0.7653695], dtype=float32)]
1.4. 预训练模型出场
stylepro_artistic 很容易使用：用 hub.Module(name=“stylepro_artistic”) 就可以加载。
加载后可以调用.style_transfer方法，参数images是元素为dict的list，dict必需的参数有2个：
【1】 ‘content’: 用cv读出来的准备转换的图片；
【2】 ‘styles’: 用cv读出来的用于训练风格的图片list。
import paddlehub as hub
#图片转为漫画
stylepro_artistic = hub.Module(name="stylepro_artistic") #加载预训练模型
def get_cartoon_img(src_img_data):
result = stylepro_artistic.style_transfer(
images=[{
'content': src_img_data ,
'styles': [load_comic_photo('/home/aistudio/work/style/93870/05/0013.png')
,load_comic_photo('/home/aistudio/work/style/93870/07/0015.png')
]
}]
#,use_gpu=True
)
t = result[0]['data']
t = cv2.cvtColor( t,cv2.COLOR_RGB2GRAY) # 先要转换为灰度图片
#ret, t = cv2.threshold(t, 140, 255,cv2.THRESH_BINARY)
t = cv2.medianBlur(t,3)
return t
[2021-12-19 01:50:10,530] [ WARNING] - The _initialize method in HubModule will soon be deprecated, you can use the __init__() to handle the initialization of the object
1.5. 准备处理这个视频：
https://vfx.mtime.cn/Video/2020/11/23/mp4/201123101722987047.mp4
# 下载：
!rm -dfr /home/aistudio/work/mov/;mkdir /home/aistudio/work/mov/;cd /home/aistudio/work/mov/ &&wget https://vfx.mtime.cn/Video/2020/11/23/mp4/201123101722987047.mp4
--2021-12-19 00:51:40-- https://vfx.mtime.cn/Video/2020/11/23/mp4/201123101722987047.mp4
Resolving vfx.mtime.cn (vfx.mtime.cn)... 222.85.26.239
Connecting to vfx.mtime.cn (vfx.mtime.cn)|222.85.26.239|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3021817 (2.9M) [video/mp4]
Saving to: ‘201123101722987047.mp4’
201123101722987047. 100%[===================>] 2.88M 11.0MB/s in 0.3s
2021-12-19 00:51:41 (11.0 MB/s) - ‘201123101722987047.mp4’ saved [3021817/3021817]
1.6.干活的函数
具体操作时，先把视频拆成连续的图片序列，利用前面的calculate_delta计算相邻两帧的差异，如果差异很大且间隔时间不是过分的短，那么把切换场景后的第一帧漫画化。
get_video_key_frame的定义包括4个参数
【1】video_src：视频文件路径；
【2】frame_path ：处理后漫画化的关键帧存放的文件夹；
【3】show_key_frame：是否显示处理前的关键帧，默认不显示；
【4】show_comic_key_frame：是否显示处理后的关键帧，默认不显示。
## 将视频转换为图片,抽取关键帧
import os
import cv2
#显示图片
def showimg(img, isgray=False):
plt.axis("off")
if isgray == True:
plt.imshow(img, cmap='gray')
else:
plt.imshow(img)
plt.show()
def get_video_key_frame(video_src,frame_path='',show_key_frame=False,show_comic_key_frame=False):
if frame_path and not os.path.exists(frame_path):
os.mkdir(frame_path)
vc = cv2.VideoCapture(video_src)
i_imp =-1000 #上一次保存的帧号
(frame_last,delta_last) =(False,0)#上一帧图、上一帧的差异指数
if vc.isOpened():
rval, frame = vc.read()
else:
rval = False
i = 0 #当前帧号
while rval:
rval, frame = vc.read()
if rval:
#cv2.imwrite(frame_path+'{}.jpg'.format(i), frame)
delta = calculate_delta(frame_last, frame )[0] # 计算差异程度
pct = abs(delta - delta_last)/(delta+0.000001) #差异程度的变化量
#print( (i,delta,pct) )
#if delta_last<0.9 and pct>0.5:
if pct>0.2 and i-i_imp>10: #差异程度变化很大，且间隔半秒以上
#print( (i,delta,pct) )
if show_key_frame:
showimg( cv2.cvtColor( frame ,cv2.COLOR_BGR2RGB) , isgray=True)
if frame_path:
cv2.imwrite(frame_path+'{}.jpg'.format(i), frame)
if show_comic_key_frame:
frame_cartoon = get_cartoon_img(frame)
showimg( frame_cartoon, isgray=True)
if frame_path:
cv2.imwrite(frame_path+'{}_c.jpg'.format(i), frame_cartoon)
i_imp = i
(frame_last,delta_last) = ( frame , delta )
i += 1
print('视频帧数：{}'.format(i))
FPS = vc.get(cv2.CAP_PROP_FPS)
SIZE = (int(vc.get(cv2.CAP_PROP_FRAME_WIDTH)),
int(vc.get(cv2.CAP_PROP_FRAME_HEIGHT)))
print('FPS:{}, SIZE:{}'.format(FPS,SIZE))
2.正式开始
2.1. 先看一下抽取关键帧的成效：
# 可以试试了
video = 'work/mov/201123101722987047.mp4'
frame_path = 'work/mov/picture/'
# 抽取关键帧
get_video_key_frame(video,frame_path,show_key_frame=True,show_comic_key_frame=False)
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-up19Awts-1641602244867)(output_19_0.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-6Ps2qxsK-1641602244867)(output_19_1.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-w17AoOhN-1641602244868)(output_19_2.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-sPaQpUax-1641602244868)(output_19_3.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-HKxnRxby-1641602244868)(output_19_4.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-09zHnuXK-1641602244868)(output_19_5.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-DgDdL4vq-1641602244869)(output_19_6.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-1W6ZbkWl-1641602244869)(output_19_7.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-eF6qukjE-1641602244869)(output_19_8.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fmhi5EGy-1641602244869)(output_19_9.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-G5Zm7Wyp-1641602244869)(output_19_10.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ygOV3mvd-1641602244870)(output_19_11.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-lbWvqOVL-1641602244870)(output_19_12.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-U2kP4UJr-1641602244870)(output_19_13.png)]
视频帧数：359
FPS:24.0, SIZE:(1280, 720)
2.2. 再看看转成漫画的效果：
# 关键帧转为漫画
get_video_key_frame(video,frame_path,show_key_frame=False,show_comic_key_frame=True)
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-BvYGg4fo-1641602244870)(output_21_0.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-olstMfEz-1641602244871)(output_21_1.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-0wU8vcK9-1641602244871)(output_21_2.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-1jYtbiSe-1641602244871)(output_21_3.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-QLj0Db5f-1641602244871)(output_21_4.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fGbgiKYG-1641602244871)(output_21_5.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-HBYMgWG8-1641602244872)(output_21_6.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Yg9CyPK5-1641602244872)(output_21_7.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-qysNwsNd-1641602244872)(output_21_8.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-O17D5Fvc-1641602244872)(output_21_9.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-UwGhiX0K-1641602244872)(output_21_10.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-j94XHY9T-1641602244873)(output_21_11.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-HdBc5GLv-1641602244873)(output_21_12.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-um1QX1ff-1641602244873)(output_21_13.png)]
视频帧数：359
FPS:24.0, SIZE:(1280, 720)
3.写在下一版之前
可以看出，抽取的关键帧图片转成漫画风格虽然成功，但是还有一些问题，比如虽然线条转换后有毛笔起落的感觉，但是对大块颜色的处理，采用了用很多小笔触模拟阴影，显得画面混乱。据高手分析，可能是模型网络中存在对细节不大友好的核，导致诡影。另外，也怀疑由于喂给模型的风格数据量过小、次数却很多，有过拟合的可能。结合整个处理过程，总结有3个方向值得改进：
1 现有的关键帧抽取算法简单粗暴，可以改用深度学习，让选择出的帧更接近人类理解的“关键帧”。
2 对于转换效果，可以调整模型网络，增加风格训练数据，达到减少残影和冗余笔划的效果，让输出更接近“小人书”风格。
3 可以使用短视频预测标签的形式，生成图片对应的文字说明，输出完整的“小人书”。
============= 我是安静的华丽分割线，你看不见我，看不见我 ===========================
以下为各种实验，请忽略
!ls /home/aistudio/data
#https://aistudio.baidu.com/aistudio/projectdetail/3260649
data93870 data94144
!ls /home/aistudio/work
!mkdir /home/aistudio/external-libraries
!pip install beautifulsoup4 -t /home/aistudio/external-libraries
# -t /home/aistudio/external-libraries
!hub install stylepro_artistic==1.0.1
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import MutableMapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Iterable, Mapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Sized
[2021-12-18 21:44:19,362] [ INFO] - Module stylepro_artistic-1.0.1 already installed in /home/aistudio/.paddlehub/modules/stylepro_artistic
import sys
sys.path.append('/home/aistudio/external-libraries')
import paddlehub as hub
import cv2
stylepro_artistic = hub.Module(name="stylepro_artistic")
def load_comic_photo(src=''):
#src = '/home/aistudio/work/s1/07/0011.png'
img = cv2.imread(src)
(h,w,_) = img.shape
#img = cv2.cvtColor(img,cv2.COLOR_RGB2GRAY)
#img = cv2.cvtColor(img,cv2.COLOR_GRAY2RGB)
#img = img[56:912, 386:1453] #y0,y1 x0,x1
img = img[int(h*0.06):int(h*0.95), int(w*0.25):int(w*0.95)]
return img
def get_caton_img(src_img_data):
result = stylepro_artistic.style_transfer(
images=[{
'content': src_img_data ,
'styles': [load_comic_photo('/home/aistudio/work/s1/05/0013.png')
,load_comic_photo('/home/aistudio/work/s1/07/0015.png')
]
}])
#print(result)
#import matplotlib.pyplot as plt
#from matplotlib.image import imread
#plt.imshow(imread(picture))
t = result[0]['data']
t = cv2.cvtColor( t,cv2.COLOR_RGB2GRAY) # 先要转换为灰度图片
#ret, t = cv2.threshold(t, 140, 255,cv2.THRESH_BINARY) # 这里的第二个参数要调，是阈值！！
t = cv2.medianBlur(t,3)
#plt.imshow( t , cmap='gray')
#plt.axis('off')
#plt.show()
#cv2.imwrite('out.jpg',result[0]['data'])
return t
#################
src = '/home/aistudio/work/pexels-photo-8055848.jpeg'
src = '/home/aistudio/output/0_1639814440.jpg'
#src = '/home/aistudio/work/8a02601e-24a7-428f-9fe4-12b5f0cc2909.jpg'
#src = '/home/aistudio/work/1f81fa79-b46e-4120-9bf4-91714a8a93f9.jpg'
#src = '/home/aistudio/work/work/1274696563185580257.jpg'
src = 'work/v1/pics/1.jpg'
src = 'work/v1/pics/130.jpg'
src = 'work/v1/pics/218.jpg'
src = 'work/v1/f4/1.jpg'
src = 'work/v1/f4/197.jpg'
img = cv2.imread(src)
#img = cv2.cvtColor(img,cv2.COLOR_RGB2GRAY)
#img = cv2.cvtColor(img,cv2.COLOR_GRAY2RGB)
result = stylepro_artistic.style_transfer(
images=[{
'content': img ,
'styles': [load_comic_photo('/home/aistudio/work/s1/05/0013.png')
,load_comic_photo('/home/aistudio/work/s1/07/0015.png')
]
}])
#print(result)
import matplotlib.pyplot as plt
from matplotlib.image import imread
#plt.imshow(imread(picture))
t = result[0]['data']
t = cv2.cvtColor( t,cv2.COLOR_RGB2GRAY) # 先要转换为灰度图片
#ret, t = cv2.threshold(t, 140, 255,cv2.THRESH_BINARY) # 这里的第二个参数要调，是阈值！！
t = cv2.medianBlur(t,3)
plt.imshow( t , cmap='gray')
plt.axis('off')
plt.show()
cv2.imwrite('out.jpg',result[0]['data'])
[2021-12-18 23:18:16,073] [ WARNING] - The _initialize method in HubModule will soon be deprecated, you can use the __init__() to handle the initialization of the object
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-EiIAkyOK-1641602244873)(output_29_1.png)]
True
#dir(stylepro_artistic)
stylepro_artistic.save_inference_model(dirname='work/v1/pics/m/', model_filename=None, params_filename=None, combined=False)
#sa = hub.Module(directory='work/v1/pics/m/')
sa = hub.Module(name="stylepro_artistic")
sa.load( 'work/v1/pics/m/' )
[2021-12-18 22:10:53,644] [ WARNING] - The _initialize method in HubModule will soon be deprecated, you can use the __init__() to handle the initialization of the object
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
/tmp/ipykernel_100/3943064399.py in <module>
3 #sa = hub.Module(directory='work/v1/pics/m/')
4 sa = hub.Module(name="stylepro_artistic")
----> 5 sa.load( 'work/v1/pics/m/' )
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/compat/paddle_utils.py in runner(*args, **kwargs)
218 def runner(*args, **kwargs):
219 with static_mode_guard():
--> 220 return func(*args, **kwargs)
221
222 return runner
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/module/module.py in load(cls, directory)
193 basename = os.path.split(directory)[-1]
194 dirname = os.path.join(*list(os.path.split(directory)[:-1]))
--> 195 py_module = utils.load_py_module(dirname, '{}.module'.format(basename))
196
197 for _item, _cls in inspect.getmembers(py_module, inspect.isclass):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/utils/utils.py in load_py_module(python_path, py_module_name)
247 sys.modules.pop(py_module_name)
248
--> 249 py_module = importlib.import_module(py_module_name)
250 sys.path.pop(0)
251
/opt/conda/envs/python35-paddle120-env/lib/python3.7/importlib/__init__.py in import_module(name, package)
125 break
126 level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)
128
129
/opt/conda/envs/python35-paddle120-env/lib/python3.7/importlib/_bootstrap.py in _gcd_import(name, package, level)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/importlib/_bootstrap.py in _find_and_load(name, import_)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)
ModuleNotFoundError: No module named 'm.module'
import paddlehub as hub
import cv2
def load_comic_photo(src=''):
#src = '/home/aistudio/work/s1/07/0011.png'
img = cv2.imread(src)
(h,w,_) = img.shape
#img = cv2.cvtColor(img,cv2.COLOR_RGB2GRAY)
#img = cv2.cvtColor(img,cv2.COLOR_GRAY2RGB)
#img = img[56:912, 386:1453] #y0,y1 x0,x1
img = img[int(h*0.06):int(h*0.95), int(w*0.25):int(w*0.95)]
return img
import matplotlib.pyplot as plt
t=load_comic_photo('/home/aistudio/work/s1/05/0013.png')
t = cv2.cvtColor(t,cv2.COLOR_RGB2GRAY) # 先要转换为灰度图片
ret, t = cv2.threshold(t, 140, 205,cv2.THRESH_BINARY) # 这里的第二个参数要调，是阈值！！
plt.imshow( t , cmap='gray')
plt.axis('off')
plt.show()
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-jhguNJol-1641602244873)(output_31_0.png)]
#print(result)
import matplotlib.pyplot as plt
from matplotlib.image import imread
#plt.imshow(imread(picture))
#.convert('L')
t = result[0]['data']
#ret, t = cv2.threshold(t, 160, 255,cv2.THRESH_BINARY) # 这里的第二个参数要调，是阈值！！
t = cv2.medianBlur(t,3)
t = cv2.cvtColor(t,cv2.COLOR_RGB2GRAY) # 先要转换为灰度图片
plt.imshow( t , cmap='gray')
#plt.axis('off')
plt.show()
# 1 只转灰度，之前去噪（不去也可） 2 二值化，略高些才能留下轮廓，但是需要去噪 3，先2值化，会有杂色，转成灰度去噪
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2uBX1ygL-1641602244874)(output_32_0.png)]
from PIL import Image
import matplotlib.pyplot as plt
import numpy as np
#获取图片
def getimg():
return Image.open('/home/aistudio/work/pexels-photo-8055848.jpeg')
#显示图片
def showimg(img, isgray=False):
plt.axis("off")
if isgray == True:
plt.imshow(img, cmap='gray')
else:
plt.imshow(img)
plt.show()
#showimg(getimg(), True)
im = getimg()
im_gray = im.convert('L')
showimg(im_gray, True)
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-R9ZdrpQh-1641602244874)(output_33_0.png)]
from cv2 import cv2
import os
def resize(image, width=None, height=None, inter=cv2.INTER_AREA):
dim = None
(h, w) = image.shape[:2]
if width is None and height is None:
return image
if width is None:
r = height / float(h)
dim = (int(w * r), height)
else:
r = width / float(w)
dim = (width, int(h * r))
resized = cv2.resize(image, dim, interpolation=inter)
return resized
def rgb_to_sketch(pic_path):
img_rgb = cv2.imread(pic_path)
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)##转为灰度图
##可改变图片的尺寸，只设置h或者只设置w就会按照等比例放大或缩小；若都指定，则按照指定的大小变换
img_gray=resize(img_gray, height = 500)
img_blur = cv2.GaussianBlur(img_gray, ksize=(21, 21), sigmaX=0, sigmaY=0)##高斯滤波
##可改变scale的值来改变生成效果，scale越小，会让一些原本比较亮的区域变得更加清晰
img_blend = cv2.divide(img_gray, img_blur, scale=225)##图像相除实现素描效果
return img_blend
f= r'work/1274696563185580257.jpg'#r'work/1f81fa79-b46e-4120-9bf4-91714a8a93f9.jpg'#r'work/8a02601e-24a7-428f-9fe4-12b5f0cc2909.jpg'#r'work/pexels-photo-8055848.jpeg'
t = rgb_to_sketch(f)
ret, t = cv2.threshold(t, int(t.mean()), 255,cv2.THRESH_BINARY)
t = cv2.medianBlur(t,3)
cv2.imwrite(f+r'.output.jpg', t)
True
t.mean()
198.83404
import paddlehub as hub
import cv2
m7 = hub.Module(name='animegan_v1_hayao_60')
# 模型预测
result = m7.style_transfer(images=[cv2.imread('work/pexels-photo-8055848.jpeg')],visualization=True)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import MutableMapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Iterable, Mapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Sized
[2021-12-18 15:59:20,648] [ WARNING] - The _initialize method in HubModule will soon be deprecated, you can use the __init__() to handle the initialization of the object
W1218 15:59:20.654083 9756 analysis_predictor.cc:1353] Deprecated. Please use CreatePredictor instead.
!rm -f work/v1/pics/*.*
!rm -f work/v2/08/*.*
## 将视频转换为图片
import os
import cv2
# 原视频读取
video = 'work/v1/201123101722987047.mp4'#'work/v1/v.f42905.mp4'#'work/v1/211217084858556132.mp4'#'work/v1/d1.mp4'#'work/v1/m2.mp4'#'work/v1/f5.mp4'#'work/v1/olds.mp4'#'work/v2/MTV_08.mp4'#
frame_path = 'work/v1/201123101722987047/'#'work/v1/v.f42905/'#'work/v1/211217084858556132/'#'work/v1/d1/'#'work/v1/m2/'#'work/v1/f5/'#'work/v1/pics/' # 'work/v2/08/' #
dog = 1
if not os.path.exists(frame_path):
os.mkdir(frame_path)
vc = cv2.VideoCapture(video)
i_imp =-1000
(frame_last,delta_last) =(False,0)
if vc.isOpened():
rval, frame = vc.read()
else:
rval = False
i = 0
while rval:
rval, frame = vc.read()
if rval:
#cv2.imwrite(frame_path+'{}.jpg'.format(i), frame)
delta = calculate(frame_last, frame )[0]
pct = abs(delta - delta_last)/(delta+0.000001)
#print( (i,delta,pct) )
#if delta_last<0.9 and pct>0.5:
if pct>0.2 and i-i_imp>10:
print( (i,delta,pct) )
showimg(frame, isgray=True)
cv2.imwrite(frame_path+'{}.jpg'.format(i), frame)
if dog>0:
frame_caton = get_caton_img(frame)
showimg( frame_caton, isgray=True)
cv2.imwrite(frame_path+'{}_c.jpg'.format(i), frame_caton)
i_imp = i
(frame_last,delta_last) = ( frame , delta )
i += 1
print('视频帧数：{}'.format(i))
FPS = vc.get(cv2.CAP_PROP_FPS)
SIZE = (int(vc.get(cv2.CAP_PROP_FRAME_WIDTH)),
int(vc.get(cv2.CAP_PROP_FRAME_HEIGHT)))
print('FPS:{}, SIZE:{}'
e)
cv2.imwrite(frame_path+'{}_c.jpg'.format(i), frame_caton)
i_imp = i
(frame_last,delta_last) = ( frame , delta )
i += 1
print('视频帧数：{}'.format(i))
FPS = vc.get(cv2.CAP_PROP_FPS)
SIZE = (int(vc.get(cv2.CAP_PROP_FRAME_WIDTH)),
int(vc.get(cv2.CAP_PROP_FRAME_HEIGHT)))
print('FPS:{}, SIZE:{}'
.format(FPS,SIZE))
(1, array([0.9369093], dtype=float32), array([0.9999989], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-7EuznbhE-1641602244874)(output_37_1.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-pnynssML-1641602244874)(output_37_2.png)]
(23, array([0.16196954], dtype=float32), array([4.8128796], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Fuy9mLcQ-1641602244874)(output_37_4.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-FGAfVKI8-1641602244875)(output_37_5.png)]
(58, array([0.24561134], dtype=float32), array([1.8473916], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-pwdXz0vk-1641602244875)(output_37_7.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Xx3dxiIE-1641602244875)(output_37_8.png)]
(90, array([0.48886403], dtype=float32), array([0.8467305], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Kj8oM5ei-1641602244875)(output_37_10.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-8XP4KU1B-1641602244875)(output_37_11.png)]
(122, array([0.32180133], dtype=float32), array([1.6550589], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-0aSQZw3p-1641602244876)(output_37_13.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-S8YLwb7Q-1641602244876)(output_37_14.png)]
(143, array([0.3552464], dtype=float32), array([1.4872069], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-28CTZjSP-1641602244876)(output_37_16.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ahc4VZiH-1641602244876)(output_37_17.png)]
(173, array([0.32388538], dtype=float32), array([1.9440843], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-U0RXkMYy-1641602244876)(output_37_19.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-IHEXKy9d-1641602244877)(output_37_20.png)]
(195, array([0.3770919], dtype=float32), array([1.3326628], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fzI5iZDO-1641602244877)(output_37_22.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-8qEFKv8l-1641602244877)(output_37_23.png)]
(218, array([0.48482153], dtype=float32), array([0.7061973], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-P6xNRUgy-1641602244877)(output_37_25.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-nJKbuzVc-1641602244877)(output_37_26.png)]
(234, array([0.34026635], dtype=float32), array([1.32831], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-5BAEFchw-1641602244877)(output_37_28.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-buT6rjlI-1641602244878)(output_37_29.png)]
(245, array([0.8021379], dtype=float32), array([0.79289573], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-szUZoMf5-1641602244878)(output_37_31.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-C5ox4rpm-1641602244878)(output_37_32.png)]
(259, array([0.3227935], dtype=float32), array([1.5863717], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-L7T2Y5t5-1641602244878)(output_37_34.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-HYwVjnhe-1641602244878)(output_37_35.png)]
(318, array([0.50448614], dtype=float32), array([0.8998087], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-WDlyLTk5-1641602244879)(output_37_37.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ALX0bCG1-1641602244879)(output_37_38.png)]
(346, array([0.41244707], dtype=float32), array([1.2286121], dtype=float32))
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ibi3x4Za-1641602244879)(output_37_40.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-9Xmdqw2H-1641602244879)(output_37_41.png)]
视频帧数：359
FPS:24.0, SIZE:(1280, 720)
from tqdm import tqdm
#import paddlevideo as pv
import paddlehub as hub
videotag = hub.Module(name="videotag_tsn_lstm")
# execute predict and print the result
results = videotag.classify(paths=["work/mov/201123101722987047.mp4"], use_gpu=False)
print(results)
学大模型，用大模型上飞桨星河社区！每天8点V100G算力免费领！免费领取ERNIE 4.0 100w Token >>>
更多推荐
0
0- 0
已为社区贡献1438条内容
所有评论(0)