AIstudio-第二期-基于MoviNet检测视频中危险暴力行为

本项目根据MoviNet检测视频或是摄像头中的内容检测是否包含可能的暴力行为。帮助相关人员及时的了解可能的危险行为，避免事态的进一步加剧。

视频分类历史

互联网上图像和视频的规模日益增长，对于视频网站而言每天就有几百小时的视频产生，这使得人们急需研究视频相关算法帮助人们更加容易得找到或是过滤内容，这些视频分类算法能自动分析视频所包含的场景或是行为信息，并进行理解。同时对视频进行自动标准，分类和描述。
视频分类的主要目标是理解视频中所包含的内容，确定视频对应的关键主题。视频分类（Video Classification）不仅仅要理解视频中的每一帧图像，同时需要识别出能够描述视频内容的关键主题。
多种特征融合如文本-图像融合，声音-视频融合对视频分类的性能或许有所帮助，但本文主要关注和研究是基于时间和画面的特征，也称作基于视觉的视频分类。

在这里插入图片描述

传统视频分类方法

在深度学习方法广泛应用之前，大多数的视频分类方法采用基于人工设计的特征和典型的机器学习方法研究行为识别和事件检测。
传统的视频分类研究专注于采用对局部时空区域的运动信息和表观（Appearance）信息编码的方式获取视频描述符，然后利用词袋模型（Bag of Words）等方式生成视频编码，最后利用视频编码来训练分类器（如SVM），区分视频类别。视频的描述符依赖人工设计的特征，如使用运动信息获取局部时空特征的梯度直方图（Histogram of Oriented Gradients，HOG），使用不同类型的轨迹的光流直方图（Histogram of Optical Flow, HOF）和运动边界直方图（Motion Boundary Histogram，MBH）。通过词袋模型或Fisher向量方法，这些特征可以生成视频编码。
当前，基于轨迹的方法（尤其是DT和IDT）是最高水平的人工设计特征算法的基础。许多研究者正在尝试改进IDT，如通过增加字典的大小和融合多种编码方法，通过开发子采样方法生成DT特征的字典，在许多人体行为数据集上取得了不错的性能。
然而，随着深度神经网络的兴起，特别是CNN、LSTM、GRU等在视频分类中的成功应用，其分类性能逐渐超越了基于DT和IDT的传统方法，使得这些传统方法逐渐淡出了人们的视野
在这里插入图片描述

深度网络方法研究

基于深度神经网络的兴起为解决大规模视频分类问题提供了新的思路和方法。在深度神经网络中，卷积神经网络（Covolution Neural Network，CNN）在处理图像相关的信息上获取了巨大的成就。卷积神经网络在静态图像识别问题中取得了空前的成功，其中包括MNIST、CIFAR和ImageNet大规模视觉识别挑战问题。CNN采用卷积与池化操作，可以自动学习图像中包含的复杂特征，在视觉对象识别任务中表现出很好的性能。基于CNN这些研究成果，国内外开始研究将CNN等深度网络应用到视频和行为分类任务中。
和图像识别相比，视频分类中视频比静态图像可以提供更多的信息，包括随着时间变化的行为信息等。视频中包含成百上千帧图像，但通常而言并不是所有图像都有用，处理这些图像需要大量计算。通常方法是将视频帧视为静态图像，按顺序随机抽样应用CNN识别每一帧。

在这里插入图片描述

项目介绍

背景

在公共场合斗殴，欺凌此类悲剧还是不停的在我们的身边重演，这种恶劣行为对于当今社会的稳定性有着非常大的影响，
在我设计这个项目途中还发生了如唐山打人案这种举国震惊的事件。对这种情况如果我们有方法
在悲剧发生的途中或是之后立刻由警方进行干预，可以极大程度的避免事态加剧或是犯人逃亡。
但传统的依赖人工报警具有非常高的不确定性和延迟性，
周遭的监控设备虽然可以实时记录但仍是被动记录为主，大多数情况下只能作为事后发生的证据。如果需要通过视频画面了解风险或是即时的阻止悲剧的发生，
现阶段主流的方式还是安排专业人员盯着显示器进行判断。然而，这种方式也只能应用于专门监控线路或是特殊情况，对应浩如烟海的通用监控网络而言，
采用人力监控的方式显然是不可及。在这里插入图片描述

因此，我的项目便是争对这一问题设计出来能够能在嵌入式平台上进行分析和预测可能的暴力行为，并及时的发送提醒给有关人员进行干预。在模型方面，为了适配jetson Nano不算充分的算力，采用了最新的轻量化网络MoviNet（timesformer之类的大型网络效果会更好），同时建立了一套完整的前后端逻辑以便用户使用

设计

设计流程图：

在这里插入图片描述

安装对应的requirement

pip install -r paddleViolenceDection/PaddleVideo/requirements.txt

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r paddleViolenceDection/PaddleVideo/requirements.txt (line 1)) (1.19.5)
Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r paddleViolenceDection/PaddleVideo/requirements.txt (line 2)) (1.1.5)
Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r paddleViolenceDection/PaddleVideo/requirements.txt (line 3)) (4.27.0)
Requirement already satisfied: PyYAML>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r paddleViolenceDection/PaddleVideo/requirements.txt (line 4)) (5.1.2)
Collecting opencv-python==4.2.0.32
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/34/a3/403dbaef909fee9f9f6a8eaff51d44085a14e5bb1a1ff7257117d744986a/opencv_python-4.2.0.32-cp37-cp37m-manylinux1_x86_64.whl (28.2 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m28.2/28.2 MB[0m [31m8.3 MB/s[0m eta [36m0:00:00[0m:00:01[0m00:01[0m
[?25hCollecting decord==0.4.2
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c0/0c/7d99cfcde7b85f80c9ea9b0b19441339ad3cef59ee7fa5386598db714efe/decord-0.4.2-py2.py3-none-manylinux1_x86_64.whl (11.8 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m11.8/11.8 MB[0m [31m12.2 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m
[?25hCollecting av==8.0.3
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/66/ff/bacde7314c646a2bd2f240034809a10cc3f8b096751284d0828640fff3dd/av-8.0.3-cp37-cp37m-manylinux2010_x86_64.whl (37.2 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m37.2/37.2 MB[0m [31m6.4 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0mm
[?25hCollecting scipy==1.6.3
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7d/e8/43ffca541d2f208d516296950b25fe1084b35c2881f4d444c1346ca75815/scipy-1.6.3-cp37-cp37m-manylinux1_x86_64.whl (27.4 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m27.4/27.4 MB[0m [31m5.4 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m
[?25hCollecting scikit-image
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2d/ba/63ce953b7d593bd493e80be158f2d9f82936582380aee0998315510633aa/scikit_image-0.19.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (13.5 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m13.5/13.5 MB[0m [31m6.4 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m
[?25hRequirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas->-r paddleViolenceDection/PaddleVideo/requirements.txt (line 2)) (2.8.2)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas->-r paddleViolenceDection/PaddleVideo/requirements.txt (line 2)) (2019.3)
Requirement already satisfied: networkx>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r paddleViolenceDection/PaddleVideo/requirements.txt (line 9)) (2.4)
Requirement already satisfied: pillow!=7.1.0,!=7.1.1,!=8.3.0,>=6.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r paddleViolenceDection/PaddleVideo/requirements.txt (line 9)) (8.2.0)
Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r paddleViolenceDection/PaddleVideo/requirements.txt (line 9)) (21.3)
Requirement already satisfied: imageio>=2.4.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r paddleViolenceDection/PaddleVideo/requirements.txt (line 9)) (2.6.1)
Collecting PyWavelets>=1.1.1
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ae/56/4441877073d8a5266dbf7b04c7f3dc66f1149c8efb9323e0ef987a9bb1ce/PyWavelets-1.3.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.4 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m6.4/6.4 MB[0m [31m8.3 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0mm
[?25hCollecting tifffile>=2019.7.26
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d8/38/85ae5ed77598ca90558c17a2f79ddaba33173b31cf8d8f545d34d9134f0d/tifffile-2021.11.2-py3-none-any.whl (178 kB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m178.9/178.9 KB[0m [31m5.6 MB/s[0m eta [36m0:00:00[0m
[?25hRequirement already satisfied: decorator>=4.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from networkx>=2.2->scikit-image->-r paddleViolenceDection/PaddleVideo/requirements.txt (line 9)) (4.4.2)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging>=20.0->scikit-image->-r paddleViolenceDection/PaddleVideo/requirements.txt (line 9)) (3.0.8)
Requirement already satisfied: six>=1.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from python-dateutil>=2.7.3->pandas->-r paddleViolenceDection/PaddleVideo/requirements.txt (line 2)) (1.16.0)
Installing collected packages: av, tifffile, scipy, PyWavelets, opencv-python, decord, scikit-image
  Attempting uninstall: scipy
    Found existing installation: scipy 1.3.0
    Uninstalling scipy-1.3.0:
      Successfully uninstalled scipy-1.3.0
  Attempting uninstall: opencv-python
    Found existing installation: opencv-python 4.1.1.26
    Uninstalling opencv-python-4.1.1.26:
      Successfully uninstalled opencv-python-4.1.1.26
[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
parl 1.4.1 requires pyzmq==18.1.1, but you have pyzmq 22.3.0 which is incompatible.[0m[31m
[0mSuccessfully installed PyWavelets-1.3.0 av-8.0.3 decord-0.4.2 opencv-python-4.2.0.32 scikit-image-0.19.3 scipy-1.6.3 tifffile-2021.11.2
[33mWARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the '/opt/conda/envs/python35-paddle120-env/bin/python -m pip install --upgrade pip' command.[0m[33m
[0mNote: you may need to restart the kernel to use updated packages.

MoviNet 网络介绍

3D卷积网络CNN在视屏识别方面的准确率是有目共睹的，但通常需要大量的计算和内存预算，此类特性是此类型网络很难在移动设备上进行工作或是在线推理。另一方面，2D卷积网络需要的资源则少的多，可以使用框架进行在线推理允许，但精度上有所不足。
为了克服以上缺点，MoviNet定义了一个搜索空间，以允许神经体系结构搜索能有效的全很时空特征表示。然后，MoviNet还设有引入流缓冲区，能以小的连续的子片段处理视频，在不牺牲长时间依赖性的前提下设定了稳定的内存来实现在线推理。最后MoviNet创建了时间合集，从流缓冲区恢复丢失的精度。

训练数据库介绍

训练数据库名字为“Real Life Violence Situations Dataset”，是由 M.Soliman 等一行人在ICICS‘19 上发布的公开数据集。数据集包含了1000个暴力与非暴力的视频片段。暴力片段则包含了在不同环境不同情况下的真实街头格斗。非暴力视频则打出取材于一些通常的人类行为如吃饭，运动，走路等。
在这里插入图片描述

模型测试

对包含暴力场景的画面进行测试

from IPython.display import Video

Video("paddleViolenceDection/test/tang2.mp4")
Your browser does not support the video element.

使用以下脚本来快速检测：

bash inference.sh 视频目录

因为模拟嵌入式环境用的是cpu进行推理，可以自行去inference.sh文件进行修改使用GPU

%cd paddleViolenceDection/
!bash inference.sh "test/tang2.mp4"


[Errno 2] No such file or directory: 'paddleViolenceDection/'
/home/aistudio/paddleViolenceDection
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import MutableMapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Iterable, Mapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Sized
No module named 'SimpleITK', [SimpleITK] package and it's dependencies is required for PP-Care.
No module named 'SimpleITK', [SimpleITK] package and it's dependencies is required for PP-Care.
No module named 'lmdb', [lmdb] package and it's dependencies is required for ActBERT.
No module named 'lmdb', [lmdb] package and it's dependencies is required for ActBERT.
[06/13 13:32:59] DALI is not installed, you can improve performance if use DALI
No module named 'ppdet', [paddledet] package and it's dependencies is required for AVA.
Inference model(MoViNet)...
[1m[35m--- Running analysis [ir_graph_build_pass][0m
[1m[35m--- Running analysis [ir_graph_clean_pass][0m
[1m[35m--- Running analysis [ir_analysis_pass][0m
[32m--- Running IR pass [simplify_with_basic_ops_pass][0m
[32m--- Running IR pass [layer_norm_fuse_pass][0m
[37m---    Fused 0 subgraphs into layer_norm op.[0m
[32m--- Running IR pass [attention_lstm_fuse_pass][0m
[32m--- Running IR pass [seqconv_eltadd_relu_fuse_pass][0m
[32m--- Running IR pass [seqpool_cvm_concat_fuse_pass][0m
[32m--- Running IR pass [mul_lstm_fuse_pass][0m
[32m--- Running IR pass [fc_gru_fuse_pass][0m
[37m---    fused 0 pairs of fc gru patterns[0m
[32m--- Running IR pass [mul_gru_fuse_pass][0m
[32m--- Running IR pass [seq_concat_fc_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_squeeze2_matmul_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_reshape2_matmul_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_flatten2_matmul_fuse_pass][0m
[32m--- Running IR pass [matmul_v2_scale_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_v2_to_mul_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_v2_to_matmul_pass][0m
[32m--- Running IR pass [matmul_scale_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_to_mul_pass][0m
[32m--- Running IR pass [fc_fuse_pass][0m
[32m--- Running IR pass [repeated_fc_relu_fuse_pass][0m
[32m--- Running IR pass [squared_mat_sub_fuse_pass][0m
[32m--- Running IR pass [conv_bn_fuse_pass][0m
[32m--- Running IR pass [conv_eltwiseadd_bn_fuse_pass][0m
[32m--- Running IR pass [conv_transpose_bn_fuse_pass][0m
[32m--- Running IR pass [conv_transpose_eltwiseadd_bn_fuse_pass][0m
[32m--- Running IR pass [is_test_pass][0m
[32m--- Running IR pass [runtime_context_cache_pass][0m
[1m[35m--- Running analysis [ir_params_sync_among_devices_pass][0m
[1m[35m--- Running analysis [adjust_cudnn_workspace_size_pass][0m
[1m[35m--- Running analysis [inference_op_replace_pass][0m
[1m[35m--- Running analysis [memory_optimize_pass][0m
I0613 13:33:00.174901 35804 memory_optimize_pass.cc:216] Cluster name : conv3d_85.tmp_0  size: 35500800
I0613 13:33:00.174947 35804 memory_optimize_pass.cc:216] Cluster name : tmp_17  size: 1120000
I0613 13:33:00.174950 35804 memory_optimize_pass.cc:216] Cluster name : data_batch_0  size: 17750400
I0613 13:33:00.174953 35804 memory_optimize_pass.cc:216] Cluster name : batch_norm_2.tmp_2  size: 35500800
I0613 13:33:00.174957 35804 memory_optimize_pass.cc:216] Cluster name : batch_norm_1.tmp_2  size: 2958400
I0613 13:33:00.174968 35804 memory_optimize_pass.cc:216] Cluster name : swish_48.tmp_0  size: 11833600
[1m[35m--- Running analysis [ir_graph_to_program_pass][0m
I0613 13:33:00.384291 35804 analysis_predictor.cc:1007] ======= optimize end =======
I0613 13:33:00.396700 35804 naive_executor.cc:102] ---  skip [feed], feed -> data_batch_0
I0613 13:33:00.403721 35804 naive_executor.cc:102] ---  skip [flatten_0.tmp_0], fetch -> fetch
file lens(1),batch size: 1
start: 0 to end:1
1
====================
Current video file: test/tang2.mp4
视频包含暴力行为
[0m

对有类似情况但非暴力行为进行检测（例如训练）

from IPython.display import Video

deo

Video("/home/aistudio/paddleViolenceDection/test/test3.mp4")
Your browser does not support the video element. 
!bash inference.sh "/home/aistudio/paddleViolenceDection/test/test3.mp4"

/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import MutableMapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Iterable, Mapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Sized
No module named 'SimpleITK', [SimpleITK] package and it's dependencies is required for PP-Care.
No module named 'SimpleITK', [SimpleITK] package and it's dependencies is required for PP-Care.
No module named 'lmdb', [lmdb] package and it's dependencies is required for ActBERT.
No module named 'lmdb', [lmdb] package and it's dependencies is required for ActBERT.
[06/13 13:40:33] DALI is not installed, you can improve performance if use DALI
No module named 'ppdet', [paddledet] package and it's dependencies is required for AVA.
Inference model(MoViNet)...
[1m[35m--- Running analysis [ir_graph_build_pass][0m
[1m[35m--- Running analysis [ir_graph_clean_pass][0m
[1m[35m--- Running analysis [ir_analysis_pass][0m
[32m--- Running IR pass [simplify_with_basic_ops_pass][0m
[32m--- Running IR pass [layer_norm_fuse_pass][0m
[37m---    Fused 0 subgraphs into layer_norm op.[0m
[32m--- Running IR pass [attention_lstm_fuse_pass][0m
[32m--- Running IR pass [seqconv_eltadd_relu_fuse_pass][0m
[32m--- Running IR pass [seqpool_cvm_concat_fuse_pass][0m
[32m--- Running IR pass [mul_lstm_fuse_pass][0m
[32m--- Running IR pass [fc_gru_fuse_pass][0m
[37m---    fused 0 pairs of fc gru patterns[0m
[32m--- Running IR pass [mul_gru_fuse_pass][0m
[32m--- Running IR pass [seq_concat_fc_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_squeeze2_matmul_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_reshape2_matmul_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_flatten2_matmul_fuse_pass][0m
[32m--- Running IR pass [matmul_v2_scale_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_v2_to_mul_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_v2_to_matmul_pass][0m
[32m--- Running IR pass [matmul_scale_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_to_mul_pass][0m
[32m--- Running IR pass [fc_fuse_pass][0m
[32m--- Running IR pass [repeated_fc_relu_fuse_pass][0m
[32m--- Running IR pass [squared_mat_sub_fuse_pass][0m
[32m--- Running IR pass [conv_bn_fuse_pass][0m
[32m--- Running IR pass [conv_eltwiseadd_bn_fuse_pass][0m
[32m--- Running IR pass [conv_transpose_bn_fuse_pass][0m
[32m--- Running IR pass [conv_transpose_eltwiseadd_bn_fuse_pass][0m
[32m--- Running IR pass [is_test_pass][0m
[32m--- Running IR pass [runtime_context_cache_pass][0m
[1m[35m--- Running analysis [ir_params_sync_among_devices_pass][0m
[1m[35m--- Running analysis [adjust_cudnn_workspace_size_pass][0m
[1m[35m--- Running analysis [inference_op_replace_pass][0m
[1m[35m--- Running analysis [memory_optimize_pass][0m
I0613 13:40:33.788962 38273 memory_optimize_pass.cc:216] Cluster name : conv3d_85.tmp_0  size: 35500800
I0613 13:40:33.789016 38273 memory_optimize_pass.cc:216] Cluster name : tmp_17  size: 1120000
I0613 13:40:33.789021 38273 memory_optimize_pass.cc:216] Cluster name : data_batch_0  size: 17750400
I0613 13:40:33.789032 38273 memory_optimize_pass.cc:216] Cluster name : batch_norm_2.tmp_2  size: 35500800
I0613 13:40:33.789036 38273 memory_optimize_pass.cc:216] Cluster name : batch_norm_1.tmp_2  size: 2958400
I0613 13:40:33.789039 38273 memory_optimize_pass.cc:216] Cluster name : swish_48.tmp_0  size: 11833600
[1m[35m--- Running analysis [ir_graph_to_program_pass][0m
I0613 13:40:33.998153 38273 analysis_predictor.cc:1007] ======= optimize end =======
I0613 13:40:34.011899 38273 naive_executor.cc:102] ---  skip [feed], feed -> data_batch_0
I0613 13:40:34.018568 38273 naive_executor.cc:102] ---  skip [flatten_0.tmp_0], fetch -> fetch
file lens(1),batch size: 1
start: 0 to end:1
1
====================
Current video file: /home/aistudio/paddleViolenceDection/test/test3.mp4
非暴力行为视频
[0m

jetsonNano 部署

git clone 到nano上指定目录，安装完对应依赖之后，运行python runtime.py启动服务的后端，程序将读取nano的sci相机数据流并进行本地缓存，缓存到指定时间后作为模型的输入。如果模型判断包含指定暴力视频则会存储到指定目录。 程序将持续循环直到手动终止
这个项目同时还有后端，运行 python app.py 可以查看一个基于Flask的网页，在页面上可以对于视频进行查看，归档。
（请务必全部使用python3.6作为环境，其他版本会有读取不了SCI相机的问题）
在这里插入图片描述

参数

在jetson nano上使用cpu 输入10s视频，大小为114x114，推理耗时 5-6s
(jetson 算力和显存严重不足，换成GPU GPU/CPU/内存开销太大，也无法转换成tensorrt加速)
服务器上 AIstudio最低档CPU推理5s视频约2-3秒

之后方向

  • 性能上为了配合嵌入式还有优化的空间，现如今使用的还是float32.
  • 训练数据集不算大，只包含1000多个5s的暴力与非暴力视频

开源链接：https://aistudio.baidu.com/aistudio/projectdetail/4212976

# 人工智能 # 音视频 # 机器学习
Logo
百度飞桨AI Studio社区

学大模型，用大模型上飞桨星河社区！每天8点V100G算力免费领！免费领取ERNIE 4.0 100w Token >>>

更多推荐

cover

利用Amazon Bedrock畅玩Claude 3等多种领先模型，抢占AI高地(体验倒计时4小时)

avatar 百度飞桨星河社区
cover

RAPTOR：索引树状 RAG，使用树结构来捕捉文本的高级和低级细节

avatar 百度飞桨星河社区

RAG介绍

LlamaIndex是一个基于大型语言模型（LLM）的应用程序数据框架，旨在连接LLM与外部数据，特别是私有或特定领域的数据，以实现更准确的文本生成和知识增强。

avatar 百度飞桨星河社区