基于双目视觉和深度学习的无人机人机交互系统

摘要

现行的无人机控制主要依靠专业的设备，由经过专业训练的人来完成。这给无人机的普及以及推广带来了不小的困难。无人机越来越多的应用场景给操控的便捷性带来了越来越高的要求，现有的依靠设备的方法在很多应用场合有着很大的局限性，限制着无人机应用的扩展。为此，本文研究了简化无人机人机交互的方法，该方法在拓展无人机应用方式上有着重要的应用意义。本文完成的工作如下：

搭建了一套无人机实验验证平台。该平台包括了一台多旋翼无人机以及以Nvidia Tegra K1芯片为核心的机载嵌入式信号处理平台。编写了机载嵌入式信号平台与无人机飞行控制系统的接口程序，为后续的无人机控制、导航以及其它类别的应用研究奠定了良好的软硬件实验验证基础。

设计并实现了一种基于双目视觉和深度学习的手势控制无人机方法。首先跟踪并提取人物所在区域，通过立体匹配获取人物和背景均包含的深度图。然后通过归一化并且阈值化的方法，将对动作识别造成干扰的背景去除，从而得到只含有的深度图序列。其次，通过对深度图序列前后两帧差分处理并且利用HSV彩空间按照时间顺序进行彩映射与叠加，将深度图序列转换为同时含有人物动作时间与空间信息的彩纹理图。然后用深度学习方法对所获得的彩纹理图进行训练和分类，从而实现手势指令的识别。由于神经网络的训练对硬件要求极高，因此本方案采用离线训练，在线分类的方式。

最后，构建了一个包含4个指令动作和一个非指令动作的数据集，利用数据集对神经网络进行训练并且进

行了测试。经验证，本文所述方法在室内和室外均可使用，有效控制范围达到10m，可以简化无人机控制复杂度，对促进无人机普及，拓展无人机应用范围都具有一定的参考价值。

关键词：无人机，人机交互，双目视觉，深度学习

ABSTRACT

Traditionally, interacting with UAV（Unmanned Aerial Vehicle）required specialized instrument and well trained operators. In many cases, the instrument based interaction method has been an obstacle in UAV application. In order to reduce the difficulty interacting with UAV. We make usage of the binocular camera on UAV which originally used in obstacle avoidance for motion capture by using depth sensing method. By using deep learning method for motion recognition, we develop a high accuracy human-robot interacting method which is non instrument based. This method reduces the interacting difficulty and has a great sense in experience of interacting with UAV. In our research, we finished following works:

We set up an UAV platform for experimental usage. The platform includes an milticopter and an embedded signal processing platform equipped with NVidia Tegra K1 processor. We wrote an API which allow us control UAV from embedded signal processing platform. This has been the basis of U

AV control, navigation and other further study.

偏心井口

We designed and realize an interacting method with UAV based on stereo vision and deep learning. Firstly, tracking the people who was allowed to control the UAV and spilt it out. We got depth image which contained both the people and the background. We filtered out the background by normalizing and threadholding the depth image. Secondly, we overlay a series differential depth image. These image is colored by mapping the color and the depth image in HSV color space according the time of image captured to generate a colored texture image which including time and space information at the same time. Finally, we classify the colored texture image using deep learning method and recognized the gesture. We trained the neural network offline and executing the image classification online as the training of neural network required power computer.

Finally, we built a data set containing four commanding gesture and a non-commanding gesture. We trained the neural network using this data set and prove the proposed classification method. The proposed method is robust for both indoor and outdoor situation and is effective in 10 meters. Make significant sense to the popularization of UAV and extend its application field.

KEY WORDS：Unmanned Aerial Vehicle, Human Robot Interact, Stereo Vision, Deep Learning

摘要..................................................................................................................... I ABSTRACT .......................................................................................................... I II 绪论 . (1)

1.1引言 (1)无毒的

1.2国内外研究现状 (2)

1.2.1无人机及其控制 (2)

1.2.2无人机人机交互方法 (3)

1.2.3人类动作数据采集 (5)

1.2.4动作识别 (6)xlr连接器

1.3论文的主要研究内容 (7)

1.4论文结构安排 (7)

基于双目视觉的深度图生成以及处理 (9)

2.1双目摄像头图像的采集 (9)

2.2双目视觉测距原理 (10)

2.3立体相机的标定 (12)

2.4深度图的生成 (13)

2.5立体图像的预处理 (16)煎蛋锅

2.6本章小结 (17)

基于深度学习的动作识别方法 (19)

3.1视频预处理 (19)

3.1.1的跟踪以及区域裁切 (19)

3.1.2彩纹理图序列的生成 (20)

3.2卷积神经网络 (22)

3.2.1 Caffe神经网络框架 (23)

3.2.2 Alexnet网络 (23)

3.3数据集的构建 (24)

3.4神经网络的训练和动作识别 (26)

3.5本章小结 (27)

多旋翼无人机控制 (29)

4.1多旋翼无人机控制 (29)

4.1.1动力模型以及控制方法 (29)

4.1.2无人机自动控制原理 (31)

4.2无人机的外部控制 (32)

4.2.1 ROS机器人操作系统 (32)

4.2.2 Mavros工具包 (33)

4.2.3系统软件架构 (35)

4.3本章小结 (37)小电流选线

实验和数据分析 (39)

5.1硬件平台 (39)

5.1.1总体方案 (39)

5.1.2嵌入式机载处理平台 (40)

5.1.3地面站 (41)

5.2手势识别算法性能比较 (41)

5.3手势识别距离测试 (42)

5.4本章小结 (44)

总结与展望 (45)

预制箱梁6.1本文工作总结 (45)

6.2进一步的工作 (46)

参考文献 (47)

发表论文和参加科研情况说明 (51)

致谢 (53)

本文发布于:2024-09-23 07:27:18，感谢您对本站的认可！

本文链接：https://www.17tex.com/tex/4/320157.html

上一篇：人机交互教学大纲

下一篇：人机交互课程设计

标签：应用控制方法动作平台训练识别

留言与评论（共有 0 条评论）