【作者】 王洋；

【导师】 杨鑫；

【作者基本信息】 大连理工大学 ， 计算机应用技术， 2025， 博士

【摘要】 面向动态运动、资源受限与动态光照等复杂应用场景,传统人工神经网络在时延、能耗与鲁棒性方面难以满足实际需求。脉冲神经网络(Spiking Neural Network,SNN)因其事件驱动特性和生物可解释机制而受到广泛关注。神经形态视觉传感器(如事件相机)以稀疏、异步的事件流提供微秒级时序信息,为SNN的高效计算与可解释决策提供了新的技术路径。然而,当前围绕神经形态视觉数据的SNN研究在实际落地中仍面临三方面关键挑战:(1)多源感知数据在模态异构、时空频率不一致条件下难以统一高效编码与融合;(2)轻量化SNN模型在保证低功耗的同时很难兼顾高准确率;(3)SNN在复杂长时程动态场景中仍存在特征退化、记忆不足与知识遗忘问题。针对上述挑战,本文围绕面向神经形态视觉的SNN模型设计与应用研究展开,分别提出三个具备多模态高效表征、轻量化学习、自适应调控能力的SNN方法框架,以实现低延时、低功耗、高鲁棒的类脑场景理解。本文主要研究成果和创新点如下:(1)基于神经元内在可塑性的多模态高效表征方法。针对SNN在复杂动态环境中多模态感知信号(事件视觉、激光雷达等)跨域差异大、频率不一致的问题,提出事件增强的多模态脉冲决策网络框架,该方法引入事件流作为运动线索与激光深度的几何信息作为互补,通过无监督表征学习的混合脉冲变分自编码器高效提取时空特征;在决策层设计带可学习阈值的中间融合模块,以神经元内在可塑性(可学习阈值机制)实现多模态信息的高效融合。在机器人动态避障任务上做应用验证的实验结果表明,与同期最先进的方法相比,该方法在静态场景下成功率最少提高了2.5%,在动态场景下成功率最少提高了10.8%。(2)基于多模态协同知识蒸馏的脉冲神经网络轻量化学习方法。针对SNN在资源受限场景下难以同时满足能效与准确性需求的问题,提出多模态协同知识蒸馏的轻量级脉冲神经网络模型。该方法结合卷积脉冲神经网络与多模态协同知识蒸馏机制,通过跨模态眼动信号时空蒸馏的方式,将高精度教师网络的关键时空特征迁移至轻量级全脉冲学生网络,从而在显著降低参数量与计算复杂度的同时保持高准确率。在眼动交互任务上做应用验证的实验结果表明,与同期最先进的方法SEEN相比,该方法在计算量降低95%的同时,在四种复杂光照场景下的准确率上平均提升了3.4%,实时推理能力提升了65.1%。(3)基于Transformer的脉冲神经网络尖峰强度自适应调控方法。针对SNN在复杂长时程视觉任务中出现的特征退化与知识遗忘问题,提出基于Transformer的全脉冲跟踪框架。该方法在训练阶段引入单模版帧和多搜索帧交互的训练方式增强长时依赖特征的捕获能力,并设计尖峰强度自适应调控模块,实现对动态长时信息的有效记忆与快速更新,从而显著提升模型在目标快速运动、形变或遮挡等复杂条件下的鲁棒性与适应性。在目标跟踪任务上做应用验证的实验表明,与同期最先进的方法相比,该方法在计算量降低74%的同时,在不同数据集上的追踪准确率平均提升了1.2%。更多还原

【Abstract】 Traditional artificial neural networks often fail to meet the practical demands for la-tency,energy efficiency,and robustness in complex real-world scenarios involving dynamic motion,limited computational resources,and changing illumination.Spiking neural network(SNN),with their event-driven operation and biological interpretability,have thus attracted increasing attention.Neuromorphic vision sensors,such as event cameras,offer sparse,asyn-chronous event streams with microsecond temporal resolution,opening new pathways to-ward efficient and interpretable SNN-based processing.However,deploying SNNs in real-world neuromorphic vision systems still faces three fundamental challenges:(1)heteroge-neous multimodal sensing with mismatched spatiotemporal frequencies hinders unified and efficient cross-modal fusion;(2)lightweight SNN models struggle to balance accuracy and power efficiency under resource constraints;(3)long-term dynamic environments cause fea-ture degradation and catastrophic forgetting.To address these challenges,this dissertation in-vestigates SNN design for neuromorphic vision and develops three complementary method-ological frameworks,integrating multimodal efficient representation,lightweight learning,and adaptive modulation.These contributions collectively enable low-latency,low-power,and robust brain-inspired scene understanding.The main contributions are summarized as follows:(1)Multimodal Efficient Representation Method Based on Neuronal Intrinsic Plastic-ity.To bridge the domain gap and frequency inconsistency between multimodal signals—such as event streams(motion cues)and Li DAR depth(geometry)—in dynamic settings,we propose an event-enhanced multimodal spiking actor network regulated by neuronal intrin-sic plasticity.A hybrid spiking variational autoencoder extracts spatiotemporal representa-tions in an unsupervised manner,while a learnable-threshold fusion module enables efficient cross-modal integration through intrinsic plasticity.Evaluated on robotic dynamic obstacle avoidance,our method improves success rates by at least 2.5% in static scenes and 10.8% in dynamic scenes over state-of-the-art approaches.(2)Lightweight Learning Method for Spiking Neural Networks Based on Multimodal Collaborative Knowledge Distillation.To jointly achieve high accuracy and low power con-sumption under resource limitations,we introduce a lightweight SNN trained through multi-modal collaborative knowledge distillation.The method transfers spatiotemporal cues from a high-accuracy teacher model to a fully spiking student model via cross-modal eye-movement distillation.On eye-tracking interaction tasks,this approach reduces computation by 95%,improves accuracy by 3.4%across four challenging illumination conditions,and enhances real-time inference speed by 65.1% compared to existing state-of-the-art method SEEN—all while maintaining competitive recognition performance.(3)Transformer-Based Adaptive Spike Intensity Regulation Method for Spiking Neu-ral Network.To mitigate feature degradation and knowledge forgetting in long-term dy-namic visual tasks,we develop a fully spiking tracking framework based on Transformers.A single-template and multi-search-frame interactive training strategy enhances long-term de-pendency modeling,while an adaptive spike intensity regulation module efficiently preserves and updates critical temporal information.Experiments on challenging tracking benchmarks demonstrate that our method reduces computational cost by 74%and improves tracking ac-curacy by an average of 1.2%across multiple datasets over leading state-of-the-art trackers.更多还原