README.md

---

SpikeFPN: Automotive Object Detection via Learning Sparse Events by Spiking Neurons

---

This work explores the membrane potential dynamics of spiking neural networks (SNNs) and their ability to process sparse, asynchronous events. We propose an innovative spike-triggered adaptive threshold mechanism that facilitates stable and effective training. Building on this foundation, we design a specialized spiking feature pyramid network (SpikeFPN) optimized for automotive event-based object detection. Comprehensive evaluations indicate that SpikeFPN achieves competitive performance compared to traditional SNNs and advanced artificial neural network (ANN) models while maintaining efficient computation.

• Official publication in IEEE Transactions on Cognitive and Developmental Systems:
  DOI: 10.1109/TCDS.2024.3410371

• Accepted manuscript available on arXiv:
  arXiv:2307.12900

Environment Configuration

In a configuration utilizing Ubuntu 22.04, CUDA 12.4, and PyTorch 2.3.1:

apt-get update # If necessary
apt-get install ffmpeg libsm6 libxext6
pip install -r requirements.txt

Experiment on GEN1 Automotive Detection (GAD) Dataset

Data Preprocessing

python ./preprocess/gad_framing.py

Training and Testing

python ./train_gad.py
python ./test_gad.py

Experiment on N-CARS Dataset

Data Preprocessing

python ./preprocess/ncars_framing.py

Data Division

	Class: background	Class: cars
For Training	0 ~ 4210	0 ~ 4395
For Validating	4211 ~ 5706	4396 ~ 5983
For Testing	5707 ~ 11692	5984 ~ 12335

Training and Testing

python ./train_ncars.py
python ./test_ncars.py

Citation

Please cite the following publication if this work was helpful to your research.

@article{spikefpn,
  author  = {Hu Zhang and Yanchen Li and Luziwei Leng and Kaiwei Che and Qian Liu and Qinghai Guo and Jianxing Liao and Ran Cheng},
  title   = {Automotive Object Detection via Learning Sparse Events by Spiking Neurons},
  journal = {{IEEE} Trans. Cogn. Dev. Syst.},
  volume  = {16},
  number  = {6},
  pages   = {2110--2124},
  year    = {2024},
  doi     = {10.1109/TCDS.2024.3410371},
}