An Event-Driven E-Skin System with Dynamic Binary Scanning and real time SNN Classification

Tactile sensing is increasingly pivotal for enabling machines to interact physically and intelligently with their environment, finding critical applications in robotics for dexterous manipulation and in human-computer interaction for rich, physical interfaces. Among the various sensing technologies, resistive-based tactile systems offer significant advantages in terms of scalability and cost-effectiveness, facilitating their deployment over large areas. These systems typically consist of arrays of sensor elements where applied pressure causes a measurable change in resistance, which is then digitized through scanning circuits and an analog front-end (AFE). However, conventional frame-based scanning methodologies—which drive most current systems—suffer from fundamental efficiency bottlenecks. Inspired by imaging sensors, these approaches periodically poll all sensors in the array to construct a full 'tactile frame', yet they are inherently mismatched to the sparse spatiotemporal nature of tactile data. By continuously acquiring data from every sensor regardless of activity, such systems generate substantial redundancy, leading to excessive power consumption and high transmission overhead.

The core problem addressed in this paper is the inefficiency of traditional tactile data acquisition methods, particularly when dealing with sparse tactile data. Frame-based scanning methods lead to substantial redundancy and high power consumption, as they continuously acquire data from every sensor regardless of activity. Additionally, existing systems often rely on artificial neural networks for classification, which mismatches with the sparse event data characteristics, leading to high computational and storage demands. The challenge is to maintain high classification accuracy while significantly reducing data acquisition and processing overhead.

The core innovations of this paper include the introduction of an event-driven binary scan strategy and the integration of a convolutional spiking neural network (Conv-SNN) in an electronic skin system. • Event-Driven Scan Strategy: This approach significantly reduces data acquisition overhead by dynamically identifying active regions and reducing scan counts. • Convolutional Spiking Neural Network: Implemented on FPGA, this multi-layer network maintains high classification accuracy while significantly reducing computational and storage demands. • Fully Integrated Event-Driven Pipeline: From analog sensing to neuromorphic classification, the system achieves efficient data processing.

• �� Sensor Array: A 16×16 piezoresistive tactile array is fabricated using precision printing techniques. • Event-Driven Scanning: A binary scan strategy dynamically identifies active regions, reducing scan counts. • Data Conversion: Sparse tactile signals are converted into spike sequences via delta modulation. • Convolutional Spiking Neural Network: Implemented on FPGA, this multi-layer network processes asynchronous event streams to extract temporal patterns. • Dataset Construction: A real neuromorphic tactile dataset is constructed using Address Event Representation (AER).

The experimental design includes a real-time handwritten digit recognition task, validated using 760 samples from 13 participants. Data is acquired using a custom hardware platform at a sampling rate of 120 Hz. The performance of the traditional convolutional neural network (CNN) and the convolutional spiking neural network (Conv-SNN) is compared, focusing on computational load, storage requirements, and classification accuracy. The effectiveness of the event-driven scanning strategy is verified by comparing scan counts and data compression rates with traditional methods.

The system achieves a 92.11% classification accuracy in real-time handwritten digit recognition while requiring only 65% of the computation and 15.6% of the weight storage compared to a traditional CNN. The event-driven scanning strategy achieves a 12.8x reduction in scan counts and a 38.2x data compression rate, with a 28.4x increase in dynamic range and 99% data sparsity. Additionally, the constructed neuromorphic tactile dataset demonstrates a fully integrated event-driven pipeline from analog sensing to neuromorphic classification.

The system has broad applications in robotic perception and human-computer interaction. • Robotic Tactile Perception: Enhances robots' ability to interact with their environment, particularly in dexterous manipulation. • Human-Computer Interaction Interfaces: Develops smarter interactive interfaces, improving user experience. • Embedded Systems: Achieves efficient data processing in resource-constrained embedded devices, suitable for various real-time applications.

Despite significant advancements, the robustness of the system in complex environments needs further validation, especially in handling multiple tactile events simultaneously. Additionally, the current implementation relies on a specific hardware platform (FPGA), and its portability and performance on other platforms are not yet clear. The system's scalability and adaptability for more complex tactile recognition tasks require further investigation. Future research directions include scaling the system to handle more complex tasks, such as multimodal tactile perception and higher-resolution tactile data.