Deep LDA-Pruning: Shrinking VGG-16 by 70x for Real-Time Face Analysis
Deep LDA-Pruned Nets for Efficient Facial Gender Classification
This paper introduces an efficient facial gender classification framework by pruning the VGG-16 network using Fisher's Linear Discriminant Analysis (LDA). By replacing the heavy Fully Connected (FC) layers with lightweight classifiers like SVM or Bayesian models, the authors achieve a 70x reduction in model size and an 11-fold speedup while maintaining or improving accuracy on LFW and CelebA datasets.
TL;DR
Researchers from McGill University have demonstrated that the "overkill" of massive deep learning models like VGG-16 can be surgically trimmed for specific tasks. By applying Fisher's Linear Discriminant Analysis (LDA) to the feature map, they reduced the model size by a factor of 70 and boosted speed by 11x, all while achieving higher accuracy in facial gender recognition than the original unpruned network.
Problem & Motivation: The Heavy Cost of "Deep"
While VGG-16 and ResNet have pushed the boundaries of computer vision, their deployment on mobile devices remains a bottleneck. The standard practice of fine-tuning a massive ImageNet-pretrained model for a simple binary task (like gender classification) leads to massive redundancy.
The authors argue against the common "weight-based pruning" (removing small weights). Their Insight is that a weight can be large but totally irrelevant to the task at hand. Instead, they look at neuron activation patterns. They found that at the deeper layers, neurons fire in a highly decorrelated manner—meaning specific neurons specialize in features like "goatee," "glasses," or "curly hair."
Methodology: Fisher to the Rescue
The core of the method is a two-step "Surgical Pruning":
- LDA-Based Selection: They measure the Intra-Class Correlation (ICC) of neurons in the last convolutional layer (Conv5_3). They only keep neurons that show low variance within a gender but high variance between genders.
- Deconv Dependency Mapping: Once a neuron in the last layer is deemed useless, they use Deconvolution (Deconv) to trace back which filters in the previous layers supported that neuron. If a filter in Conv4 only supports "dead" neurons in Conv5, it is pruned entirely.
By replacing the Final Fully Connected (FC) layers—which usually account for ~90% of model parameters—with an SVM or a Bayesian Classifier, they eliminated the biggest memory hogs in the network.
Experimental Results: Less is More
The results on the Labeled Faces in the Wild (LFW) dataset are striking:
- Accuracy: The original VGG-16 achieved 90.3%. The LDA-pruned version with an RBF-SVM hit 92.4%.
- Efficiency: Using only 4 neurons in the final layer was sufficient to maintain high accuracy, leading to a radical compression of the feature space.
As shown in the table above, the speedup ratio in layers like Conv3_2 reached over 53x, and the total system speedup was over 11x on standard CPU hardware.
Critical Analysis & Conclusion
Takeaway
This work proves that for specific downstream tasks, we don't need the "general-purpose" complexity of an ImageNet model. The high decorrelation of neurons in deep layers suggests that deep nets naturally learn a sparse, disentangled representation that LDA can easily exploit.
Limitations
- Retraining requirement: Pruning leads to a sudden drop in accuracy that requires a specific "retraining" phase to recover the performance of the surviving weights.
- Task Specificity: While effective for binary classification (Male/Female), the pruning rate might be lower for complex multi-class problems where more feature diversity is required.
Future Outlook
This approach paves the way for "on-chip" AI where models are stored entirely in SRAM rather than slow off-chip memory, enabling truly real-time, privacy-preserving face analysis on the edge.