Securing Express File Uploads: A Complete Guide with Pompelmi
Securing Express File Uploads: A Complete Guide with Pompelmi
Express.js applications that accept user uploads are exposed to common file-based attack patterns. From ZIP bombs to executable files disguised as images, this guide shows how to build a practical upload security layer with Pompelmi.
The State of File Upload Security
Before diving into implementation, let’s understand the threat landscape:
- File upload endpoints are a common attack surface when validation is weak.
- ZIP bombs can amplify small compressed payloads into large extraction workloads.
- Polyglot files can bypass basic MIME type checks
- Nested archives can contain hundreds of thousands of files
Quick Start: Basic Express Setup
import express from 'express';import multer from 'multer';import { createUploadGuard } from '@pompelmi/express-middleware';import { composeScanners, CommonHeuristicsScanner, createZipBombGuard,} from 'pompelmi';
const app = express();
// Always buffer in memory — never write to disk before scanningconst upload = multer({ storage: multer.memoryStorage(), limits: { fileSize: 10 * 1024 * 1024 }, // 10 MB hard cap at parser});
const scanner = composeScanners( [ ['zipGuard', createZipBombGuard({ maxEntries: 100, maxTotalUncompressedBytes: 50 * 1024 * 1024, maxCompressionRatio: 100, })], ['heuristics', CommonHeuristicsScanner], ], { parallel: false, stopOn: 'malicious', tagSourceName: true });
const guard = createUploadGuard({ includeExtensions: ['jpg', 'jpeg', 'png', 'pdf'], maxFileSizeBytes: 10 * 1024 * 1024, stopOn: 'suspicious', failClosed: true, scanner,});
app.post('/upload', upload.single('file'), guard, (req, res) => { if (!req.file) { return res.status(400).json({ error: 'No file uploaded' }); }
const { verdict } = (req as any).pompelmi;
if (verdict === 'clean') { // Move to permanent storage res.json({ ok: true, filename: req.file.originalname }); } else { res.status(422).json({ error: 'File failed security scan', verdict }); }});
app.listen(3000);});
app.listen(3000, () => { console.log(‘Secure file upload server running on port 3000’); });
## Advanced Configuration Options
### MIME Type Validation
Pompelmi goes beyond simple extension checking by analyzing file headers:
```javascriptconst scanner = expressFileScanner({ mimeValidation: { // Strict mode: file content must match declared MIME type strict: true,
// Allow specific MIME types allowList: [ 'image/jpeg', 'image/png', 'image/webp', 'application/pdf', 'text/plain' ],
// Block dangerous MIME types denyList: [ 'application/x-executable', 'application/x-msdownload', 'application/vnd.ms-cab-compressed' ],
// Custom validation for specific types customValidators: { 'image/jpeg': (buffer) => { // Validate JPEG structure return buffer[0] === 0xFF && buffer[1] === 0xD8; } } }});ZIP Bomb Protection
Configure comprehensive archive scanning:
const scanner = expressFileScanner({ zipLimits: { // Maximum number of entries in archive maxEntries: 1000,
// Maximum nesting depth (zip in zip in zip...) maxDepth: 10,
// Maximum total uncompressed size maxTotalSize: 100 * 1024 * 1024, // 100MB
// Maximum size for individual entries maxEntrySize: 10 * 1024 * 1024, // 10MB
// Compression ratio threshold (compressed:uncompressed) maxCompressionRatio: 100,
// Scan archive contents recursively scanContents: true,
// Block suspicious archive structures blockSuspiciousStructures: true }});File Type Detection
Advanced file type analysis beyond extensions:
const scanner = expressFileScanner({ fileTypeDetection: { // Use magic bytes for detection useMagicBytes: true,
// Analyze file structure structuralAnalysis: true,
// Detect polyglot files (files that are valid in multiple formats) detectPolyglots: true,
// Custom magic byte patterns customMagicBytes: { 'custom/format': [0x43, 0x55, 0x53, 0x54] // "CUST" } }});Real-World Security Scenarios
Scenario 1: Image Upload with Metadata Stripping
const imageUpload = expressFileScanner({ allowedMimeTypes: ['image/jpeg', 'image/png', 'image/webp'], maxFileSize: 5 * 1024 * 1024, // 5MB
// Strip potentially dangerous metadata sanitization: { stripExifData: true, stripColorProfiles: true, removeComments: true },
// Additional image validation imageValidation: { maxDimensions: { width: 4096, height: 4096 }, minDimensions: { width: 10, height: 10 }, allowAnimated: false }});
app.post('/upload-image', upload.single('image'), imageUpload, (req, res) => { // Handle cleaned image upload});Scenario 2: Document Upload with Content Analysis
const documentUpload = expressFileScanner({ allowedMimeTypes: [ 'application/pdf', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' ],
// Document-specific security documentSecurity: { // Scan for embedded macros scanMacros: true,
// Check for suspicious JavaScript in PDFs scanJavaScript: true,
// Validate document structure validateStructure: true,
// Maximum number of pages/sheets maxPages: 100 },
// Advanced content analysis contentAnalysis: { // Scan text content for suspicious patterns textPatterns: [ /(?:cmd|powershell|bash)\s*\.\s*exe/gi, /<script[^>]*>/gi ],
// Check for embedded files scanEmbeddedFiles: true }});Scenario 3: Multi-File Upload with Batch Scanning
const batchUpload = multer({ dest: 'uploads/', limits: { fileSize: 10 * 1024 * 1024, files: 10 // Max 10 files }}).array('files', 10);
const batchScanner = expressFileScanner({ // Scan all files in the batch batchMode: true,
// Fail fast: reject entire batch if any file fails failFast: true,
// Cross-file analysis crossFileAnalysis: { // Check for suspicious file combinations detectSuspiciousCombinations: true,
// Analyze relationships between files analyzeRelationships: true }});
app.post('/upload-batch', batchUpload, batchScanner, (req, res) => { const files = req.files; const scanResults = req.scanResults; // Array of results
const cleanFiles = scanResults.filter(r => r.verdict === 'clean'); const threats = scanResults.filter(r => r.verdict !== 'clean');
res.json({ processed: files.length, clean: cleanFiles.length, threats: threats.length, threatDetails: threats.map(t => ({ filename: t.filename, verdict: t.verdict, findings: t.findings })) });});Error Handling and Logging
Implement comprehensive error handling and security logging:
const scanner = expressFileScanner({ // Configure security logging logging: { // Log all scan results logAll: true,
// Log only threats and errors logThreatsOnly: false,
// Custom logger logger: (event, data) => { console.log(`[SECURITY] ${event}:`, data);
// Send to security monitoring system if (event === 'threat_detected') { securityMonitor.alert(data); } } },
// Error handling onError: (error, req, res, next) => { console.error('Scan error:', error);
// Don't expose internal errors to client res.status(500).json({ error: 'File processing failed', requestId: req.id }); },
// Threat handling onThreat: (scanResult, req, res) => { // Log security event securityLogger.warn('Malicious file upload attempt', { ip: req.ip, userAgent: req.get('User-Agent'), filename: req.file.originalname, threats: scanResult.findings });
// Quarantine file quarantine.moveFile(req.file.path);
// Response res.status(422).json({ error: 'File rejected due to security concerns', code: 'MALICIOUS_FILE_DETECTED' }); }});Performance Optimization
For high-throughput applications:
const scanner = expressFileScanner({ // Performance tuning performance: { // Parallel processing for multi-file uploads parallel: true, maxConcurrent: 4,
// Cache scan results caching: { enabled: true, ttl: 3600, // 1 hour maxSize: 1000 // Cache 1000 results },
// Skip expensive checks for trusted sources trustedSources: ['192.168.1.0/24'],
// Timeout for long scans timeoutMs: 30000 // 30 seconds }});Testing Your Security
Validate your implementation with security tests:
const request = require('supertest');const fs = require('fs');
describe('File Upload Security', () => { test('should reject ZIP bomb', async () => { const zipBomb = fs.readFileSync('test/fixtures/zipbomb.zip');
const response = await request(app) .post('/upload') .attach('file', zipBomb, 'innocent.zip') .expect(422);
expect(response.body.error).toContain('security'); });
test('should accept clean images', async () => { const cleanImage = fs.readFileSync('test/fixtures/clean.jpg');
const response = await request(app) .post('/upload') .attach('file', cleanImage, 'photo.jpg') .expect(200);
expect(response.body.message).toContain('success'); });});Monitoring and Alerting
Set up real-time security monitoring:
const scanner = expressFileScanner({ monitoring: { // Real-time metrics metrics: { endpoint: 'https://metrics.yourapp.com/security', interval: 60000 // 1 minute },
// Alert thresholds alerts: { // Alert if threat rate exceeds threshold threatRateThreshold: 0.1, // 10% threat rate
// Alert on specific threat types alertOnThreats: ['zip_bomb', 'executable', 'macro'],
// Webhook for immediate alerts webhook: 'https://alerts.yourapp.com/security' } }});Best Practices Summary
- Layer your defenses: Use multiple validation techniques
- Log everything: Security events need comprehensive logging
- Monitor continuously: Set up real-time threat monitoring
- Test regularly: Include security tests in your CI/CD
- Update frequently: Keep Pompelmi and its rules updated
- Fail securely: Default to rejection when in doubt
- Educate users: Provide clear error messages and guidance
Conclusion
File upload security is not optional for applications accepting untrusted files. With Pompelmi’s Express middleware, you can implement a practical upload gate without sacrificing developer ergonomics. The examples in this guide provide a strong foundation for reducing file-based risk in Express applications.
Remember: security is a journey, not a destination. Regularly review and update your security policies as new threats emerge.
Want to dive deeper? Check out our YARA Integration Guide for advanced threat detection techniques.