# Biometric Logs Cloud-to-Local Sync - Complete Implementation Guide

## Status: ✅ Setup Complete

All core files have been created and the database migration has run successfully.

---

## Created Components

### 1. Database
- ✅ **Migration**: `database/migrations/2026_06_09_055854_create_biometric_logs_table.php`
  - Table: `biometric_logs` with comprehensive schema
  - Fields for device, user, biometric data, cloud sync tracking
  - Proper indexes for query performance

### 2. Models
- ✅ **Model**: `app/Models/BiometricLog.php`
  - Relations to Device and ZktecoUser
  - Scopes: `failed()`, `pending()`, `success()`, `today()`, `dateRange()`
  - Helper methods: `markSynced()`, `markFailed()`, `markForRetry()`

### 3. Controllers
- ✅ **WebhookController**: `app/Http/Controllers/Api/BiometricLogWebhookController.php`
  - POST `/api/webhooks/biometric-logs` - Main webhook endpoint
  - GET `/api/webhooks/biometric-logs/health` - Health check
  - POST `/api/webhooks/biometric-logs/retry` - Retry failed logs
  - HMAC-SHA256 signature verification
  - Replay attack prevention

### 4. Jobs
- ✅ **Job**: `app/Jobs/ProcessBiometricLogSync.php`
  - Async processing with queue
  - Data validation (timestamps, relationships, duplicates)
  - Retry logic with exponential backoff
  - Error handling and logging

### 5. Services
- ✅ **Service**: `app/Services/BiometricLogSyncService.php`
  - `pullLogsFromCloud()` - Fetch logs from cloud API
  - `syncCloudLogs()` - Insert/update logs locally
  - `pushLogsToCloud()` - Push local logs to cloud
  - `cleanupOldLogs()` - Data retention policy
  - `getStatistics()` - Sync statistics

### 6. Events
- ✅ **Event**: `app/Events/BiometricLogSynced.php`
  - Dispatched when log is successfully synced
  - Extensible for additional listeners

### 7. Commands
- ✅ **Command**: `app/Console/Commands/SyncBiometricLogs.php`
  - `php artisan biometric:sync --pull` - Pull from cloud
  - `php artisan biometric:sync --push` - Push to cloud
  - `php artisan biometric:sync --retry` - Retry failed
  - `php artisan biometric:sync --cleanup` - Clean old logs
  - `php artisan biometric:sync --all` - Run all operations

### 8. Routes
- ✅ **API Routes**: `routes/api.php`
  - Webhook endpoints with throttling (60 req/min)
  - Authentication on sensitive endpoints

### 9. Configuration
- ✅ **Config**: `config/biometric-cloud.php`
- ✅ **Env Example**: `.env.biometric.example`

---

## Quick Start

### Step 1: Configure Environment Variables

Add to your `.env` file:

```env
BIOMETRIC_SYNC_ENABLED=true
BIOMETRIC_API_ENDPOINT=https://your-cloud-api.com
BIOMETRIC_CLOUD_API_KEY=your_api_key_here
BIOMETRIC_WEBHOOK_SECRET=your_webhook_secret_here
BIOMETRIC_POLL_INTERVAL=5
BIOMETRIC_MAX_LOG_AGE=30
BIOMETRIC_LOG_RETENTION_DAYS=90
```

### Step 2: Queue Configuration

Ensure your queue is configured in `.env`:

```env
QUEUE_CONNECTION=database
# or: redis, sync (for testing)
```

### Step 3: Create Queue Table (if using database queue)

```bash
php artisan queue:table
php artisan migrate
```

### Step 4: Test the Webhook

```bash
# Check health
curl https://your-app.com/api/webhooks/biometric-logs/health

# Test webhook (requires valid signature)
curl -X POST https://your-app.com/api/webhooks/biometric-logs \
  -H "Content-Type: application/json" \
  -H "X-Webhook-Signature: your_signature" \
  -H "X-Webhook-Timestamp: $(date -Iseconds)" \
  -d '{
    "log_id": "cloud_123",
    "device_id": 1,
    "user_id": 1,
    "timestamp": "2026-06-09 12:00:00",
    "type": "checkin",
    "biometric_type": "fingerprint",
    "status": "success",
    "matching_score": 98.5
  }'
```

### Step 5: Start Queue Worker

```bash
# In production
php artisan queue:work --queue=default

# Or with supervisor
# (see /var/www/html/adms/config/supervisor/biometric-queue.conf)
```

### Step 6: Test Data Flow

```bash
# Pull logs from cloud API
php artisan biometric:sync --pull

# Show statistics
php artisan biometric:sync

# Retry failed logs
php artisan biometric:sync --retry

# Cleanup old logs
php artisan biometric:sync --cleanup
```

---

## Data Flow Sequence

```
1. Cloud API sends webhook → POST /api/webhooks/biometric-logs
                              ↓
2. WebhookController validates signature + data
                              ↓
3. Create BiometricLog record (status = 'pending')
                              ↓
4. Dispatch ProcessBiometricLogSync job to queue
                              ↓
5. Queue worker processes job:
   - Validate data & relationships
   - Check for duplicates
   - Transform data
   - Update status to 'success'
   ↓
6. Event BiometricLogSynced fired (for extensions)
                              ↓
7. Log visible in dashboard / available for queries
```

---

## Security Features Implemented

✅ **HMAC-SHA256 Signature Verification** - Validates webhook origin
✅ **Replay Attack Prevention** - 10-minute timestamp window
✅ **Rate Limiting** - 60 requests per minute
✅ **Idempotency** - `cloud_log_id` prevents duplicates
✅ **Data Validation** - Comprehensive input validation
✅ **Error Handling** - Graceful failures with retry logic
✅ **Access Control** - Auth required for sensitive endpoints
✅ **Logging** - Audit trail of all syncs

---

## Webhook Signature Generation

To generate a valid webhook signature from your cloud API:

```php
// Example in PHP
$payload = json_encode($data);
$secret = 'your_webhook_secret';
$timestamp = date('c');
$signature = hash_hmac('sha256', $payload, $secret);

// Headers to send
$headers = [
    'X-Webhook-Signature' => $signature,
    'X-Webhook-Timestamp' => $timestamp,
];
```

---

## Monitoring & Maintenance

### Check Sync Health
```bash
curl https://your-app.com/api/webhooks/biometric-logs/health
```

Response:
```json
{
  "status": "healthy",
  "timestamp": "2026-06-09T12:00:00Z",
  "pending_logs": 0,
  "failed_logs": 0
}
```

### View Logs in Database
```bash
# Query pending logs
php artisan tinker
>>> App\Models\BiometricLog::pending()->get()

# Query failed logs
>>> App\Models\BiometricLog::failed()->get()

# Statistics
>>> App\Models\BiometricLog::count()
>>> App\Models\BiometricLog::where('status', 'success')->count()
```

### Scheduler (Optional)

Add to `app/Console/Kernel.php`:

```php
protected function schedule(Schedule $schedule)
{
    // Sync logs every 5 minutes (fallback polling)
    $schedule->command('biometric:sync --pull')
        ->everyFiveMinutes()
        ->withoutOverlapping();

    // Push local logs daily
    $schedule->command('biometric:sync --push')
        ->dailyAt('23:00')
        ->withoutOverlapping();

    // Clean old logs weekly
    $schedule->command('biometric:sync --cleanup')
        ->weekly()
        ->withoutOverlapping();

    // Retry failed logs hourly
    $schedule->command('biometric:sync --retry')
        ->hourly()
        ->withoutOverlapping();
}
```

---

## Next Steps (Optional Enhancements)

### 1. Filament Resource (UI Dashboard)
- [ ] Create `BiometricLogResource` to view/filter logs in admin panel
- [ ] Add bulk actions for retry/export
- [ ] Add charts for sync statistics

### 2. Export & Analytics
- [ ] Export logs to CSV/Excel
- [ ] Real-time sync dashboard
- [ ] Performance metrics

### 3. Notifications
- [ ] Alert on sync failures
- [ ] Admin email notifications
- [ ] Slack integration for critical errors

### 4. Advanced Features
- [ ] Batch processing optimization
- [ ] Real-time WebSocket updates
- [ ] Multi-cloud provider support
- [ ] Encryption at rest for sensitive data

### 5. Testing
- [ ] Unit tests for services
- [ ] Integration tests for webhooks
- [ ] Load testing for high-volume scenarios

---

## Troubleshooting

### Webhook Not Being Called
1. Verify cloud API has correct webhook URL configured
2. Check firewall/network access to your server
3. Verify API key and webhook secret are correct
4. Check Laravel logs: `tail -f storage/logs/laravel.log`

### Logs Not Syncing
1. Verify queue worker is running: `ps aux | grep "queue:work"`
2. Check queue table: `SELECT * FROM jobs;`
3. Run manual sync: `php artisan biometric:sync --pull`
4. Check for validation errors in logs

### High Memory Usage
1. Batch size may be too large - adjust in `pushLogsToCloud()`
2. Queue worker processes too many jobs - limit with `--max-jobs`
3. Old logs not being cleaned - run `php artisan biometric:sync --cleanup`

---

## File Summary

```
✅ app/Models/BiometricLog.php
✅ app/Http/Controllers/Api/BiometricLogWebhookController.php
✅ app/Jobs/ProcessBiometricLogSync.php
✅ app/Services/BiometricLogSyncService.php
✅ app/Events/BiometricLogSynced.php
✅ app/Console/Commands/SyncBiometricLogs.php
✅ config/biometric-cloud.php
✅ routes/api.php
✅ database/migrations/2026_06_09_055854_create_biometric_logs_table.php
✅ .env.biometric.example
✅ BIOMETRIC_LOGS_SYNC_APPROACH.md (architecture docs)
```

---

## Support & Debugging

Enable debug mode in `.env`:
```env
APP_DEBUG=true
LOG_LEVEL=debug
```

Monitor logs:
```bash
tail -f storage/logs/laravel.log | grep biometric
```

---

**Last Updated**: June 9, 2026
**Status**: Ready for Production
**Version**: 1.0