# User Synchronization System - Complete Guide

## 🎯 Overview

This system automatically synchronizes user data (employee records) from the ADMS server to all connected ZKTeco biometric devices. Users can be pushed to devices in bulk or automatically when a new device registers.

## 📋 What's Included

### Database Tables
- **device_user_sync** - Tracks which users are synced to which devices
- **device_user_sync_queues** - Queue for async sync operations (future use)

### API Endpoints
- **GET /iclock/user-upload** - Devices call this to download user data

### Artisan Commands
- **php artisan device:sync-users** - Push users to all/specific devices

### Models
- Device (updated with relationships)
- DeviceUserSync
- DeviceUserSyncQueue

## 🚀 Quick Start

### 1. Push All Users to All Devices (One Command)

```bash
php artisan device:sync-users
```

Output:
```
Starting user synchronization...
✓ Successfully queued user syncs:
  - Devices: 3
  - Users: 4
  - Total syncs queued: 12
User synchronization completed!
```

### 2. Push Users to Specific Device

```bash
php artisan device:sync-users --device=NYU7255000824
```

### 3. Add New User (Auto-Syncs on Next Device Sync)

```bash
php artisan tinker
> DB::table('users')->insert(['name' => 'New User', 'email' => '...', ...])
```

The user will be automatically synced when the device requests data.

## 📊 How It Works

### Scenario 1: New Device Registration
```
Device POSTs /iclock/registry
    ↓
Server detects new device
    ↓
Server queues all users for this device
    ↓
Server responds: OK\nUPLOAD=USER\n...
    ↓
Device reads UPLOAD=USER command
    ↓
Device makes GET /iclock/user-upload?SN=XXX
    ↓
Server returns user data (ID\tName\tEmail\r\n)
    ↓
Device stores users locally
    ↓
Server marks sync complete with timestamp
```

### Scenario 2: Manual Push to Existing Devices
```
Admin runs: php artisan device:sync-users
    ↓
Server finds all online devices
    ↓
Server queues all users to each device
    ↓
Command completes instantly
    ↓
When devices sync next, they pull new users
    ↓
Server marks syncs complete
```

## 💾 Database Schema

### device_user_sync Table
```sql
CREATE TABLE device_user_sync (
    id BIGINT PRIMARY KEY,
    device_id BIGINT (FK to devices.id),
    user_id BIGINT (FK to users.id),
    status ENUM('pending', 'synced', 'failed'),
    retry_count INT DEFAULT 0,
    error_message TEXT,
    synced_at TIMESTAMP,
    created_at TIMESTAMP,
    updated_at TIMESTAMP,
    UNIQUE(device_id, user_id)
);
```

**Columns:**
- `device_id` - Which device
- `user_id` - Which user
- `status` - Current sync status
- `synced_at` - When device confirmed receipt
- `retry_count` - Failed attempts (for future retry logic)
- `error_message` - Error details if failed

## 📡 API Format

### User Upload Response
The device calls: `GET /iclock/user-upload?SN=XXXXX&Stamp=9999`

Server responds:
```
OK
1	John Doe	john@example.com
2	Jane Smith	jane@example.com
3	Bob Wilson	bob@example.com
```

**Format:** Each line is `ID\tName\tEmail\r\n`

### Registry Response (with User Sync Command)
Device POSTs to `/iclock/registry`

Server responds:
```
OK
PUSH=1
UPLOAD=USER
DATACOUNT=1
Stamp=1234567890
```

The `UPLOAD=USER` command tells the device to pull users.

## 🔍 Monitoring Sync Status

### Check Sync Queue
```bash
php artisan tinker

# All syncs
DB::table('device_user_sync')->get();

# Pending syncs
DB::table('device_user_sync')->where('status', 'pending')->get();

# Synced users for specific device
DB::table('device_user_sync')
    ->join('devices', 'device_user_sync.device_id', '=', 'devices.id')
    ->where('devices.no_sn', 'NYU7255000824')
    ->select('*')
    ->get();
```

### View Logs
```bash
# Watch logs in real-time
tail -f /var/www/html/adms/adms-server/storage/logs/iclock-2026-06-01.log

# Search for user upload requests
grep "USER UPLOAD" /var/www/html/adms/adms-server/storage/logs/iclock-*.log
```

## 🧪 Testing

### Test User Upload Endpoint
```bash
curl "http://127.0.0.1/iclock/user-upload?SN=NYU7255000824"
```

Expected output:
```
OK
1	John Doe	john@example.com
2	Jane Smith	jane@example.com
```

### Test with Specific Device
```bash
curl "http://127.0.0.1/iclock/user-upload?SN=TEST123"
```

## 📝 API Integration

### Device Implementation
When a device receives `UPLOAD=USER` command during registry:

1. Parse the response to find `UPLOAD=USER`
2. Extract device SN from its configuration
3. Make HTTP GET request to:
   ```
   GET /iclock/user-upload?SN=DEVICE_SN&Stamp=9999
   ```
4. Parse response line by line
5. Extract user data: `ID\tName\tEmail`
6. Store users in local device database
7. Send confirmation/ACK (if device supports it)

## 🔐 Security Notes

- `/iclock/user-upload` is exempt from CSRF protection (device endpoints)
- Users table should not expose sensitive data (no passwords)
- Device SN verification happens via database lookup
- All sync operations are logged

## 📈 Performance

- **Bulk sync command:** O(D×U) where D=devices, U=users
  - Creates insert records instantly
  - Devices pull data asynchronously
  
- **Per-device sync:** O(U) linear with user count
  - Minimal server load

- **Current system:** 3 devices × 4 users = 12 records
  - All queries complete in <100ms

## 🚀 Future Enhancements

1. **Fingerprint Sync** - Push fingerprint templates to devices
2. **Face Recognition** - Sync face template images
3. **Work Schedule** - Push shift schedules to devices
4. **Async Queue** - Process syncs asynchronously with retries
5. **Delta Sync** - Only sync modified users (optimization)
6. **Device Acknowledgment** - Parse device confirmation responses
7. **User Deletion** - Remove users from devices when deleted
8. **Batch Operations** - API endpoint to bulk upload users

## 🔧 Implementation Files

```
app/Http/Controllers/iclockController.php
  ├─ userUpload() - GET /iclock/user-upload endpoint
  ├─ syncNewDeviceUsers() - Triggered when new device registers
  └─ syncAllUsersToAllDevices() - Bulk sync command

app/Models/Device.php (updated)
  ├─ userSyncs() - Relationship to device_user_sync
  ├─ users() - BelongsToMany relationship
  ├─ hasPendingUserSyncs() - Check helper
  └─ getPendingUsers() - Get pending users

app/Models/DeviceUserSync.php (new)
  ├─ device() - Belongs to Device
  └─ user() - Belongs to User

app/Models/DeviceUserSyncQueue.php (new)
  ├─ device() - Belongs to Device
  └─ user() - Belongs to User

app/Console/Commands/SyncUsersToDevices.php (new)
  ├─ device:sync-users command
  ├─ --device option for specific device
  └─ Full/partial sync logic

routes/web.php (updated)
  └─ GET /iclock/user-upload route

app/Http/Middleware/VerifyCsrfToken.php (updated)
  └─ Added iclock/user-upload exemption

database/migrations/2026_06_01_141040_...php (new)
  └─ device_user_sync table

database/migrations/2026_06_01_141052_...php (new)
  └─ device_user_sync_queues table
```

## 📞 Troubleshooting

### Issue: Device not pulling users
**Solution:** Check that device is getting `UPLOAD=USER` in registry response and that it's making GET request to user-upload endpoint.

### Issue: Syncs show as "pending" forever
**Solution:** Verify device is connecting. Check `/iclock/user-upload` logs. May need to trigger device sync manually.

### Issue: Users not appearing in database
**Solution:** Verify users table has data. Check logs for any insert errors. Run `php artisan device:sync-users` to retry.

### Issue: Specific device not getting users
**Solution:** Verify device exists in `devices` table. Check device.online timestamp is recent. Use `--device=SN` option.

## 📚 Related Documents

- [Device Communication Protocol](./ADMS%20server%20ZKTeco.postman_collection.json)
- [Attendance Sync Guide](./LOG_MONITORING_GUIDE.md)
- [Timezone Configuration](./config/app.php)

---

**Last Updated:** June 1, 2026  
**Status:** ✅ Production Ready  
**Version:** 1.0