Remote Desktop
Overview
ET Ducky’s browser-based remote desktop lets you view and control any online agent’s screen directly from the dashboard. No VPN, no VNC client, no RDP configuration, no firewall rules — just click “Remote” on any online agent and start working.
Remote desktop sessions run alongside existing features. You can have a live query session and a remote desktop session with the same agent simultaneously.
Starting a Session
- Navigate to the Agents page
- Click the Remote button on any online agent row (the button is disabled for offline agents)
- A fullscreen modal opens with a connecting overlay
- The server creates a session record and signals the agent via SSE
- The agent initializes screen capture and connects its WebSocket
- Once both sides are connected, frames begin streaming and the overlay disappears
Typical connect time: 1–3 seconds on LAN, 2–5 seconds on WAN.
Toolbar Controls
| Control | Description |
|---|---|
| Status Indicator | Color-coded dot: amber (connecting), green (connected), red (disconnected) |
| FPS / Bandwidth | Live readout of frames per second and data throughput |
| Quality | Dropdown: Low (30), Medium (50), High (70), Ultra (90). Adjusts encoding quality in real time |
| Scale Mode | Fit (maintain aspect ratio), Stretch (fill viewport), Native (1:1 pixels with scroll) |
| Display Picker | Appears when the agent has multiple monitors. Each display gets a button; click to focus one, or shift-click to add additional displays to the active set and span the canvas across them (see Multi-Display Sessions below) |
| Keyboard Grab | Toggle to capture browser shortcuts (Ctrl+Tab, Alt+F4, etc.) and send them to the remote system |
| Fullscreen | Enter true browser fullscreen for an immersive experience |
| Disconnect | End the session, release all resources, and close the modal |
On a phone or tablet the toolbar is replaced by a single bottom control bar with the same capabilities — see Controlling from Your Phone below.
Input & Clipboard
Mouse
- Mouse movement, left/middle/right click, and scroll wheel are all forwarded to the remote agent
- Coordinates are normalized (0.0–1.0) and translated to absolute screen coordinates on the agent via SendInput
- Mouse move events are throttled to 60/second to prevent overwhelming the connection
Keyboard
- All key presses are captured using browser
KeyboardEvent.codevalues and mapped to Windows virtual key codes - Modifier keys (Ctrl, Alt, Shift, Win) are tracked independently to prevent stuck-key issues
- Enable Keyboard Grab in the toolbar to intercept browser shortcuts and forward them to the remote system
- All held modifier keys are automatically released when a session ends
Clipboard
- Text clipboard syncs bidirectionally between browser and agent
- Agent monitors clipboard changes via
AddClipboardFormatListenerand pushes updates instantly - Browser clipboard access requires page focus (Clipboard API security requirement)
Controlling from Your Phone
On a touch device the session switches to a mobile layout automatically: the desktop toolbar folds into a single control bar at the bottom of the screen (within thumb reach), the view starts at the top of the remote screen, and frames render at your phone’s full native resolution. The first tap enters fullscreen, and fullscreen switches to landscape automatically on Android — even if your phone’s rotation lock is on (on iPhone, rotate the phone manually). While a session is open, the phone’s back gesture and pull-to-refresh are disabled so a stray swipe can’t end the session — leave with End.
Pointer modes
| Mode | Behavior |
|---|---|
| Trackpad (default) | The screen works like a laptop trackpad: glide anywhere to move the virtual cursor, tap to click. Precise — best for ordinary desktop UIs. |
| Tap | Touches act directly at the point you touch. Best when zoomed in, or for kiosk-style UIs with big targets. |
Switch modes any time with the Trackpad / Tap button on the control bar; your choice is remembered.
Gestures
| Gesture | Action |
|---|---|
| Glide one finger | Move the cursor (trackpad mode) / move the pointer under your finger (tap mode) |
| Tap | Left click — tap twice for a double-click |
| Tap, then tap and hold | Drag: the left button stays held while you glide — select text, move windows, drag files. Released when you lift. |
| Hold still (∼0.7 s) | Right-click (trackpad mode; a short vibration confirms). Moving your finger cancels it — a gliding finger never right-clicks. |
| Two-finger drag | Scroll (mouse wheel) |
| Two-finger tap | Right-click (trackpad mode) |
| Pinch | Zoom the view 1–4× around your fingers; in tap mode a two-finger drag pans while zoomed. 1:1 on the control bar resets. |
The control bar
| Control | Description |
|---|---|
| Connection badge | Live FPS and latency, colored green / amber / red by connection quality |
| Trackpad / Tap | Switches pointer mode (table above) |
| 💬 Tools | Opens the session tools (Ask Ducky, Shell, Files) as a bottom sheet. Opening does not raise the phone keyboard; it appears when you tap an input field. Tap again (or the sheet’s close button) to give the remote view the whole screen back |
| ⌨ Type | Summons your phone’s keyboard for typing into the remote machine (including Backspace and Enter) |
| Ctrl⋯ Quick keys | A strip of keys phones lack: Esc, Tab, Win, arrows, Del, latching Ctrl / Alt / Shift for combos, and one-tap Ctrl+Alt+Del |
| 1:1 | Resets pinch zoom to fit |
| ⛶ Fullscreen | Toggles fullscreen (auto-landscape on Android) |
| ⚙ Settings | The desktop toolbar’s session controls: Quality, Scale Mode, the Display picker for multi-monitor agents, Keyboard Grab, and event-collection pause |
| End | Disconnects the session |
Performance & Limits
| Metric | Typical Value |
|---|---|
| Input latency | <50 ms LAN, <150 ms WAN |
| Frame rate (idle) | 1–5 FPS |
| Frame rate (active) | 15–30 FPS |
| Bandwidth — H.264 (idle) | ∼30 KB/s |
| Bandwidth — H.264 (active, typical) | ∼400–600 KB/s |
| Bandwidth — WebP/JPEG fallback (low quality) | 0.5–2 Mbps |
| Bandwidth — WebP/JPEG fallback (high quality, 1080p) | 2–5 Mbps |
| Capture method | DXGI Desktop Duplication (GPU-accelerated); GDI+ fallback for RDP sessions |
| Encoding | H.264 decoded in-browser via WebCodecs where available (hardware-accelerated, up to 30 FPS, dramatically lower bandwidth); WebP or JPEG with dirty-region encoding as the fallback when the browser lacks WebCodecs or the agent's H.264 encoder can't initialize |
| Max concurrent sessions | 2 per organization |
| Idle timeout | 15 minutes |
Adaptive quality: The encoder automatically reduces quality when bandwidth is constrained and increases it when headroom is available.
Troubleshooting
- Black screen after connecting: The agent may be in an RDP session where DXGI is unavailable. The agent automatically falls back to GDI+ capture, which may take an extra second.
- High latency or stuttering: Reduce quality to Low or Medium in the toolbar. Check network connectivity between the agent and the ET Ducky server.
- Keyboard shortcuts not working remotely: Enable Keyboard Grab in the toolbar to intercept browser shortcuts.
- Clipboard not syncing: Ensure the browser tab is focused. The Clipboard API requires page focus for security reasons.
- “Agent is not online” error: The agent must be online (green status) to start a remote desktop session. Check agent connectivity.
- “Maximum sessions” error: Your organization has 2 concurrent remote desktop sessions active. End one before starting another.
- Linux (Wayland): you can see the screen but not control it. Agent versions before 2.5.9.6 asked the desktop portal to remember the input grant, which Wayland portals refuse for remote-desktop sessions — the portal answered by granting zero input devices instead of returning an error, so the session ran view‑only with nothing to explain it. Upgrade the agent. If it persists on 2.5.9.6 or later, the compositor itself is refusing input: check the agent log for
granted zero input devices. Accepting the on‑screen prompt is not the issue — a declined prompt fails the session outright, so if you have a picture at all, the prompt was accepted. Screen capture, file transfer and the shell are unaffected either way; only pointer and keyboard injection stop. - Linux (Wayland): the picture is frozen but the session is healthy. Wayland screen capture is change‑driven — a desktop where nothing is moving produces no frames by design, which is not a fault. Moving the mouse on the remote machine, or anything else that repaints the screen, resumes the stream immediately.
Multi-Display Sessions
On agents with multiple monitors, you can capture and control more than one display in a single session. Useful when you’re troubleshooting an app whose window crosses the bezel, or when you want to see a dashboard on one monitor while you click around on another.
Selecting displays
- The toolbar shows one button per monitor (e.g. Display 1, Display 2, Display 3).
- Click a button to focus that display — the canvas resizes to that monitor’s resolution and only that monitor is captured.
- Shift-click a button to add it to the active set. The canvas grows to span the union of selected monitors and the frame stream switches to a composite encoder that stitches them together server-side.
- Shift-click an already-active display to remove it from the set (one display always stays selected).
Input mapping
When multiple displays are active, mouse and keyboard input are routed using MOUSEEVENTF_VIRTUALDESK coordinates — coordinates map to the agent’s virtual-desktop space rather than a single screen. Click anywhere on the composite canvas and the cursor lands on the corresponding monitor on the remote host.
Performance impact
Each additional display roughly doubles the bytes-per-frame budget. On a low-bandwidth link, drop the Quality dropdown to Medium or Low before adding more monitors. The composite stream uses the same encoder pipeline (DXGI Desktop Duplication on Windows, x11vnc on Linux), so the per-monitor frame rate stays consistent.
Multi-display support is available in agent versions 2.0.1.2 and later, both Windows and Linux. Older agents will only expose a single display button until upgraded.