Skip to main content

Offline Troubleshooting

Diagnose why a Level device shows offline — AV/EDR interference, network requirements, and the --check diagnostic command.

Introduction

If a device is showing offline in Level but is actually powered on and connected, the cause is almost always AV/EDR interference or a firewall blocking the agent's outbound connections. Start with the --check command — it identifies the exact failure point without guesswork.

ℹ️ NOTE: In most cases, Level works without any firewall changes. The network requirements below only apply if you're on a restrictive network and actively experiencing connectivity problems.


Offline Troubleshooting

Step 1: Run the Diagnostic Check

Run --check on the affected device while it has network access. It tests every part of the agent's connectivity to Level and reports exactly where the failure is.

🖥️ PLATFORM NOTE:

  • Windows:& 'C:\Program Files\Level\level.exe' --check

  • macOS:sudo /usr/local/bin/level --check

  • Linux:sudo /usr/local/bin/level --check

Windows --check Example

The two sections to focus on:

Level checks — shows whether the agent service and watchdog task are in the expected state (Running / Ready). If either shows a problem, AV/EDR has most likely quarantined the agent binary. The watchdog keeps the service running under normal conditions — if it's not, something external stopped it.

Connection checks — shows the status of online.level.io, agents.level.io, uptime monitor, and realtime client. The first three are ping (ICMP) checks, while realtime client reports the agent's separate realtime connection. Error: no ping stats means the ping check received no ICMP replies. If the realtime client is Connected, the agent still has its realtime connection even when one or more ping checks fail.


Step 2: AV/EDR Interference

If --check shows the service or watchdog not in the expected state, or if --check won't run at all, check whether the Level binary is still present on disk:

🖥️ PLATFORM NOTE:

  • Windows: look for level.exe in C:\Program Files\Level\

  • macOS: look for level at /Applications/Level.app/Contents/MacOS/level

  • Linux: look for level at /usr/local/bin/level

If the binary is missing, AV/EDR removed it. Level has no mechanism to delete its own binary — a missing executable means your security software quarantined or deleted it.

To investigate: Check your security software's quarantine log and activity history around the time the device went offline. Look for any action taken against level.exe or related processes. Some products — SentinelOne, ESET, and certain Defender configurations — do this silently with no visible alert.

To fix: Restore the binary from quarantine if possible, then add the appropriate exclusions before reinstalling. See AV/EDR False Detections for exclusion paths, certificate details, and a Windows Defender automation you can deploy across devices. If the binary is still present but the service is stopped, the same exclusion steps apply — the AV is likely blocking execution rather than removing the file.

⚠️ WARNING: If you reinstall the agent without adding exclusions first, AV/EDR will remove the binary again. Add exclusions before reinstalling. Also note: AV/EDR detections against Level are behavior-based, not signature-based. This means the same security software can run on 50 devices and only flag Level on a few — it depends on what the agent was doing at the time the detection was triggered, not on a definition update. Don't assume a clean device means your exclusions are working correctly.


Step 3: Check Network Access

If the Level checks section looks healthy but Connection checks show failures, compare the three ping checks with realtime client:

  • If realtime client also shows an error, the agent may be unable to reach Level's servers. Check for a firewall or proxy blocking the outbound connections below.

  • If the ping checks show Error: no ping stats while realtime client is Connected, check whether the device or network firewall blocks ICMP. That result alone does not mean the agent's realtime connection is blocked.

The agent needs outbound access to these URLs:

URL

Purpose

agents.level.io

Agent communication with Level

online.level.io

Connectivity status checks

builds.level.io

Agent updates

downloads.level.io

Initial agent installation

realtime.ably.io

Real-time WebSocket for Level API

prd-level-storage.s3.wasabisys.com

File storage for automations

global.turn.twilio.com

TURN relay (used when P2P fails)

global.stun.twilio.com

STUN (used when P2P fails)

logs.logdna.com

Log collection for troubleshooting

💡 TIP: For firewalls that support wildcard rules, *.level.io and *.twilio.com cover the Level and Twilio entries above.

Troubleshoot filtered WebSocket connections

Allowing a hostname does not always allow the realtime connection. A content filter, firewall, or proxy can still block, inspect, rewrite, or close the WebSocket traffic.

If realtime client reports an error on a filtered network, ask the network administrator or filtering vendor to:

  • Allow outbound TCP 443 WebSocket connections to *.ably.io and *.ably-realtime.com.

  • Permit the WebSocket upgrade for those hosts.

  • Bypass SSL/TLS inspection, deep packet inspection (DPI), threat inspection, and content rewriting for those hosts.

  • Make sure proxy and session timeout policies do not terminate long-lived WebSocket connections.

These settings are troubleshooting requirements for filtered networks. A realtime client error does not, by itself, prove that filtering is the cause. After changing the network policy, run --check again and confirm that realtime client reports Connected.

Required ports (outbound only):

Port

Protocol

Purpose

Notes

80

TCP

HTTP

Basic connectivity

443

TCP

HTTPS

Primary agent traffic

3478

TCP & UDP

TURN

Used when P2P fails

5349

TCP

TURN TLS

Last-resort fallback only

10,000–60,000

UDP

TURN relay ports

Allocated by Twilio when TURN is used

ℹ️ NOTE: Ports 3478, 5349, and the UDP range are only needed when P2P connections can't be established. Start with 80 and 443 — that covers the vast majority of scenarios. See Relay/P2P Troubleshooting if remote connections specifically are failing.


Step 4: Contact Support

If --check doesn't point to a clear cause, contact Level Support with:

  • The full output from --check

  • AV/EDR software in use (name and version)

  • Any firewall or proxy software between the device and the internet


FAQ

  • Why is the agent pinging 8.8.8.8, and why does --check show Error: no ping stats while realtime is Connected? The agent runs a built-in latency probe (ICMP to 8.8.8.8, about every 50 seconds). That probe is separate from the Network Ping Monitor and is hard-coded today — there is no setting to change the target or disable only that probe. ICMP is not required for the agent's core API or realtime connection. If the device or network blocks ICMP, --check may report Error: no ping stats for online.level.io, agents.level.io, and the uptime monitor, and the average-ping value may stay blank — even when realtime client shows Connected. See Network Ping Monitor.

  • The device was online yesterday and just went offline. Where do I start? Run --check first. If it won't run at all, check whether level.exe (Windows) or the level binary (macOS/Linux) is still present on disk — if it's gone, AV/EDR removed it. If the binary is there but --check shows the service stopped or connection check failures, see AV/EDR False Detections for the quarantine log investigation and exclusion steps.

  • I don't have any way to access the device remotely right now. What can I do? If Level is your only remote access tool on the device, you'll need physical or out-of-band access (iDRAC, iLO, KVM, etc.) to investigate. Once you have access, run --check to identify the cause before doing anything else.

  • Do I need to open all these ports in my firewall? Not unless you're experiencing problems. Most networks work with just 80 and 443 outbound. The TURN ports (3478, 5349) and UDP range are only relevant if P2P connections are failing — see Relay/P2P Troubleshooting before opening additional ports.

  • The --check output looks fine but the device is still showing offline in Level. The console can lag a minute or two after connectivity is restored. If it doesn't update, the issue may be intermittent — try running --check again when the offline status recurs to catch the failure in the act.

Did this answer your question?