ServicesBlogPricingContactContact Now
AI3 min read

SGLang RCE Exploit Hits NEAR AI Inference Server

SGLang RCE Exploit Hits NEAR AI Inference Server

Introduction

On Sunday an attacker leveraged a remote code execution vulnerability in the open‑source inference engine SGLang to gain unauthorized code execution on one of NEAR AI's inference servers. The company disclosed the breach earlier this week, confirming that the compromise allowed the adversary to observe every request processed by that server during the day, including traffic that users had submitted through a confidential mode designed to run inside a trusted execution environment.

Technical Overview of the Exploit

SGLang serves as a high‑performance serving layer for large language models, exposing APIs that accept model inference requests. The flaw resides in the request handling path where untrusted input is deserialized without adequate sandboxing, enabling an attacker to inject arbitrary commands that the server executes with the privileges of the inference process. Because the affected server hosted multiple tenant workloads, a single successful injection gave the attacker visibility into all concurrent sessions.

Scope of Data Exposure

NEAR AI reported that the compromised server processed requests from eighteen distinct agents and roughly one hundred individual users on the day of the intrusion. The confidential mode, which isolates request data inside a hardware‑based trusted execution environment, was also active on that server. Although the TEE is intended to protect data from the host operating system, the attacker’s code execution occurred at the application layer before the request entered the enclave, meaning the plaintext payloads were observable.

Why the Confidential Mode Did Not Prevent the Leak

Confidential computing guarantees that data remains encrypted while at rest and in memory, but it does not protect against a compromised application that reads the data before it is sealed. In this case the exploit triggered code execution in the inference runtime prior to the hand‑off to the enclave, so the attacker could capture the clear‑text prompts and any attached metadata. This illustrates a classic limitation: the security boundary of a TEE ends at the point where the application hands off data.

  • Apply the latest upstream patches for SGLang as soon as they are released; monitor the project’s security advisories for a formal fix.
  • Deploy inference workloads in isolated containers or virtual machines with strict seccomp profiles that deny execve and other dangerous syscalls.
  • Enforce network segmentation so that inference servers cannot reach sensitive internal services.
  • Enable runtime integrity monitoring (e.g., Falco, Sysdig) to alert on unexpected process creation or file writes inside the inference container.
  • Rotate any API keys, model weights, or user tokens that may have been exposed during the window of compromise.
  • Conduct a post‑incident review of request logs to identify any anomalous payloads that could indicate further lateral movement.

Grivyonx Expert Analysis

The incident underscores how a single deserialization flaw in an AI serving stack can cascade into a multi‑tenant data breach, even when confidential computing hardware is present. Organizations adopting large‑language‑model inference at scale should treat the serving layer as a critical attack surface, applying defense‑in‑depth controls such as sandboxing, immutable images, and continuous runtime attestation.

Conclusion

The SGLang remote code execution event demonstrates that open‑source AI infrastructure can become a conduit for large‑scale data leakage if not hardened rigorously. Security teams should prioritize patch management, workload isolation, and real‑time behavioral monitoring for all inference endpoints. Grivyonx’s AI automation and DevSecOps services help teams embed these controls into CI/CD pipelines and cloud deployments, reducing the window of exposure for similar vulnerabilities.

  • #SGLang
  • #RCE
  • #NEAR AI
  • #inference engine
  • #supply chain
  • #confidential computing
  • #incident response
  • #AI security
Gourav Rajput

Gourav Rajput

Founder of Grivyonx Technologies