Back to AI Research

AI Research

SAT-Edge-Agent: Hardware-in-the-Loop Edge-Agent Orc... | AI Research

Key Takeaways

  • SAT-Edge-Agent is a hardware-in-the-loop (HIL) system designed to manage onboard satellite intelligence by translating mission intent into actionable tool ca...
  • Onboard satellite intelligence requires a task layer that translates mission intent into local tool calls, exposes execution state, and returns machine-consumable artifacts under communication and power constraints.
  • We present SAT-Edge-Agent, a hardware-in-the-loop (HIL) edge-agent system deployed on a commercial off-the-shelf ARM-based heterogeneous edge system-on-chip.
  • A browser workspace and FastAPI agent coordinate a local OpenAI-compatible language service with a project-internal YOLO-style oriented-object-detection endpoint that returns FAIR1M metadata-backed structured results.
  • Two fixed FAIR1M workloads, one single-image and one serial two-image request, were repeated 20 times each and completed 20/20 attempts.
Paper AbstractExpand

Onboard satellite intelligence requires a task layer that translates mission intent into local tool calls, exposes execution state, and returns machine-consumable artifacts under communication and power constraints. We present SAT-Edge-Agent, a hardware-in-the-loop (HIL) edge-agent system deployed on a commercial off-the-shelf ARM-based heterogeneous edge system-on-chip. A browser workspace and FastAPI agent coordinate a local OpenAI-compatible language service with a project-internal YOLO-style oriented-object-detection endpoint that returns FAIR1M metadata-backed structured results. Two fixed FAIR1M workloads, one single-image and one serial two-image request, were repeated 20 times each and completed 20/20 attempts. Mean Full-Agent latency was 29.353 s and 60.937 s, with empirical P95 values of 31.166 s and 66.882 s. Mean detector time was 861.386 ms and 1510.920 ms, only 2.93% and 2.48% of the corresponding Full-Agent means. Profiling indicates that most visible latency occurs outside detector execution. Mean CPU utilization was 20.761% and 20.482%. A 200-ms NPU-load field averaged 100% for both workloads, but it represents a shared-accelerator software field rather than detector-only occupancy or calibrated utilization. The public evidence package provides sanitized request-level records, redacted JSON, normalized SSE examples, and scripts reproducing the reported statistics. These results establish a reproducible HIL boundary for observable satellite edge-agent orchestration, but do not establish detector accuracy, a new geolocation method, calibrated energy efficiency, or flight readiness.

SAT-Edge-Agent is a hardware-in-the-loop (HIL) system designed to manage onboard satellite intelligence by translating mission intent into actionable tool calls. Developed by Longji He and Jeto Xu, the system acts as an orchestration layer that coordinates local perception and language services on a commercial off-the-shelf (COTS) ARM-based edge system-on-chip (SoC). Its primary goal is to enable spacecraft to process imagery locally, producing structured mission artifacts—such as object detections and geographic metadata—before deciding whether to downlink raw data.

System Architecture

The system organizes onboard intelligence into four distinct layers: a browser-based operator workspace, a FastAPI-based agent backend, a YOLO-style oriented-object-detection (OBB) tool, and a local OpenAI-compatible language service. When a mission request is received, the agent backend determines if detection is necessary, invokes the vision tool, and processes the results. The system is designed to stream execution state to the operator via Server-Sent Events (SSE), separating machine-facing structured artifacts from optional natural-language summaries. This allows downstream software to consume detection data independently of the narrative output.

Experimental Performance

The authors evaluated the system using two fixed FAIR1M remote-sensing workloads: a single-image request and a serial two-image request. Across 20 repeated attempts for each workload, the system achieved a 100% completion rate.
Performance metrics indicate that the majority of latency occurs outside the actual detector execution. For the single-image workload, the mean Full-Agent latency was 29.353 seconds, while the detector time averaged 861.386 milliseconds (2.93% of the total). For the serial two-image workload, the mean Full-Agent latency was 60.937 seconds, with the detector accounting for 1,510.920 milliseconds (2.48% of the total). CPU utilization remained consistent at approximately 20.5% to 20.8% across both workloads.

Limitations and Scope

The authors specify that these results are intended to establish a reproducible HIL boundary for edge-agent orchestration rather than to claim flight readiness or superior performance. The study does not provide an accuracy benchmark for the detector, nor does it propose a new geolocation method. While the system propagates geographic metadata from the FAIR1M dataset, it does not implement independent sensor-model or map-registration fallbacks. Furthermore, the reported NPU load of 100% is noted as a shared-accelerator software field rather than a calibrated measure of detector-only occupancy.

Why It Matters

This research addresses the systems-level challenge of moving intelligence closer to the sensor in low-Earth-orbit missions. By providing a framework that wraps edge inference in a reproducible agent workflow, the authors demonstrate that spacecraft can function as constrained edge computers capable of generating mission-ready products. The public evidence package, which includes sanitized request-level records and reproduction scripts, is intended to help researchers evaluate integration and service contracts in a controlled, hardware-based environment.

Comments (0)

No comments yet

Be the first to share your thoughts!