NVIDIA has documented a local workflow for developing AI applications on DGX Spark with TensorRT Edge-LLM. Developers can test a text model in a terminal and expose it through an OpenAI-compatible local interface. The runtime links this development environment with supported embedded platforms, while the published example remains a prototype-and-testing workflow rather than a production deployment.
NVIDIA has published a workflow for developing embedded AI applications on DGX Spark, its development computer. TensorRT Edge-LLM, the company’s C++ model-running software for embedded workloads, downloads a selected model and prepares a TensorRT inference engine through the trtedge command-line tool. The example uses Nemotron 3.5 Lightning, a text model, to test responses directly in a terminal.[1], [2]
The same setup can serve the model through a local interface compatible with OpenAI’s application programming interface. An application under development can send prompts to that local server and receive streamed responses. Matching engines are cached for reuse on later launches. The documented sequence covers preparing the engine, checking that the server is ready and sending a test request.[1]
NVIDIA describes the runtime as spanning development on Spark and deployment on supported Jetson Thor, DRIVE AGX Thor and IGX Thor platforms. The guide directs developers to model-specific requirements for their eventual target. Its demonstrated local server is for application prototyping and testing. Moving to another supported device still involves the requirements of that model and platform.[1]
