Eigen RadarAI
Archive
Analysis

GitHub Copilot starts using desktop apps with user permission

GitHub opened desktop-app control in Copilot as a public preview on Windows and macOS. The assistant can click, type and move information through visible interfaces, including software without a dedicated connection. Users approve each application before control begins and can keep or reset that permission. Organizations can disable the feature, while macOS also requires operating-system permissions.

Artificial Intelligence··Evening
A screen showing blank colored application windows, a keyboard and a trackpad.

Copilot acts through the visible application interface

GitHub opened desktop-application control in Copilot, its AI assistant, as a public preview on Windows and macOS. Supported actions include clicking, typing, scrolling and dragging. The feature works through an application’s visible interface; the release remains a preview.[1], [2]

Both the Copilot command-line tool and desktop app expose the capability. The assistant can read accessibility information and visual context, then navigate or edit text. GitHub’s examples include entering an expense in a Safari form, updating part of a presentation and moving information between applications.[1]

Approval is granted per application

Copilot requests approval before taking control of an application. Users can grant persistent permission and later review or reset stored approvals. On macOS, the operating system also requires Accessibility and Screen Recording permissions. Application approval identifies which software the assistant may control, without a promise of separate approval for every action within it.[1]

Legacy software gains a route without dedicated connections

GitHub describes computer use as a route into legacy software lacking a dedicated API or command-line connection. Organization administrators can disable the feature entirely. Users can enable or turn it off in the command-line tool, inspect current permissions, or use the desktop app’s computer-use settings. GitHub recommends specifying the intended result, applications and constraints in a request.[1]

References

  1. News sourceGitHubGitHub Copilot previews control of desktop applications↩1↩2↩3↩4
  2. News sourceThe WinCentralGitHub Copilot previews control of desktop applications↩
AI safety and regulation

More analysis from this desk

View all Brief analyses from the Artificial Intelligence desk →