OpenAI's ChatGPT is starting to work with other apps on your computer.

On Thursday, the startup announced the ChatGPT desktop app for macOS can now read code in a handful of developer-focused coding apps, such as VS Code, Xcode, TextEdit, Terminal, and iTerm2.

That means developers will no longer have to copy and paste their code into ChatGPT, which has become a common way to use the chatbot. Now when the feature is enabled, OpenAI will automatically send the section of code you're working on through its chatbot as context, alongside your prompt.

However, unlike popular AI coding tools such as Cursor or GitHub Copilot, ChatGPT is currently unable to write code directly into developer apps on your behalf.

The feature, called Work with Apps, is far from an AI agent, but OpenAI says getting ChatGPT to understand other apps is a "key building block" toward building agentic systems. One of the biggest challenges facing AI agents today is getting them to understand the rest of your computer screen, as opposed to prompts or their own responses.

OpenAI says it's focusing this feature on coding apps to start; this is likely because AI coding assistants have taken off as one of the most popular use cases for LLMs. The feature is available to Plus and Teams users today and will roll out to Enterprise and Edu in the next few weeks. OpenAI says ChatGPT will be able to work with other types of apps moving forward, specifically text-based apps that could be used for writing tasks.

In a demo with TechCrunch, an OpenAI employee opened the ChatGPT app and an Xcode environment containing a simple project modeling the solar system — although it was missing Earth. The employee selected an Xcode tab within ChatGPT, which tells the AI chatbot to look at the app, and prompted the chatbot to "add the missing planets." The chatbot was able to complete the task, writing a line of code to represent Earth that matched the rest of the project's format. They still had to paste ChatGPT's answer back into their environment, though.

In order to read different apps, OpenAI is mostly relying on the macOS accessibility API to read text and translate it to ChatGPT, according to OpenAI desktop product lead Alexander Embiricos. The screen reader on macOS, which helps Apple's VoiceOver feature work, has been around for nearly two decades. It's generally considered pretty reliable for most common apps, but not everything.

For some apps, such as Microsoft's VS Code, Work with Apps requires users to install a special extension to query content. And, as the name suggests, Apple's screen reader can only read text, so it can't help ChatGPT understand visual elements, such as photos, the orientation of objects, or videos.