Desktop Automation on Mac: AI Controls Your System Settings, Windows, and Apps

AI assistants can write your code and answer your questions. But ask one to move a window to the left half of your screen, or toggle dark mode, or mute your Mac before a meeting, and it can't. It doesn't have hands. It doesn't have access to your operating system. It can generate text about window management, but it can't manage a single window.

The gap between "AI that talks" and "AI that acts" is where most tools stop. On your Mac, that gap is now closed.

Why can't existing tools bridge AI and macOS automation?

Mac automation has existed for decades, but every option requires you to become a programmer or learn a niche tool.

AppleScript is powerful but archaic. Its syntax reads like English written by someone who hates English. Writing a script to snap a window to the left half of your screen takes 15 lines of tell blocks and coordinate math. Debugging it takes longer than doing the task by hand.

Apple Shortcuts are GUI-only and limited. You can build simple automations by dragging blocks, but anything involving window management, system preferences, or multi-step logic hits a wall quickly. And there is no AI integration.

Hammerspoon is free and genuinely powerful, but it requires learning Lua scripting. You are writing actual code to automate your desktop. That is a fine hobby if you enjoy it, but it is not a solution for people who want to say "mute my Mac" and have it happen.

Keyboard Maestro costs $36 and has a steep learning curve. It can do almost anything, but building complex macros involves a visual programming environment that takes weeks to master.

None of these tools integrate with AI. None of them let you describe what you want in plain language and have it executed. They all require you to learn their specific interface, syntax, or scripting language first.

What does it mean for AI to control your Mac?

Here is what actually happens when AI controls your desktop.

An LLM receives tool definitions for system actions. These definitions describe what each action does, what parameters it accepts, and what it returns. When you ask the AI to do something, like "set display brightness to 60%," it selects the right tool (action_display), calls it with the right parameters ({"action": "set_brightness", "level": 60}), and the action executes natively through macOS APIs.

This is not screen scraping. It is not simulated mouse clicks. It is not AppleScript running under the hood. The actions call real macOS frameworks, the same APIs that Apple's own apps use. The AI decides what to do. The native code does it.

You interact through natural language, either by typing in a chat or by speaking a voice command. The AI handles the translation from your intent to the specific API call. There is no macro editor, no recording step, no configuration file to maintain. You describe an outcome and the system produces it.

This also means multi-step commands work naturally. "Mute my Mac, turn on Do Not Disturb, and set the display to 40% brightness" is a single instruction. The AI breaks it into three tool calls and executes them in sequence. Try doing that in Shortcuts without building three separate blocks and wiring them together.

What can ActionPiper actually control?

ActionPiper is a free macOS menu bar app that exposes 26 action domains with roughly 137 individual actions. Here is what those domains cover.

Window management. List all windows, snap to left/right/top/bottom halves, move to specific coordinates, resize, close, minimize, toggle fullscreen, focus by app name, and arrange windows in grid layouts. No manual coordinate math. Say "snap Safari to the left" and it happens.

Input simulation. Type text, press key combinations, click at coordinates, move the mouse, drag, and scroll. Useful for automating repetitive form-filling or interacting with apps that don't have proper APIs.

Audio. Volume up, down, set to a specific percentage, mute, and select input or output devices. "Switch audio output to my AirPods" is a single command.

Display. Adjust brightness, toggle Night Shift, control True Tone. "Turn on Night Shift and set brightness to 40%" works as one instruction.

Network. Toggle Wi-Fi, scan for networks, join a specific network, check DNS, and test connectivity. Useful for quick diagnostics without opening System Settings.

Bluetooth. Toggle Bluetooth on/off and list connected devices.

Dock. Show or hide the Dock, change its position (left, bottom, right), adjust size, and control magnification.

Desktop. Set wallpaper and toggle Stage Manager.

Spaces. List your Spaces, switch between them, and create new ones.

Focus and Do Not Disturb. Toggle Focus, list available Focus modes, and activate a specific mode. "Turn on Do Not Disturb" before a meeting, without touching System Settings.

Media. Play, pause, skip to the next or previous track. Works with whatever media app is currently active.

Power. Sleep, restart, shut down, check battery status, and start the screen saver.

App control. Launch apps, quit apps, list what is running, take accessibility snapshots, and run assertions against app state.

Process management. List running processes, kill by name or PID, and check CPU and memory usage.

Finder. Open a path, reveal a file in Finder, and empty the trash.

Accessibility. Toggle VoiceOver, control zoom, and adjust contrast settings.

Appearance. Switch between dark and light mode, set the accent color, and toggle Reduce Motion.

Notifications. Post a macOS notification with custom text, or clear existing notifications.

Calendar. List upcoming events and create new ones.

Contacts. Search your contacts and create new entries.

Reminders. List reminders, create new ones, and mark them complete.

System info. Get your hostname, macOS version, and hardware specs.

Storage. Check disk usage and list mounted volumes.

Defaults. Read and write macOS user defaults, the underlying preference system that apps use for configuration.

Location. Get your current location, if you have granted permission.

Shortcuts. Run any Siri Shortcut by name. This means every Shortcut you have already built becomes accessible through AI.

How does AI integration work?

All 26 domains are exposed as 29 MCP tools. Some domains that share a natural grouping are combined into a single tool with sub-actions.

You can use these tools in two ways.

In ModelPiper chat. Open the chat, type "snap this window to the left half of my screen," and the AI dispatches the action_window tool with the correct parameters. You see the action execute and get a confirmation.

Via Claude Code or any MCP client. If you use Claude Code, Cursor, or another tool that supports MCP (Model Context Protocol), you can add ActionPiper's tools to your environment. Then AI actions become part of your development workflow. "Mute my Mac, switch to dark mode, and open the project in Finder" becomes a single prompt.

Push-to-Command. Hold the Right Command key, speak a command, and release. Speech-to-text transcribes your words on the Neural Engine. The LLM interprets them against the full set of tool definitions. ActionRouter executes the action. A macOS notification confirms what happened. The entire loop, from voice to execution, runs locally on your hardware. No internet required. No audio sent to a server. You speak, your Mac acts.

This is genuinely different from Siri. Siri routes your voice through Apple's servers (unless you have enabled on-device processing for a limited set of commands). Siri supports a fixed set of intents defined by Apple. ActionPiper's LLM interprets free-form language against 29 tool definitions with full parameter schemas. The difference is flexibility: Siri understands "set a timer for 5 minutes." ActionPiper understands "snap the frontmost window to the top-right quarter, set brightness to 70%, and switch audio output to my external speakers."

What are the honest limitations?

ActionPiper is powerful, but it is not magic. Here is what you should know before installing.

Permissions are required. Many actions need accessibility permissions, which you grant in System Settings, Privacy and Security, Accessibility. Calendar, Contacts, and Reminders each require their own permission grant. macOS asks you to approve each one individually, which is how it should be. You are in control of what the app can access.

ActionPiper is DMG-distributed, not on the App Store. The macOS App Store sandbox prohibits the accessibility APIs that ActionPiper needs to function. Window management, input simulation, and process control all require entitlements that Apple does not allow in sandboxed apps. You download the DMG from modelpiper.com and install it manually.

macOS only. ActionPiper uses Apple-specific frameworks. There is no Windows or Linux version.

AI interpretation is not perfect. The LLM does its best to map your natural language to the correct action and parameters, but ambiguous commands can produce unexpected results. "Make the window bigger" is vague. "Resize Safari to 1200 by 800" is precise. The more specific you are, the better it works.

How does this compare to other Mac automation tools?

The comparison is not entirely apples to apples, because ActionPiper's primary interface is natural language while the alternatives are manual configuration. But here is how they stack up on the dimensions that matter.

ActionPiper requires zero scripting. You describe what you want, and the AI figures out the API call. Keyboard Maestro and Hammerspoon both require you to build the automation yourself. Apple Shortcuts is visual but limited in scope. Only ActionPiper has AI integration, voice control, and MCP tool exposure.

The tradeoff: Hammerspoon and Keyboard Maestro can build automations that ActionPiper cannot, because they support arbitrary logic, loops, and conditionals. If you need "every Tuesday at 9 AM, check if Slack is running, and if not, launch it and set Focus to Work," that is a multi-condition timed automation that ActionPiper does not support today. ActionPiper's strength is not unlimited programmability. It is that you don't need to program at all.

There is also a cost consideration. ActionPiper is free. Keyboard Maestro is $36 with no subscription but charges for major upgrades. Hammerspoon is free but demands a real time investment in Lua. Apple Shortcuts is built-in but cannot do half of what the others can. ActionPiper sits in a unique spot: free, zero learning curve, AI-native, and broad enough to cover the actions most people actually need.

How do you get started?

Download ActionPiper from modelpiper.com. It installs as a menu bar app. Grant the accessibility permission when macOS prompts you. Grant additional permissions (Calendar, Contacts, Reminders) as needed for those specific domains.

Open ModelPiper chat and try a command: "Toggle dark mode." "Set volume to 30%." "List my running apps." "Snap this window to the right half." Each one dispatches the corresponding action through native macOS APIs.

For voice control, hold Right Command and speak. For MCP integration with Claude Code, add the ToolPiper MCP server to your configuration and all 29 action tools become available in your development environment. The setup command is one line: claude mcp add toolpiper -- ~/.toolpiper/mcp.

Your commands, your system state, and your automation logic never leave your Mac.

This is part of a series on local-first AI workflows on macOS. Next up: Push-to-Talk AI, voice commands that control your Mac without touching the keyboard.