Docs / AI agents

Agent tools

All 44 tools Devicewing gives an AI agent, grouped by what they act on.

Device and view

Tool What it does
status What Devicewing is showing and which simulator it uses
set_pose Fold the iPhone Duo to flat, closed, book, laptop or tent
fold Set the iPhone Duo’s hinge: 0 closed to 180 open flat
rotate Turn a device about its x, y and z axes, absolute or relative
set_view Finish, zoom, background, window theme, keep on top, active tool
board Show the Board, pick its devices, set the lead, Sync, True Size, presets
list_devices The devices Devicewing can show
list_simulators Their simulators
select_simulator Use a particular simulator, by UDID or name

set_pose, fold and rotate return once iOS has finished redrawing for the move.

The screen

Tool What it does
capture_screen The screen in use, upright, one pixel per point; returns the image
describe_screen The accessibility elements on screen, in the same points
tap Tap at a point
tap_element Tap by label, value or identifier
swipe Swipe between two points
pinch Two fingers, apart or together, with an optional twist
type Type into the focused field
key Press a key, with modifiers
get_pasteboard, set_pasteboard Read or set the pasteboard
button Home, lock, side, top, Action, Siri, App Switcher, volume
wait_for_element Wait until an element appears, or is gone
wait_for_idle Wait until the screen stops redrawing
wait_for_change Wait until the screen changes

Apps

Tool What it does
install_app Install a simulator build; returns its bundle ID
launch_app Launch by bundle ID, with arguments, optionally relaunching
terminate_app Quit it
uninstall_app Remove it
list_apps Installed apps
app_logs An app’s recent log lines
open_url Open a URL in the simulator

Simulator settings

Tool What it does
set_appearance Light or dark
set_status_bar Time, battery, signal, operator; or clear it
set_location A latitude and longitude, or clear
set_permission Grant, revoke or reset a privacy permission for an app
send_push Send a push notification payload to an app

Capture

Tool What it does
screenshot The 3D view, at a size or fitted; returns the image
capture_options Read or change every capture setting
capture_set Every pose, both Duo screens, or both, into a folder
capture_board Every Board screen and one image of them side by side
play_move A full turn, side to side, fold open or fold shut, recorded or not
record_start, record_stop Record the 3D view
screen_record_start, screen_record_stop Record the screen; on the Board, every screen

send_push only reaches an app that asked for notification permission and was allowed.

Something wrong on this page? Email [email protected].