2026 AI & Machine Learning
WWDC26 · 17 min · AI & Machine Learning
Build AI-powered scripts with the fm CLI and Python SDK
Explore all the new ways to leverage Apple Foundation Models on macOS. The Foundation Models SDK for Python lets you integrate with popular tooling and evaluation packages in the Python ecosystem. Find out how to use the brand new fm command introduced in macOS 27 to streamline scripting, automate model workflows, and accelerate your development process.
Watch at developer.apple.com ↗Chapters
- 0:00 — Introduction
- 1:22 — Introducing the fm CLI and Python SDK
- 3:23 — Command line tool
- 5:02 — fm respond and structured output
- 6:11 — Automating file management with fm
- 8:52 — Python SDK
- 9:42 — Prompting, tool calling and guided generation
- 10:44 — Building an evaluation pipeline in Python
- 15:20 — Next steps
Code shown on screen · 6 snippets
Prompt the on-device model with fm respond
$ fm respond "Provide a basic regex in Swift to parse an email address"
# Here is a basic regex to parse an email address in Swift: [...]
$ fm respond "Provide a comprehensive regex in Swift to parse an email address" --model pcc
# [...] Here's a robust Swift implementation using 'NSRegularExpression' to validate a typical email address:
$ fm respond "What app is the user using in this screenshot?" --model pcc \
--image Screenshot.png
# The user is using the Mail app.
$ fm schema object --name AppsIdentified --string app_names --array > schema.json
$ fm respond "What apps are the user actively using in this screenshot?" \
--image Screenshot.png --model pcc --schema schema.json
# {"app_names": ["Messages", "Mail", "Calendar"]}
$ fm respond --help Sort files with fm respond and a schema
fm schema object --name "TriagedFileList" \
--string 'final_files' --array \
--string 'draft_files' --array > /tmp/schema.json
output=$(fm respond \
--instructions "I just completed a project, and I need help triaging the latest version of the files from the previous versions. I will give you a list of files. Return a list of the latest files (i.e., all files that, you can infer from their name in the list, are the latest versions), and then return separately a list of all draft files (i.e., all files that weren't considered final)." \
"This is the list of all files:\n\n${files_list}" \
--schema /tmp/schema.json
)
echo "${output}" | jq -r '.final_files[]' | while read -r file; do
cp "${DIRECTORY_TO_TRIAGE}/${file}" "${FINAL_FILES_STORAGE_DIRECTORY}"
done
echo "${output}" | jq -r '.draft_files[]' | while read -r file; do
mv "${DIRECTORY_TO_TRIAGE}/${file}" "${DRAFT_FILES_STORAGE_DIRECTORY}"
done Install the Foundation Models Python SDK
pip install apple_fm_sdk Create a session and respond to a prompt
import apple_fm_sdk as fm
INSTRUCTIONS = "You're an AI assistant for Cupertino Mart, a grocery store with in-app ordering."
async def answer_question(prompt: str) -> str:
session = fm.LanguageModelSession(instructions=INSTRUCTIONS)
return await session.respond(prompt) Define a Tool for the language model
class GetPastOrdersTool(fm.Tool):
name = "get_past_orders"
description = "Retrieves information about this user's past orders."
class Arguments:
number_orders: str = fm.guide("How many of the last orders to retrieve")
def arguments_schema(self) -> fm.GenerationSchema:
return self.Arguments.generation_schema()
async def call(self, args: fm.GeneratedContent) -> str:
number_orders = args.value(int, for_property="number_orders")
return await Orders.load_last_orders(user_id=user_id, amount=number_orders) Generate structured output with @fm.generable
class ItemsSuggestion:
item_names: list[str] = fm.guide("Names of the suggested items")
INSTRUCTIONS = "You're an AI assistant tasked with returning potential grocery items that the user might be interested in."
async def generate_suggested_cart_items(user_input: Optional[str]) -> ItemsSuggestion:
session = fm.LanguageModelSession(instructions=INSTRUCTIONS, tools=load_tools())
prompt = """Using the tools to load the user's previous orders, \
return a list of items the user has already ordered \
and that they might be interested in again \
as they're getting ready to place a new grocery order."""
if user_input is not None:
prompt += f"\nAccount for the following request from the user: {user_input}"
return await session.respond(prompt, generating=ItemsSuggestion) Resources
Related sessions
-
21 min -
11 min -
21 min -
22 min