Writing Scripts with Sidekick¶

Sidekick provides an interface through the Automation Workbench to create, refine, and run scripts that blend both Python code and the capabilities of large language models (LLMs) to automate repeated tasks.

Elements of the Automation Workbench¶

Scripts¶

At the most fundamental level, a script in the Automation Workbench is Python code. The script executes within an environment that has built-in access to both the Binary Ninja Python API and a set of Sidekick Python APIs that include the following functionality:

Perform a task carried out by a large language model
Define parameters for the script
Query the binary using BNQL
Read from and write to Indexes
Display progress

Since scripts are just Python code, you can do anything with them that you can with Python, provided you import and install the necessasary dependencies into your Python environment. Since scripts have access to the full Binary Ninja Python API, anything you can do within Binary Ninja using the Python API, you can do with a script (including modifying the binary).

One of the powerful features of the Automation Workbench is that it allows you to blend the capabilities of LLMs with Python code to automate tasks that would otherwise be difficult or time-consuming to do manually. This is accomplished through the use of LLM Operators. To learn more about them and how to use them in your scripts, refer to LLM Operators.

Scripting Assistant¶

The Automation Workbench includes an AI-powered Scripting Assistant that provides interactive assistance for creating and refining scripts based on user request. The Scripting Assistant performs the following tasks:

Interpret the user's requests within the context of the conversation and generate Python code that implements the given request
Determine what LLM Operators are necessary for the described task and automatically generate the necessary LLM Operator definitions and their uses in the script
Inspect the script's execution output as part of the conversation context (so that it can refine the script based on the output)

Scripts Tab¶

The Automation Workbench contains the Scripts tab that has the following modes of operation:

Search Mode: Allows users to search for existing scripts and run, edit, or delete them; or describe a task to generate a new script
Edit Mode: Allows users to create, refine, and run scripts with optional support from the Scripting Assistant

Search Mode¶

The Scripts tab of the Automation Workbench opens in Search Mode by default. This mode provides the following UI elements:

Search text box for entering a search term to find existing scripts, or enter a task description for a new script
List of existing scripts that can be sorted by title, last modified, or last executed
Hamburger menu for additional actions:
- New Script: Create a new script
- Sort by: Sort the list of scripts by title, last modified, or last executed
Run button for executing selected scripts

Actions¶

The Search Mode provides the following actions:

Action	Description	Steps
Search	Filters existing scripts based on a search term	Enter a search term in the search text box
New Script from Description	Creates a new script from a task description, switches to Edit Mode, and prompts the Scripting Assistant to generate script code for the task	Enter a new task description in the search text box and press `Enter`
Import Scripts	Imports scripts into the Workbench from a .sidekick-scripts file	Click the hamburger menu, select `Import Scripts...`, and select the sidekick-scripts file to import
Export Scripts	Exports selected scripts to a .sidekick-scripts file	Select script(s), right-click the selection and select `Export Scripts...` or click the hamburger menu and select `Export Selected Scripts...`. In the `Export Scripts` dialog, click `Export`. (Note: Check option to include script revision history and Scripting Assistant messages if needed.)
Filter by Active Binary Views	Filters the list of scripts to only show those associated with the currently active binary views	Click the hamburger menu and select `Filter by Active Binary Views`
Associate Scripts with Binary	Associates the selected script(s) with the current binary	Select script(s), right-click the selection and select `Associate with <binary>`. (Note: This action associates the script(s) with the current binary, allowing them to be filtered using the `Filter by Active Binary Views` option.)
Run Script(s)	Executes the selected script(s)	Select script(s) and press `Cmd + Enter` (on MacOS) or `Cntrl + Enter` (on Windows/Linux); or right-click the selected script(s) and select `Run Script`
Edit Script	Opens the selected script in Edit Mode	Double-click the script, select the script and press `Enter`, right-click the selected script and select `Edit Script`, or select the script and press `Cmd + ]` (on MacOS) or `Cntrl + ]` (on Windows/Linux)
Delete Script(s)	Deletes the selected script(s)	Select script(s), right-click it, and select `Delete Script`
New Empty Script	Creates a new empty script	Click the hamburger menu and select `New Script`
Sort by	Sorts the list of scripts by title, last modified, or last executed	Click the hamburger menu and select the desired sort option

Note

When new scripts are added or imported into the script database, the list of scripts may not automatically update. One method for updating them is to enter any search term and then clear the search box.

Note

Scripts are executed sequentially from a queue as they are selected for execution by the user. Consequently, the user can queue up a script for execution while another is currently executing.

Edit Mode¶

The Scripts tab of the Automation Workbench can be switched to Edit Mode by selecting an existing script and performing one of the following actions:

Double-click the script
Select the script and press Enter
Right-click the selected script and select Edit Script

Edit Mode provides the following UI elements:

Left arrow button for returning to Search Mode
Title text box for setting the title of the script
Run button for executing the script code
Cancel button for canceling the execution of the script code (only while running)
Revisions drop-down combobox for viewing revisions of the script code
Main tab for editing the script code
Operator: <llm_operator_name> tab(s) containing a form for editing the LLM Operator (if included, one for each defined LLM Operator)
- Name: Unique identifier used to reference the LLM Operator
- Task Description: Textual high-level description of the task for the LLM Operator
- Model Name: Name of the model used for this LLM Operator
- Prompt Definition:
  - Input Variables: JSON description of variables required by the prompt instructions
  - Instructions: Plain text description of the instructions for the LLM to follow for the task
  - Output Schema: JSON description that defines the format of the desired output generated by the LLM
Scripting Assistant chat widget for interactions with the Scripting Assistant:
- Message box for entering a request to the Scripting Assistant
- Chat history for viewing the conversation with the Scripting Assistant
- Context Menu:
  - Revert script to here: Reverts the script code to the revision associated with the selected message
Output widget for displaying separate tabs for Output and Execution Details for an execution
Hamburger menu for additional actions:
- Show Chat: Displays/hides the Scripting Assistant chat widget
- Show Output: Displays/hides the output widget
- Delete Script: Deletes the current script
- Clone Script: Makes a copy of the current script
- Show Snippet Code: Displays the code necessary to run the current script from a code snippet
- Associate with <binary>: Associates the current script with the currently active binary view
- Disassociate from <binary>: Disassociates the current script from the currently active binary view
- Configure LLMs...: Opens the LLM Operator Model Catalog
- Code Preferences...: Sets coding preferences for the script (stored in the Binary Ninja setting sidekick.workbench.code_preferences)
- Submit Feedback...: Opens a dialog for submitting feedback on the Automation Workbench

Actions¶

The Edit Mode provides the following actions:

Action	Description	Steps
Return to Search Mode	Returns to Search Mode	Click the left arrow button at the front of the title text box, or press `Cmd + [` (on MacOS) or `Cntrl + [` (on Windows/Linux)
Set Script Title	Sets the title of the script	Enter the new title in the title text box
Edit Script Code	Edits the script code	Enter text in the `Main` code editor
Edit LLM Operator	Edits an LLM Operator definition	Select an `Operator: <llm_operator_name>` tab to edit, and apply edits in the form
View Revisions	Views the revisions of the script code	Click the `Revisions` drop-down combobox and select a specific revision
Save Revisions	Saves revisions of the script code	Revisions are automatically saved when the script is created, the Scripting Assistant updates the code, the `Run` or `Build` action is selected, or the user switches to Search/Edit Mode
Run Script	Executes the current script code	Click the `Run` button. During execution of the script code, output will be displayed in the `Output` tab of the Output widget.
Cancel Execution	Cancels the execution of the current script code	Click the `Cancel` button (only while running)
Send Message	Sends a message containing a request for the Script Assistant to complete based the conversation context	Enter a request in the Message box and press `Enter` or click the `Send a message` button
Revert Script to Here	Reverts the script code to the revision associated with the selected message in the chat history	Right-click a message in the chat history and select `Revert script to here`
Clone Script	Makes a copy of the current script	Click the hamburger menu and select `Clone Script`. This action copies the current script with a new title and opens the new script in Edit Mode.
Delete Script	Deletes the current script	Click the hamburger menu and select `Delete Script`. This action deletes the script and switches to Search Mode.
Show Snippet Code	Displays the code necessary to run the current script from a code snippet	Click the hamburger menu and select `Show Snippet Code`. This action opens a dialog window that contains the code necessary to run the current script from a code snippet, which includes the UUID of the script. This allows a user to run the script programmatically using the API.
Show Chat	Displays/hides the Scripting Assistant chat widget	Click the hamburger menu and select `Show Chat`
Show Output	Displays/hides the output widget	Click the hamburger menu and select `Show Output`
Associate with	Associates the current script with the currently active binary view	Click the hamburger menu and select `Associate with <binary>`. This action associates the script with the current binary, allowing it to be filtered using the `Filter by Active Binary Views` option in Search Mode.
Disassociate from	Disassociates the current script from the currently active binary view	Click the hamburger menu and select `Disassociate from <binary>`. This action disassociates the script from the current binary, allowing it to be filtered out using the `Filter by Active Binary Views` option in Search Mode.
Configure LLMs	Opens the LLM Operator Model Catalog	Click the hamburger menu and select `Configure LLMs...`
Set Code Preferences	Sets coding preferences for the script	Click the hamburger menu and select `Code Preferences...`. In the `Code Writing Preferences` dialog window that opens, enter instructions that capture the preferences you want and click `Save`.
Submit Feedback	Opens a dialog for submitting feedback on the Automation Workbench	Click the hamburger menu and select `Submit Feedback...`. This action opens a Feedback dialog window. Select whether or not the script accomplished your task and provide details on what worked or didn't work. Click `Submit` when finished.

Execution Monitor Tab¶

The Automation Workbench contains the Execution Monitor tab that displays the execution history of scripts. This tab includes a table that lists details for each script execution, including the following columns:

Status: Indicates the current status of the script execution
Started: Timestamp for when the script execution started
Finished: Timestamp for when the script execution finished
Script: Title of the script that was executed
Binary: Name of the binary that the script was executed on

Actions¶

The Execution Monitor tab provides the following actions:

Action	Description	Steps
Cancel Execution	Cancels the execution of a script that is currently running	Right-click the script execution entry and select `Cancel Execution`
Clear History	Clears the execution history table	Right-click the `Execution History` table and select `Clear History`

Handling Script Errors¶

Sometimes scripts will encounter errors during execution, which are displayed in the Output tab within Edit Mode. The Scripting Assistant has access to this output to help refine the script and resolve errors, but it needs to be instructed to do something about it. When this happens, send a message to the Scripting Assistant to let it know about the error (e.g. "An error occurred" or "Please fix that error").

Setting Code Generation Preferences¶

The Binary Ninja setting sidekick.workbench.code_preferences is used to store the code generation preferences for the Scripting Assistant. These preferences are used to generate code that aligns with your coding style and requirements. They are set from the Set Code Preferences action within Edit Mode (described above).

Setting Revisions to Keep¶

The Binary Ninja setting sidekick.workbench.revisions_to_keep is used to set the number of revisions to keep for each script when not associated with conversation messages (default=5). These are revisions that the user applies manually to a script. All revisions generated by the Scripting Assistant as a result of conversation messages are kept.

Writing Scripts¶

Defining Parameters¶

When writing scripts, you can define parameters that can be passed to the script when it is executed. This allows you to create reusable scripts that can be customized based on the input parameters. When creating a new script from the Scripts tab in Search Mode, the script editor will include a default set of parameters that can be modified within the script tool definition. You can add, remove, or change the parameters as needed.

Example default script tool definition with parameters:

tool = {
    "name": "my_tool",
    "description": "my tool description",
    "parameters": {
        "properties": {
            "arg1": {
                "type": "string",
                "description": "my arg1 description",
                "default": "my default value"
            }
        },
        "required": []
    }
}

try:
    # `get_tool_arguments` returns a dict containing the arguments
    args = get_tool_arguments(tool)
except Exception as e:
    print(f"Error: {e}")
    exit(1)

# ... your code here ...

When running a script that has parameters defined, you will be prompted to enter values for those parameters. The values you enter will be passed to the script as a dictionary, which can be accessed using the get_tool_arguments() API defined above.

Querying Using BNQL¶

From within your scripts, you can use the Binary Ninja Query Language (BNQL) to search for items in the binary. The query_binary() API defined below allows you to perform BNQL queries on a BinaryView object.

def query_binary(bv: BinaryView, query: str, limit: Optional[int] = None, time_limit_secs: Optional[int] = 30) -> Generator[Any, None, None]:
    """
    Search for objects in the binary view that match the given query.

    :param bv: The BinaryView to search.
    :param query: The query to search.
    :param limit: The maximum number of objects to return.
    :param time_limit_secs: The maximum time in seconds to spend evaluating the query. If None, there is no time limit.

    Examples:
        ```python
        # Search for all functions in the binary view.
        query_binary(bv, '/function')

        # Search for all data variables that start with "g_var" or "f_var".
        query_binary(bv, '/data[@name~=r"g_var.*|f_var.*"]')

        # Search for all strings that contain "http".
        query_binary(bv, '/string[@value~=".*http.*"]')

        # Search with a time limit of 5 seconds.
        query_binary(bv, '/function', time_limit_secs=5)
        ```
    """

Accessing Indexes¶

From within a script, you can access Indexes to store and retrieve entries related to your analysis.

Opening Indexes¶

In order to access an Index, you first need to open it. You can do this using the open_index() API defined below:

def open_index(bv: BinaryView, index_name: str, mode: IndexMode = IndexMode.WRITE, source_id: Optional[str] = None):
    """
    Opens an index for writing or appending entries.

    :param bv: The BinaryView to which the index pertains.
    :param index_name: The name of the index to open.
    :param mode: The mode in which to open the index.
    :param source_id: The unique identifier of the source script that is writing to the index.
    """

Example usage of open_index() to open an index named "My Index":

with open_index(bv, 'My Index') as index:
    pass

Opening an index in either IndexMode.READ or IndexMode.WRITE will create the index if it does not already exist. If the index already exists, it will be opened in the specified mode.

Writing to Indexes¶

To write to an opened Index, you can use the add_entry() method of the index object to add an entry and optional metadata. The entry is a Binary Ninja API object (e.g., Function, Variable, String, Symbol, HighLevelILInstruction, BasicBlock, etc), and metadata is a dictionary containing additional information about the entry.

Example usage of writing to an index:

with open_index(bv, 'All Functions', mode=IndexMode.WRITE) as index:
    for func in bv.functions:
        entry = func
        insn_cnt = len(list(func.instructions))
        metadata = {"instruction_count": insn_cnt}
        index.add_entry(entry, metadata)

Reading from Indexes¶

To read from an opened Index, you can iterate over the entries in the index using the index object returned by open_index(). The index object provides access to the entries stored in the index, which includes the address, object, and metadata associated with each entry.

Example usage of reading from an index:

with open_index(bv, 'High-Level Functions', mode=IndexMode.READ) as index:
    for entry in index:
        print(f"Address: {entry.address:#x}, Object: {entry.object}, Metadata: {entry.metadata}")

Displaying Progress¶

You can display progress in your script using the notify_progress() API defined below:

def notify_progress(current: int, total: int, message: str):
    """
    Displays a progress message in the Automation Workbench.
    :param current: The current progress value (e.g., the current iteration).
    :param total: The total value for the progress (e.g., the total number of iterations).
    :param message: The message to display with the progress.
    """

Example usage of notify_progress() to display progress in a script:

funcs = list(bv.functions)
for i, func in enumerate(funcs)
    notify_progress(i, len(funcs), f"Processing Function: {func.name}")
    # Perform some operation on the function...

Progress messages can be viewed in the following locations while a script is running:

In the area above the script editor widget within the Scripts tab while in Edit Mode
In the Status column of the Execution History table within the Execution Monitor tab while in Search Mode