From 512fb997c3c97ed2b4d0018be923fca56f963884 Mon Sep 17 00:00:00 2001 From: Brian Potchik Date: Sat, 16 Nov 2024 13:52:31 -0500 Subject: Update documentation for the Workflows feature. --- docs/dev/workflows.md | 663 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 530 insertions(+), 133 deletions(-) (limited to 'docs/dev') diff --git a/docs/dev/workflows.md b/docs/dev/workflows.md index 963371b9..5021fb49 100644 --- a/docs/dev/workflows.md +++ b/docs/dev/workflows.md @@ -1,206 +1,603 @@ -Binary Ninja Workflows Documentation -=== +# Binary Ninja Workflows --- -# Early Feature Preview -This capability is experimental with no guarantees of API stability or future compatibility. Binary Ninja Workflows is actively under development as a functional prototype. Feedback and questions are welcome! +## **Contents** +- [Introduction](#introduction) +- [Concepts](#concepts) + - [Workflow](#workflow) + - [Activity](#activity) + - [Workflow Machine](#workflow-machine) +- [Workflow Fundamentals](#workflow-fundamentals) + - [Workflow Selection](#workflow-selection) + - [Foundational Workflows](#foundational-workflows) + - [Workflow Cloning and Registration](#workflow-cloning-and-registration) + - [Querying Registered Workflows](#querying-registered-workflows) + - [Example: NOP'ing Module-Level Analysis](#example-noping-module-level-analysis) + - [Example: Strings-Only Module-Level Analysis](#example-strings-only-module-level-analysis) + - [Example: Custom Function-Level Analysis](#example-custom-function-level-analysis) + - [Workflow Configuration](#workflow-configuration) +- [Activity Fundamentals](#activity-fundamentals) + - [Key Characteristics](#key-characteristics) + - [Activity Configuration](#activity-configuration) + - [Activity Roles](#activity-roles) + - [Activity Eligibility](#activity-eligibility) + - [Defining Activities](#defining-activities) + - [Activity with Run-Once Eligibility](#activity-with-run-once-eligibility) + - [Activity with Auto-Generated Eligibility](#activity-with-auto-generated-eligibility) + - [Activity with Setting-Based Eligibility](#activity-with-setting-based-eligibility) + - [Activity with View-Type Eligibility](#activity-with-view-type-eligibility) + - [Activity with Compound Eligibility](#activity-with-compound-eligibility) + - [Activity Eligibility Overrides](#activity-eligibility-overrides) +- [Workflow Machine Fundamentals](#workflow-machine-fundamentals) + - [WorkflowMachine CLI Commands](#workflowmachine-cli-commands) +- [Workflow Topology](#workflow-topology) +- [Advanced Topics (Coming Soon)](#advanced-topics-coming-soon) ---- -# Contents --- -- [What is Workflows](#what-is-workflows) -- [Getting Started](#getting-started) - - [Enable](#enable) - - [Explore](#explore) - - [Examples](#examples) -- [Basic Concepts](#basic-concepts) - - [Activity](#activity) - - [Workflow](#workflow) - - [Analysis Context](#analysis-context) - - +## Introduction ---- -# What Is Workflows ---- +Binary Ninja's **Workflows** provide a powerful and flexible framework for customizing and extending binary analysis. This framework allows you to tailor the analysis process to meet your specific needs. With Workflows, you can: -Binary Ninja Workflows is an analysis orchestration framework which simplifies the definition and execution of a computational binary analysis pipeline. The extensible pipeline accelerates program analysis and reverse engineering of binary blobs at various levels of abstraction. Workflows supports hybridized execution models, where the ordering of activities in the pipeline can be well-known and procedural, or dynamic and reactive. Currently, the core Binary Ninja analysis is made available as a procedural model and is the aggregate of both module and function-level analyses. +- **Customize Analysis Pipelines**: Define your own sequences of analysis steps, adjusting the order, inclusion, or exclusion of specific analyses to suit different binaries, architectures, or project requirements. + +- **Gain Granular Control Over Analysis Stages**: Introduce additional, configurable analysis stages or modify existing ones to focus on areas of interest, optimize performance, or handle unique binary features. + +- **Extend and Scale Analysis Capabilities**: Seamlessly integrate new analysis stages, plugins, or custom scripts, allowing the analysis pipeline to adapt as new challenges or requirements arise. + +- **Utilize Concurrent and Asynchronous Execution**: Leverage advanced execution models like subflows and tasks to perform analyses concurrently, improving efficiency and enabling complex analysis strategies. + +By leveraging Workflows, you gain unparalleled control over the decompilation and analysis process. This flexibility empowers you to perform in-depth, tailored binary analysis, adapting quickly to evolving needs and supporting future enhancements. ---- -# Getting Started --- +## Concepts +### Workflow -## Explore +A **Workflow** in Binary Ninja defines the set of analyses to perform on a binary, including their dependencies and execution order. Workflows are represented as Directed Acyclic Graphs (DAGs), where each node corresponds to an **Activity** (an individual analysis or action). We have built an API around modifying and extending this structure, enabling you to tailor the decompilation and analysis process to your specific needs. -The procedural core analysis model resembles a behavior tree and the activities execute as a depth-first pre-order traversal of the tree. In order to represent complex data flows, there is fine-grained control for conditional execution and subtree parallelism. It's possible to visualize the pipeline topology in the UI with `show_topology`, as shown below. There are two visual representations for the pipeline topology, a *Composite View* and a *Sequential View*. +Workflows enable you to: -``` -Workflow().show_topology() -``` +- **Control Execution Flow**: Precisely define the order in which analyses are executed, ensuring that all dependencies are respected. -It's possible to create multiple workflows to satisfy the need for tailored analysis across a diverse set of binaries and goals. List available workflows as shown below. +- **Customize Analysis Composition**: Include, exclude, or modify analysis stages to focus on areas of interest or optimize performance. -``` ->>> list(Workflow) -[, , , ] -``` +- **Define Dependencies**: Clearly specify how different analyses depend on each other, facilitating complex analysis strategies. -## Examples +- **Extend Functionality**: Seamlessly integrate custom analyses or third-party plugins into the analysis process. ---- -### Python +There are two distinct types of Workflows: +- **Module Workflows**: + - Applied to `BinaryView` objects. + - Perform analysis on the entire binary. + - Coordinate high-level analysis tasks and oversee function-level analysis. -``` -pwf = Workflow().clone("PythonLogWarnWorkflow") -pwf.show_topology() -pwf.register_activity(Activity("PythonLogWarn", action=lambda analysis_context: binaryninja.log.log_warn("PythonLogWarn Called!"))) -pwf.insert("core.function.basicBlockAnalysis", ["PythonLogWarn"]) -pwf.register() -``` +- **Function Workflows**: + - Applied to `Function` objects. + - Perform analysis at the function granularity. + - Pipelines can run concurrently and be executed independently of the Module Workflow. -Then open a file with options and select *PythonLogWarnWorkflow* for the *Function Workflow* setting. +### **Activity** ---- -### C++ +An **Activity** is an individual analysis or action within a Workflow, serving as the fundamental building block of the analysis process. Each Activity is responsible for a specific task and can be configured to define its execution behavior, dependencies, and eligibility criteria. Activities work together within the Workflow to carry out complex analysis tasks, with roles and configurations determining how they interact and execute. The Activity operates on an `AnalysisContext`, providing access to the binary data, analysis state, and transformation utilities. -Several C++ examples are available in [binaryninja-api/examples/workflows](https://github.com/Vector35/binaryninja-api/tree/dev/examples). These examples are distributed with Binary Ninja and available via the *Function Workflow* setting. Steps for building the examples can be found [here](https://github.com/Vector35/binaryninja-api#building). +### Workflow Machine -- Custom Tail Call Analysis -- Call-Site Function Inliner -- Objective C Workflow +The **Workflow Machine** is the engine that manages and executes a Workflow. It controls the execution flow according to the Workflow's blueprint, handling states, commands, and activity eligibility. The Workflow Machine ensures that analyses are performed in the correct order and manages state transitions during the analysis process. We have built an API to interact with the Workflow Machine, allowing you to issue commands, query and modify state, and control activity eligibility. + +In Binary Ninja, the terms **Workflow Machine** and **Pipeline** are used interchangeably to describe the execution of a Workflow on a particular target (such as a `BinaryView` or `Function`). ---- -# Basic Concepts --- -## Activity +## Workflow Fundamentals -An Activity is the basic building block of the Workflows framework and it is also the extensibility mechanism which allows attachment of an arbitrary callback routine. The degree of coupling between activities is discretionary. This means that an activity can be part of a tightly coupled dataflow transform, or an activity can be completely disassociated with other activities in the pipeline. Binary Ninja provides numerous core activities which together provide the program analysis results and IL transformation system. Activities are dynamically parameterized with an analysis context which enables it to produce or consume information in a coordinated fashion. +Configuring and customizing Workflows in Binary Ninja allows you to tailor the analysis process to your needs. This section covers key concepts and operations related to Workflow configuration and management. -## Workflow +### Workflow Selection -A Workflow is a repository of activities along with a unique strategy to execute them. Workflows are also composable, meaning workflows can include other workflows. It's possible to control various aspects of the execution strategy such as course-grained debugging as well as conditional, and parallel execution. Binary Ninja provides a default workflow which contains all of the core module and function analyses. +By default, `Function` and `BinaryView` objects get their assigned workflow from the settings system: -### Workflow Lifecycle +- **Function Workflow Setting**: `analysis.workflows.functionWorkflow` +- **Module Workflow Setting**: `analysis.workflows.moduleWorkflow` -A Workflow starts in the unregistered state from either creating a new empty Workflow, or cloning an existing Workflow. While unregistered it's possible to add and remove activities, as well as change the execution strategy. In order to use the Workflow on a binary it must be registered. Once registered the Workflow is immutable and available for use. +These settings are configurable in the Binary Ninja Settings UI. Additionally, as new workflows are registered, the name and description are automatically added to the UI for easy selection. -## Analysis Context +![workflow-settings](../img/workflow-settings.png "Workflows"){ width="800" } -The Analysis Context provides access to the current analysis hierarchy along with APIs to query and inform new analysis information. +These settings are scoped to the Resource level, meaning they are saved within each BinaryView. This allows each BinaryView to have its own unique workflow settings. Additionally, Binary Ninja supports function instance-level settings, enabling you to customize settings (and workflows) per function instance. In the UI, the *Edit Function Properties* dialog allows you to set the workflow for a specific function instance, as shown below: +![function-properties-workflow](../img/function-properties-workflow.png "Function Properties Workflow"){ width="800" } ---- -# Execution Model ---- +As always, it's possible to interact with these settings programmatically using the Binary Ninja API. For example: -## State Graph -```plantuml -@startuml +- **Modify Workflow for a Function Instance**: + ```python + # Function settings scope + Settings().get_string("analysis.workflows.functionWorkflow", current_function) + 'core.function.metaAnalysis' + Settings().set_string("analysis.workflows.functionWorkflow", 'core.function.defaultAnalysis', current_function) + Settings().get_string("analysis.workflows.functionWorkflow", current_function) + 'core.function.defaultAnalysis' -[*] --> Idle -Idle : Pipeline Not Configured -Idle --> Ready: Run/Configure -Idle --> Suspend: Disable + # BinaryView settings scope + Settings().get_string("analysis.workflows.functionWorkflow", bv) + 'core.function.metaAnalysis' + Settings().get_string("analysis.workflows.moduleWorkflow", bv) + 'core.module.metaAnalysis' + ``` -Suspend : Pipeline Disabled -Suspend --> Idle: Enable +### Foundational Workflows -Ready : Pipeline Configured -Ready : * Abort --> Idle -Ready --> Ready: Configure -Ready --> Idle: Reset/ -Ready --> Active: Run/Step/Next +Binary Ninja provides a set of foundational workflows that serve as the basis for both module-level and function-level analysis. These workflows are designed to meet standard analysis needs while also serving as a starting point for customization. -Active : Pipeline Executing -Active : Break Conditions -Active --> Wait: -Active --> Idle: Finish -Active --> Stall: Transfer +**Immutable Core Workflows** +These workflows are immutable and represent the core analysis functionality in Binary Ninja. They provide a reliable baseline for analysis and cannot be altered directly: -Inactive : Pipeline Execution Paused -Inactive : Debug Commands {Step, Next, Breakpoint} -Inactive : * Abort --> Idle -Inactive --> Active: -Inactive --> Active: Run +- **Default Module Analysis** (`core.module.defaultAnalysis`): Defines the core analysis tasks for module-level analysis on a `BinaryView` object. This workflow ensures consistent and thorough analysis of the entire binary. +- **Default Function Analysis** (`core.function.defaultAnalysis`): Defines the core analysis tasks for function-level analysis. It provides a stable and reliable framework for disassembling and analyzing individual functions. -Wait : Waiting for Control -Wait --> Inactive: +**Mutable Meta Workflows** +These workflows are clones of the default core workflows but are mutable, allowing for advanced customization and extensibility: -Stall : Waiting for Event -Stall : * Abort --> Idle -Stall --> Active: Run +- **Meta Module Analysis** (`core.module.metaAnalysis`): A mutable version of the default module analysis workflow. It can be customized to add or remove analysis stages, modify dependencies, or integrate new analyses. +- **Meta Function Analysis** (`core.function.metaAnalysis`): A mutable version of the default function analysis workflow. It can be customized to extend or modify the analysis process for individual functions. -@enduml -``` +These foundational workflows are designed to be cloned and customized to create new workflows tailored to specific needs. By leveraging these base workflows, you can build upon existing analyses and extend the capabilities of Binary Ninja. - +## Advanced Topics (Coming Soon) + +This section covers advanced topics related to Workflows, Activities, and the Workflow Machine. It explores emerging concepts, best practices, and strategies for optimizing and extending the analysis pipeline. + +--- \ No newline at end of file -- cgit v1.3.1