diff options
Diffstat (limited to 'docs/dev')
| -rw-r--r-- | docs/dev/annotation.md | 16 | ||||
| -rw-r--r-- | docs/dev/batch.md | 8 | ||||
| -rw-r--r-- | docs/dev/bnil-hlil.md | 32 | ||||
| -rw-r--r-- | docs/dev/bnil-llil.md | 16 | ||||
| -rw-r--r-- | docs/dev/bnil-mlil.md | 20 | ||||
| -rw-r--r-- | docs/dev/bnil-modifying.md | 215 | ||||
| -rw-r--r-- | docs/dev/bnil-overview.md | 14 | ||||
| -rw-r--r-- | docs/dev/concepts.md | 48 | ||||
| -rw-r--r-- | docs/dev/flags.md | 48 | ||||
| -rw-r--r-- | docs/dev/index.md | 2 | ||||
| -rw-r--r-- | docs/dev/plugins.md | 60 | ||||
| -rw-r--r-- | docs/dev/themes.md | 4 | ||||
| -rw-r--r-- | docs/dev/typelibraries.md | 4 | ||||
| -rw-r--r-- | docs/dev/uidf.md | 8 | ||||
| -rw-r--r-- | docs/dev/workflows.md | 4 |
15 files changed, 250 insertions, 249 deletions
diff --git a/docs/dev/annotation.md b/docs/dev/annotation.md index e83d1a2e..d350700f 100644 --- a/docs/dev/annotation.md +++ b/docs/dev/annotation.md @@ -308,7 +308,7 @@ Here's a few useful concepts when working Binary Ninja's type system. In Binary Ninja the name of a class/struct/union or enumeration is separate from its type definition. This is much like how it's done in C. The mapping between a structure's definition and its name is kept in the Binary View. -Thus if we want to associate a name with our type we need an extra step. +Thus, if we want to associate a name with our type we need an extra step. ```python bv.define_user_type('Foo', Type.structure(members=[(Type.int(4), 'field_0')])) @@ -356,7 +356,7 @@ struct Bas #### Mutable Types -As `Type` objects are immutable, the Binary Ninja API provides a pure python implementation of types to provide mutability, these all inherit from `MutableType` and keep the same names as their immutable counterparts minus the `Type` part. Thus `Structure` is the mutable version of `StructureType` and `Enumeration` is the mutable version of `MutableType`. `Type` objects can be converted to `MutableType` objects using the `Type.mutable_copy` API and, `MutableType` objects can be converted to `Type` objects through the `MutableType.immutable_copy` API. Generally speaking you shouldn't need the mutable type variants for anything except creation of structures and enumerations, mutable type variants are provided for convenience and consistency. Building and defining a new structure can be done in a few ways. The first way would be the two step process of creating the structure then defining it. +As `Type` objects are immutable, the Binary Ninja API provides a pure python implementation of types to provide mutability, these all inherit from `MutableType` and keep the same names as their immutable counterparts minus the `Type` part. Thus, `Structure` is the mutable version of `StructureType` and `Enumeration` is the mutable version of `MutableType`. `Type` objects can be converted to `MutableType` objects using the `Type.mutable_copy` API and, `MutableType` objects can be converted to `Type` objects through the `MutableType.immutable_copy` API. Generally speaking you shouldn't need the mutable type variants for anything except creation of structures and enumerations, mutable type variants are provided for convenience and consistency. Building and defining a new structure can be done in a few ways. The first way would be the two-step process of creating the structure then defining it. ```python s = StructureBuilder.create(members=[(IntegerType.create(4), 'field_0')]) @@ -373,7 +373,7 @@ s.append(IntegerType.create(4)) bv.define_user_type('Foo', s) ``` -Finally you can use the built-in context manager which automatically registers the created type with the provided `BinaryView` (`bv`) and name(`Foo`). Additionally when creating TypeLibraries a `Type` can be passed instead of a `BinaryView` +Finally, you can use the built-in context manager which automatically registers the created type with the provided `BinaryView` (`bv`) and name(`Foo`). Additionally, when creating TypeLibraries a `Type` can be passed instead of a `BinaryView` ```python with StructureBuilder.builder(bv, 'Foo') as s: @@ -412,7 +412,7 @@ There are 3 categories of object which can have `Type` objects applied to them. * Variables (i.e. local variables) * DataVariables (i.e. global variables) -As of the 3.0 API its much easier to apply types to Variables and DataVariables +As of the 3.0 API it's much easier to apply types to Variables and DataVariables #### Applying a type to a `Function` @@ -463,13 +463,13 @@ Importing a header goes through the same code path as parsing source directly. Y (<types: [...], variables: [...], functions: [...]>, []) ``` -Using these APIs will give you a Python object with all of the results, but won't actually apply them to the analysis in any way. If you want to replicate the importing behavior seen in the UI widget, there are a few additional steps you will need to take: +Using these APIs will give you a Python object with all the results, but won't actually apply them to the analysis in any way. If you want to replicate the importing behavior seen in the UI widget, there are a few additional steps you will need to take: 1. Use `BinaryView.define_user_types()` to add all the created types to the analysis -1. Update the function and data variable types +2. Update the function and data variable types 1. Look up matching functions and symbols by using `BinaryView.get_symbol_by_raw_name`. Also look up potentially modified function names starting with `_` or `__` - 1. Once you have found a function whose name matches one found in the header file, set its type using `Function.type = ftype` - 1. Similarly for Data Variables, once one is found with a matching name, set its type using `DataVariable.type = dvtype` + 2. Once you have found a function whose name matches one found in the header file, set its type using `Function.type = ftype` + 3. Similarly for Data Variables, once one is found with a matching name, set its type using `DataVariable.type = dvtype` #### Exporting a Header diff --git a/docs/dev/batch.md b/docs/dev/batch.md index 1448d34f..ab7e1830 100644 --- a/docs/dev/batch.md +++ b/docs/dev/batch.md @@ -2,7 +2,7 @@ An often asked question of Binary Ninja is "How do I enable batch-processing mode?". The answer is that we don't have one--but the good news is because it doesn't need it! We have an even better solution. BN is simply a library that is trivial to include in your own scripts for batch analysis. As long as you have a Commercial or Ultimate license (or a [headless](https://binary.ninja/purchase/#container:~:text=This%20works%20especially%20well%20with%20our,that%20are%20designed%20for%20headless%2Donly%20installs.) license), it's possible to invoke the core analysis library with all of its APIs without even launching the UI. -This document describes some general tips and tricks for effective batch processing. In particular, because Binary Ninja is multi-threaded, some methods for faster processing like [multiprocessing](https://docs.python.org/3/library/multiprocessing.html) can have dangerous consequences. +This document describes some general tips and tricks for effective batch processing. In particular, because Binary Ninja is multithreaded, some methods for faster processing like [multiprocessing](https://docs.python.org/3/library/multiprocessing.html) can have dangerous consequences. ## Dedicated Python @@ -74,11 +74,11 @@ Opening /bin/ls which has 50 functions ... ``` -Notice that we have far fewer functions in `/bin/ls` this time. By shortcutting the analysis we've prevented further function identification but we've done so much more quickly. +Notice that we have far fewer functions in `/bin/ls` this time. By shortcutting the analysis we've prevented further function identification, but we've done so much more quickly. ### Single Function Analysis -A common workflow is to analyze a single (or small number) of functions in a particular binaries. This can be done using the analysis hold feature: +A common workflow is to analyze a single (or small number) of functions in a particular binary. This can be done using the analysis hold feature: ```python from binaryninja import load @@ -127,7 +127,7 @@ To use this API, copy the contents of your license file into a string and pass i ## Parallelization -Of course, one of the main reasons you might want to build some automation so to spin up a number of threads to process multiple files. Be aware though, that Binary Ninja itself is multithreaded. In fact, if the bottle neck for your analysis script is BN itself, you're almost certainly better off not using any parallelization because the multithreading BN does on its own will provide more benefit than you'd gain by extra parallelization. +Of course, one of the main reasons you might want to build some automation so to spin up a number of threads to process multiple files. Be aware though, that Binary Ninja itself is multithreaded. In fact, if the bottleneck for your analysis script is BN itself, you're almost certainly better off not using any parallelization because the multithreading BN does on its own will provide more benefit than you'd gain by extra parallelization. That said, there are certainly several good use cases where splitting your analysis makes sense. When processing many files for example, you might want to use multiple processes so that a single big slow file doesn't slow down the rest of the analysis as much. Or if you're working with potentially malformed files that may trigger bugs or crashes and you're worried about a single script failing. diff --git a/docs/dev/bnil-hlil.md b/docs/dev/bnil-hlil.md index 4829ad4a..b66081dc 100644 --- a/docs/dev/bnil-hlil.md +++ b/docs/dev/bnil-hlil.md @@ -16,7 +16,7 @@ The High Level Intermediate Language (HLIL) is Binary Ninja's decompiler output. ## Debug Report -To observe the transformartions that occur from MLIL to HLIL, you can use the built-in [`debug report`](https://api.binary.ninja/binaryninja.function-module.html#binaryninja.function.Function.request_debug_report) API: +To observe the transformations that occur from MLIL to HLIL, you can use the built-in [`debug report`](https://api.binary.ninja/binaryninja.function-module.html#binaryninja.function.Function.request_debug_report) API: ```py > current_function.request_debug_report("hlil") @@ -51,22 +51,22 @@ There are a number of properties that can be queried on the [`HighLevelILInstruc * `HLIL_TAILCALL` - This instruction calls the expression `dest` using `params` as input and `output` for return values not exist * `HLIL_SYSCALL` - Make a system/service call with parameters `params` and output `output` -* `HLIL_WHILE` - -* `HLIL_DO_WHILE` - -* `HLIL_FOR` - -* `HLIL_SWITCH` - -* `HLIL_CASE` - -* `HLIL_BREAK` - -* `HLIL_CONTINUE` - +* `HLIL_WHILE` - +* `HLIL_DO_WHILE` - +* `HLIL_FOR` - +* `HLIL_SWITCH` - +* `HLIL_CASE` - +* `HLIL_BREAK` - +* `HLIL_CONTINUE` - ### Variable Reads and Writes * `HLIL_VAR_DECLARE` - A declaration of `var` * `HLIL_VAR_INIT` - Initializes `dest` to the result of an expression `src` * `HLIL_ASSIGN` - Sets a variable `dest` to the result of an expression `src` -* `HLIL_ASSIGN_UNPACK` - +* `HLIL_ASSIGN_UNPACK` - * `HLIL_VAR` - A variable expression `src` -* `HLIL_VAR_PHI` - A `PHI` represents the combination of several prior versions of a variable when differnet basic blocks coalesce into a single destination and it's unknown which path was taken. +* `HLIL_VAR_PHI` - A `PHI` represents the combination of several prior versions of a variable when different basic blocks coalesce into a single destination and it's unknown which path was taken. * `HLIL_MEM_PHI` - A memory `PHI` represents memory modifications that could have occured down different source basic blocks similar to a `VAR_PHI`. * `HLIL_ADDRESS_OF` - The address of variable `src` * `HLIL_CONST` - A constant integral value `constant` @@ -76,11 +76,11 @@ not exist * `HLIL_FLOAT_CONST` - A floating point constant `constant` * `HLIL_IMPORT` - A `constant` integral value representing an imported address * `HLIL_LOW_PART` - `size` bytes from the low end of `src` expression -* `HLIL_STRUCT_FIELD` - -* `HLIL_ARRAY_INDEX` - +* `HLIL_STRUCT_FIELD` - +* `HLIL_ARRAY_INDEX` - * `HLIL_SPLIT` - A split pair of variables `high`:`low` which can be used a single expression * `HLIL_DEREF` - Dereferences `src` -* `HLIL_DEREF_FIELD` - +* `HLIL_DEREF_FIELD` - ### Arithmetic Operations @@ -161,6 +161,6 @@ not exist * `HLIL_UNDEF` - The expression performs undefined behavior * `HLIL_UNIMPL` - The expression is not implemented * `HLIL_UNIMPL_MEM` - The expression is not implemented but does access `src` memory -* `HLIL_BLOCK` - -* `HLIL_LABEL` - -* `HLIL_UNREACHABLE` - +* `HLIL_BLOCK` - +* `HLIL_LABEL` - +* `HLIL_UNREACHABLE` - diff --git a/docs/dev/bnil-llil.md b/docs/dev/bnil-llil.md index cf1cc5f6..e8b77a07 100644 --- a/docs/dev/bnil-llil.md +++ b/docs/dev/bnil-llil.md @@ -1,6 +1,6 @@ # Binary Ninja Intermediate Language: Low Level IL -Make sure to checkout the [BNIL overview](bnil-overview.md) first if you haven't already. Or feel free to skip to [part 2](bnil-mlil.md) which covers MLIL, or [part 3](bnil-hlil.md) which covers HLIL. This developer guide is intended to cover some of the mechanics of the LLIL to distinguish it from the other ILs in the BNIL family. +Make sure to checkout the [BNIL overview](bnil-overview.md) first if you haven't already. Or feel free to skip to [part 2](bnil-mlil.md) which covers MLIL, or [part 3](bnil-hlil.md) which covers HLIL. This developer guide is intended to cover some mechanics of LLIL to distinguish it from the other ILs in the BNIL family. If you've already read the introduction, let's get right into the details of LLIL! @@ -42,7 +42,7 @@ Next we get the [`lowlevelil.LowLevelILFunction`](https://api.binary.ninja/binar Finally, we can print out the attributes of the instruction. We first print out `address` which is the address of the corresponding assembly language instruction. Next, we print the `instr_index`, this you can think of as the address of the IL instruction. Since translating assembly language is a many-to-many relationship, we may see multiple IL instructions needed to represent a single assembly language instruction, and thus each IL instruction needs to have its own index separate from its address. Finally, we print out the instruction text. -In python, iterating over a class is a distinct operation from subscripting. This separation is used in the `LowLevelILFunction` class. If you iterate over a `LowLevelILFunction` you get a list of `LowLevelILBasicBlocks`, however if you subscript a `LowLevelILFunction` you actually get the `LowLevelILInstruction` whose `instr_index` corresponds to the subscript: +In python, iterating over a class is a distinct operation from subscripting. This separation is used in the `LowLevelILFunction` class. If you iterate over a `LowLevelILFunction` you get a list of `LowLevelILBasicBlocks`, however if you subscript a `LowLevelILFunction` you actually get the `LowLevelILInstruction` whose `instr_index` corresponds to the subscript: ``` >>> list(current_function.low_level_il) @@ -55,7 +55,7 @@ In python, iterating over a class is a distinct operation from subscripting. Th ``` ## Low Level IL Instructions -Now that we've established how to access LLIL Functions, Blocks, and Instructions, let's focus in on the instructions themselves. LLIL instructions are infinite length and structured as an expression tree. An expression tree means that instruction operands can be composed of operation. Thus we can have an IL instruction like this: +Now that we've established how to access LLIL Functions, Blocks, and Instructions, let's focus in on the instructions themselves. LLIL instructions are infinite length and structured as an expression tree. An expression tree means that instruction operands can be composed of operation. Thus, we can have an IL instruction like this: ``` eax = eax + ecx * 4 @@ -159,7 +159,7 @@ Going into gross detail on all the instructions is out of scope of this article, ### Registers, Constants & Flags -When parsing an instruction tree the terminals are registers, constants and flags. This provide the basis from which all instructions are built. +When parsing an instruction tree the terminals are registers, constants and flags. This provides the basis from which all instructions are built. * `LLIL_REG` - A register, terminal * `LLIL_CONST` - A constant integer value, terminal @@ -173,8 +173,8 @@ Reading and writing memory is accomplished through the following instructions. * `LLIL_LOAD` - Load a value from memory. * `LLIL_STORE` - Store a value to memory. -* `LLIL_PUSH` - Store value to stack; adjusting stack pointer by sizeof(value) after the store. -* `LLIL_POP` - Load value from stack; adjusting stack pointer by sizeof(value) after the store. +* `LLIL_PUSH` - Store value to stack; adjusting stack pointer by `sizeof(value)` after the store. +* `LLIL_POP` - Load value from stack; adjusting stack pointer by `sizeof(value)` after the store. ### Control Flow & Conditionals @@ -206,7 +206,7 @@ As you can see from the above code, labels are really just used internally and a * `LLIL_SYSCALL` - System call instruction * `LLIL_TAILCALL ` - This instruction calls the expression `dest` using `params` as input and `output` for return values * `LLIL_IF` - `If` provides conditional execution. If condition is true execution branches to the true label and false label otherwise. -* `LLIL_GOTO` - `Goto` is used to branch to an IL label, this is different than jump since jump can only jump to addresses. +* `LLIL_GOTO` - `Goto` is used to branch to an IL label, this is different from jump since jump can only jump to addresses. * `LLIL_FLAG_COND` - Returns the flag condition expression for the specified flag condition. * `LLIL_CMP_E` - equality * `LLIL_CMP_NE` - not equal @@ -275,7 +275,7 @@ The double precision instruction multiply, divide, modulus instructions are part ### Floating Point Conditionals -These are identical to their native counterparts but are lifted separately so that the operations can impact different flags. See "Control FLow & Conditionals" above. +These are identical to their native counterparts but are lifted separately so that the operations can impact different flags. See [Control Flow & Conditionals](#control-flow--conditionals) above. * `LLIL_FCMP_E ` - See above * `LLIL_FCMP_NE ` - See above diff --git a/docs/dev/bnil-mlil.md b/docs/dev/bnil-mlil.md index bdade82c..57cccb00 100644 --- a/docs/dev/bnil-mlil.md +++ b/docs/dev/bnil-mlil.md @@ -34,7 +34,7 @@ First, it's important to understand what we mean when we talk about a MLIL varia <class 'binaryninja.function.Variable'> ``` -Variables in MLIL have a very specific meaning, that is not completely obvious at first. They represent a single storage location within the scope of a single function. To those not well versed in program analysis, a storage location is where a value is located at a given point in time. In the process of compilation a compiler conducts a step called _Register Allocation_; this is the process of figuring out how to map the potentially infinite number of variables specified in the original source code to a finite set of registers. When there are more variables and intermediate values than registers available, the compiler _spills_ them on to the stack. Thus a single high-level-language variable can be mapped across a number of storage locations. A variable can simultaneously be in multiple registers and on the stack at the same time. However, unlike high-level-language variables, MLIL variables represent one and only one storage location. Binary Ninja's High Level IL (HLIL) will be responsible for storing this mapping. +Variables in MLIL have a very specific meaning, that is not completely obvious at first. They represent a single storage location within the scope of a single function. To those not well versed in program analysis, a storage location is where a value is located at a given point in time. In the process of compilation a compiler conducts a step called _Register Allocation_; this is the process of figuring out how to map the potentially infinite number of variables specified in the original source code to a finite set of registers. When there are more variables and intermediate values than registers available, the compiler _spills_ them on to the stack. Thus, a single high-level-language variable can be mapped across a number of storage locations. A variable can simultaneously be in multiple registers and on the stack at the same time. However, unlike high-level-language variables, MLIL variables represent one and only one storage location. Binary Ninja's High Level IL (HLIL) will be responsible for storing this mapping. So let's look at the properties available on a [`Variable`](https://api.binary.ninja/binaryninja.variable-module.html#binaryninja.variable.Variable) object. @@ -76,7 +76,7 @@ The `storage` property changes meaning depending on the [`VariableSourceType`](h '-0x260' ``` -Given the above information it might now be intuitive how variable names are constructed. First we determine the `source_type` of the variable. If it's a `RegisterVariableSourceType` we just use the register's name directly. If it’s a `StackVariableSourceType` then we use `var_` + `hex(-storage)`. Finally, we append a count each time that that storage location is reused. +Given the above information it might now be intuitive how variable names are constructed. First we determine the `source_type` of the variable. If it's a `RegisterVariableSourceType` we just use the register's name directly. If it’s a `StackVariableSourceType` then we use `var_` + `hex(-storage)`. Finally, we append a count each time that that storage location is reused. ### `index` The `index` is an identifier chosen to be unique across different analysis passes. @@ -128,7 +128,7 @@ A boolean type is an integer which has a value of False (0) or True (!0). ### IntegerTypeClass -An integer type has a sign, a width (in bytes), and a display type. The display type determines how the integer should be displayed; the options are self explanatory: +An integer type has a sign, a width (in bytes), and a display type. The display type determines how the integer should be displayed; the options are self-explanatory: ``` enum IntegerDisplayType @@ -150,7 +150,7 @@ enum IntegerDisplayType ### FloatTypeClass -The float type is a IEEE 754 variable precision type, and can represent floating point numbers up to 10 bytes in width. All floating point numbers are assumed to be signed. +The float type is an IEEE 754 variable precision type, and can represent floating point numbers up to 10 bytes in width. All floating point numbers are assumed to be signed. ### WideCharTypeClass @@ -163,7 +163,7 @@ A varargs type is used to indicate that a function is variadic and thus represen ### ValueTypeClass -A value type is simply a constant value. It is used mainly in demangling for types which only have a have a name or value. +A value type is simply a constant value. It is used mainly in demangling for types which only have a name or value. ### FunctionTypeClass @@ -189,7 +189,7 @@ Array types function similarly to pointer types however the array type knows how * `target`/`element_type` - the type of element this array is constructed of * `count` - the count of array elements -* `width` - the size of the array (count * target.width) +* `width` - the size of the array (`count * target.width`) ### EnumerationTypeClass @@ -243,7 +243,7 @@ From the code in [`mediumlevelil.py`](https://github.com/Vector35/binaryninja-ap MediumLevelILOperation.MLIL_CALL: [("output", "var_list"), ("dest", "expr"), ("params", "expr_list")], ``` -Thus we can query the call's `output` which is a list of variables: +Thus, we can query the call's `output` which is a list of variables: ``` >>> inst.output @@ -288,7 +288,7 @@ The parameter list can be accessed through the `params` property: * `MLIL_IF` - Branch to the `true`/`false` MLIL instruction identifier depending on the result of the `condition` expression * `MLIL_GOTO` - Branch to the `dest` expression id * `MLIL_TAILCALL` - This instruction calls the expression `dest` using `params` as input and `output` for return values -* `MLIL_TAILCALL_UNTYPED ` - A tailcall where the stack stack resolution could not be determined and thus a list of parameters and return values do not exist +* `MLIL_TAILCALL_UNTYPED ` - A tailcall where the stack resolution could not be determined and thus a list of parameters and return values do not exist * `MLIL_SYSCALL` - Make a system/service call with parameters `params` and output `output` * `MLIL_SYSCALL_UNTYPED` - Makes a system/service call, but an exact set of parameters couldn't be determined. @@ -308,8 +308,8 @@ The parameter list can be accessed through the `params` property: * `MLIL_VAR_ALIASED_FIELD` - * `MLIL_VAR_FIELD` - A variable and offset expression `src`, `offset` * `MLIL_VAR_SPLIT` - A split pair of variables `high`:`low` which can be used a single expression -* `MLIL_VAR_PHI` - A `PHI` represents the combination of several prior versions of a variable when differnet basic blocks coalesce into a single destination and it's unknown which path was taken. -* `MLIL_MEM_PHI` - A memory `PHI` represents memory modifications that could have occured down different source basic blocks similar to a `VAR_PHI`. +* `MLIL_VAR_PHI` - A `PHI` represents the combination of several prior versions of a variable when different basic blocks coalesce into a single destination and it's unknown which path was taken. +* `MLIL_MEM_PHI` - A memory `PHI` represents memory modifications that could have occurred down different source basic blocks similar to a `VAR_PHI`. * `MLIL_ADDRESS_OF` - The address of variable `src` * `MLIL_ADDRESS_OF_FIELD` - The address and `offset` of the variable `src` * `MLIL_CONST` - A constant integral value `constant` diff --git a/docs/dev/bnil-modifying.md b/docs/dev/bnil-modifying.md index ac521d4f..bcd02b5b 100644 --- a/docs/dev/bnil-modifying.md +++ b/docs/dev/bnil-modifying.md @@ -19,7 +19,7 @@ Key: | Medium Level IL | ✅ | ✅ | ❌ | | High Level IL | ⚠️* | ❌ | ❌ | -\* Modifying High Level IL in C++, while some APIs exist, is incomplete and [certain more complicated operations are not possible yet](#notes-on-hlil). +\* Modifying High Level IL in C++, while some APIs exist, is incomplete and [certain more complicated operations are not possible yet](#notes-on-hlil). ## Choosing A Level of IL To Modify @@ -37,7 +37,7 @@ Notably, any `PossibleValueSet` calculations require MLIL or higher, so only `Re There are certain ILs which you cannot modify, for a variety of reasons: -- **SSA Forms** are generated by the core from the various Non-SSA Form ILs at each stage of analysis. You should not modify them, as they are considered analysis derived from the Non-SSA Form. They will likely be regenerated from the source Non-SSA Form without consulting any changes you might make to them. +- **SSA Forms** are generated by the core from the various Non-SSA Form ILs at each stage of analysis. You should not modify them, as they are considered analysis derived from the Non-SSA Form. They will likely be regenerated from the source Non-SSA Form without consulting any changes you might make to them. - **Disassembly** should be modified by changing the Architecture plugin if it is open source, or by using an Architecture Extension if it is not. A Workflow Activity cannot affect Disassembly. - **Mapped MLIL** is lifted directly from LLIL SSA Form and not user-editable. Generally speaking, users and developers of Workflow Actions should not need to consult Mapped MLIL. - **Pseudo-C** and other Language Representation forms are not a distinct level of IL, but are simply alternate renderings of HLIL. @@ -48,7 +48,7 @@ Based on your IL of choice, there are a bunch of places you can insert your modi - **Lifted IL** modification needs to happen before the `core.function.analyzeAndExpandFlags` stage, which translates Lifted IL into Low Level IL. After that stage, Lifted IL is never again consulted. - Modifying flag definitions and usages is likely impossible, as the Semantic Flags system will consult the Architecture to determine what IL to generate when a flag is used, not your Workflow Action. There is currently no way to change this behavior. -- **Low Level IL** modification is generally done right before the `core.function.generateMediumLevelIL` stage, after the final Low Level IL function is emitted. After Medium Level IL is generated, changes to the Low Level IL will have limited effects on the other ILs (because most things refer to MLIL). +- **Low Level IL** modification is generally done right before the `core.function.generateMediumLevelIL` stage, after the final Low Level IL function is emitted. After Medium Level IL is generated, changes to the Low Level IL will have limited effects on the other ILs (because most things refer to MLIL). - If you want to make changes to LLIL that can affect the stack pointer resolution, you will want to insert your action before `core.function.analyzeStackAdjustment` instead. - **Medium Level IL** modification is generally done right after `core.function.generateMediumLevelIL` since (as of writing) the MLIL translation is all done in that one monolithic step and future steps are simply processing and annotations. This may change some time in the future. - **High Level IL** modification is generally done right after `core.function.generateHighLevelIL` since HLIL generation is also monolithic. This also may change eventually. @@ -63,12 +63,12 @@ Support for this is planned soon, but not implemented as of Binary Ninja 5.1. - An **IL Function** is a collection of **IL Instructions**, grouped into **IL Basic Blocks** - An **IL Basic Block** is a contiguous sequence of **IL Instructions** between a start and end **Instruction Index**, which only has incoming branches at the start and outgoing branches at the end. - An **IL Expression** is a single-operation expression, which may have child expressions or be the child of another expression. - - The `*LevelILInstruction` class represents a single **IL Expression**. The class name may be confusing since **IL Instructions** are a different concept. + - The `*LevelILInstruction` class represents a single **IL Expression**. The class name may be confusing since **IL Instructions** are a different concept. - An **Expression Index** points to an **IL Expression** and may be used as an operand of another **IL Expression**. - An **IL Instruction** is an **IL Expression** that has been added as a top-level instruction to the function with a call to `*LevelILFunction.append()` (C++: `*LevelILFunction::AddInstruction`) - An **Instruction Index** is assigned to an **IL Expression** that is added as an **IL Instruction**. Child expressions of an **IL Instruction** do not inherently have an **Instruction Index**, though the API tries to provide this for you. - A **Mutate Transformation** is a simple modification of **IL Instructions** in an **IL Function** as per [Replacing Trivial Instructions (Mutate Transformation)](#replacing-trivial-instructions-mutate-transformation) below. -- A **Copy Transformation** is a more in-depth modification of an **IL Function** which involves copying its **IL Instructions** into a newly constructed **IL Function** with modifications made along the way, as per [Adding Instructions and Replacing Multiple Instructions (Copy Transformation)](#adding-instructions-and-replacing-multiple-instructions-copy-transformation) below. +- A **Copy Transformation** is a more in-depth modification of an **IL Function** which involves copying its **IL Instructions** into a newly constructed **IL Function** with modifications made along the way, as per [Adding Instructions and Replacing Multiple Instructions (Copy Transformation)](#adding-instructions-and-replacing-multiple-instructions-copy-transformation) below. ## Writing a Transformation @@ -79,24 +79,24 @@ Once you figure out which level of IL you want to modify, and where in the Workf In order for your modifications to run, you need to register a Workflow Activity. There are a couple of steps involved in this process: -First, you need to make a clone of the default workflow, making it mutable and allowing you to insert your Activity. +First, you need to make a clone of the default workflow, making it mutable and allowing you to insert your Activity. You can either name the clone the same thing as the metaAnalysis workflow to make your Activity available by default, or clone to a different name, which will require users to select that workflow in Open with Options. Also note that, as of writing, Objective-C support registers its own workflow as `core.function.objectiveC`. If you want your Workflow Activity to apply to Objective-C files, you will need to do this whole process twice, once for `core.function.metaAnalysis` and once for `core.function.objectiveC`. -The same applies if any of your other plugins register a custom workflow that is not `core.function.metaAnalysis`. +The same applies if any of your other plugins register a custom workflow that is not `core.function.metaAnalysis`. === "Python" ```py - # This Workflow will replace metaAnalysis and be used by default + # This Workflow will replace metaAnalysis and be used by default wf = Workflow("core.function.metaAnalysis").clone("core.function.metaAnalysis") - + # Users will need to pick this Workflow in Open with Options wf = Workflow("core.function.metaAnalysis").clone("MyCustomWorkflow") - + # As of 5.1: To modify functions in binaries using the Objective-C workflow, - # you need to do the entire rest of this section twice: + # you need to do the entire rest of this section twice: # once as above, and once for the Objective-C workflow as shown here. wf = Workflow("core.function.objectiveC").clone("core.function.objectiveC") ``` @@ -105,10 +105,10 @@ The same applies if any of your other plugins register a custom workflow that is ```c++ // This Workflow will replace metaAnalysis and be used by default auto wf = Workflow::Instance("core.function.metaAnalysis")->Clone("core.function.metaAnalysis"); - + // Users will need to pick this Workflow in Open with Options auto wf = Workflow::Instance("core.function.metaAnalysis")->Clone("MyCustomWorkflow"); - + // As of 5.1: To modify functions in binaries using the Objective-C workflow, // you need to do the entire rest of this section twice: // once as above, and once for the Objective-C workflow as shown here. @@ -122,7 +122,7 @@ Then, define a new Activity on the Workflow, which allows you to run your code t ```py def rewrite_action(context: AnalysisContext): # Actual modification code goes here (documented below)... - + # Define the custom activity configuration wf.register_activity(Activity( configuration=json.dumps({ @@ -137,7 +137,7 @@ Then, define a new Activity on the Workflow, which allows you to run your code t }), action=rewrite_action )) - + # Add the new action to the workflow, with position as described above wf.insert_after("core.function.generateMediumLevelIL", [ "extension.my_extension.do_the_thing" @@ -152,11 +152,11 @@ Then, define a new Activity on the Workflow, which allows you to run your code t { // Actual modification code goes here (documented below)... } - + BINARYNINJAPLUGIN bool CorePluginInit() { // Rest of your plugin init here ... - + // Define the custom activity configuration wf->RegisterActivity(new Activity(R"~( { @@ -195,7 +195,7 @@ If you wish to replace more than one instruction, or replace an instruction with ### Adding Instructions and Replacing Multiple Instructions (Copy Transformation) If you want to insert new instructions into a function, or want to replace an instruction with than one instruction, you will need to construct a new **IL Function** based on the original **IL Function**, and make your changes while copying the instructions. -This is a rather cumbersome process and it can be easy to make mistakes. +This is a rather cumbersome process, and it can be easy to make mistakes. First, you need to construct a new **IL Function** based on the existing function. Let's call the new function `new_func` and the existing function `old_func`. @@ -206,16 +206,16 @@ Let's call the new function `new_func` and the existing function `old_func`. # ... workflow boilerplate def rewrite_action(context: AnalysisContext): old_func = context.mlil - + # Create a new IL Function based on the old one. # Make sure you use the LLIL from the analysis context, # and *do not* go through Function.llil (it will not have been updated yet) new_func = MediumLevelILFunction(old_func.arch, low_level_il=context.llil) - - # Tell the new function that we are copying from an existing function + + # Tell the new function that we are copying from an existing function # (this transfers various metadata like some block labels) new_func.prepare_to_copy_function(old_func) - + # continues ... ``` @@ -231,14 +231,14 @@ Let's call the new function `new_func` and the existing function `old_func`. // and *do not* go through Function::GetLowLevelIL (it will not have been updated yet) // Also (C++ specific), make sure you use Ref<> to hold - // a strong reference so the new function doesn't get deleted + // a strong reference so the new function doesn't get deleted Ref<MediumLevelILFunction> newFunc = new MediumLevelILFunction( oldFunc->GetArchitecture(), oldFunc->GetFunction(), context->GetLowLevelILFunction() ); - - // Tell the new function that we are copying from an existing function + + // Tell the new function that we are copying from an existing function // (this transfers various metadata like some block labels) newFunc->PrepareToCopyFunction(oldFunc); @@ -249,9 +249,9 @@ Let's call the new function `new_func` and the existing function `old_func`. ???+ Note "Note" You may notice that the sample code above mentions always using `AnalysisContext.llil` (C++: `AnalysisContext::GetLowLevelIL()`) to access the relevant Low Level IL function, and you may be wondering why the sample doesn't instead use `AnalysisContext.function.llil` (C++: `AnalysisContext::GetFunction()->GetLowLevelILFunction()`). - This is because you must **always access IL Functions directly through `AnalysisContext`** when writing Workflow Activities, as that is where the current, up-to-date analysis information is stored. + This is because you must **always access IL Functions directly through `AnalysisContext`** when writing Workflow Activities, as that is where the current, up-to-date analysis information is stored. During analysis, the `LowLevelILFunction` object stored in the `AnalysisContext` object contains the in-progress analysis information from the current run of the Workflow, - whereas the `LowLevelILFunction` object stored on the Function object is from the previous run of the Workflow and contains stale analysis data. + whereas the `LowLevelILFunction` object stored on the Function object is from the previous run of the Workflow and contains stale analysis data. Then, going block-by-block, copy the instructions from `old_func` to `new_func`. When you reach a part where you want to insert new instructions (or replace instructions), do that instead of simply copying the old instructions. @@ -260,22 +260,22 @@ When you reach a part where you want to insert new instructions (or replace inst ```py # previous code... - + # Copy each block in the old function to the new function - for old_block in old_func.basic_blocks: + for old_block in old_func.basic_blocks: # Copy some block labels and tell new IL instructions they came from this block new_func.prepare_to_copy_block(old_block) - + # Copy each instruction from the old block to the new function # Blocks "contain" instructions by knowing their start and end instruction indices # Any instructions in that range as in the block for old_instr_index in range(old_block.start, old_block.end): # Retrieve instruction from old function old_instr: MediumLevelILInstruction = old_func[old_instr_index] - + # Make sure the new instruction we insert has the correct location information new_func.set_current_address(old_instr.address, old_block.arch) - + # continues... ``` @@ -283,13 +283,13 @@ When you reach a part where you want to insert new instructions (or replace inst ```c++ // ... previous code - + // Copy each block in the old function to the new function for (auto& oldBlock: oldFunc->GetBasicBlocks()) { // Copy some block labels and tell new IL instructions they came from this block newFunc->PrepareToCopyBlock(oldBlock); - + // Copy each instruction from the old block to the new function // Blocks "contain" instructions by knowing their start and end instruction indices // Any instructions in that range as in the block @@ -297,39 +297,39 @@ When you reach a part where you want to insert new instructions (or replace inst { // Retrieve instruction from old function auto oldInstr = oldFunc->GetInstruction(oldInstrIndex); - + // Make sure the new instruction we insert has the correct location information newFunc->SetCurrentAddress(oldBlock->GetArchitecture(), oldInstr.address); - + // continues... ``` For each instruction, determine if you want to insert new instructions before it, or replace it with something else. You _probably_ want to do this step ahead of time and apply all the changes at once, but if your changes are simple enough you can get away with doing it inside the loop. === "Python" - + ```py # previous code... - + if want_changes: # Create new instructions in the function for whatever changes you desire - + # All ILs: Be sure to pass the location of the previous instruction # when creating a new expression, so mappings work new_expr = new_func.nop(ILSourceLocation.from_instruction(old_instr)) - + # MLIL and higher: pass the location of the previous instruction # to append(), so cross-IL mappings are preserved new_func.append(new_expr, ILSourceLocation.from_instruction(old_instr)) else: # Copy the instruction as-is - + # Copy the instruction from the old function to the new function # using `copy_to` (deep copy) # MLIL and higher: also pass the location of the previous instruction # to append() so cross-IL mappings are preserved new_func.append(old_instr.copy_to(new_func), ILSourceLocation.from_instruction(old_instr)) - + # continues ... ``` @@ -337,15 +337,15 @@ For each instruction, determine if you want to insert new instructions before it ```C++ // previous code... - + if (wantChanges) { // Create new instructions in the function for whatever changes you desire - + // All ILs: Be sure to pass the location of the previous instruction // when creating a new expression, so mappings work auto newExpr = newFunc->Nop(oldInstr); - + // MLIL and higher: pass the location of the previous instruction // to AddInstruction(), so cross-IL mappings are preserved newFunc->AddInstruction(newExpr, oldInstr); @@ -353,7 +353,7 @@ For each instruction, determine if you want to insert new instructions before it else { // Copy the instruction as-is - + // Copy the instruction from the old function to the new function // using `CopyTo()` (deep copy) // MLIL and higher: also pass the location of the previous instruction @@ -368,16 +368,16 @@ For each instruction, determine if you want to insert new instructions before it And finally, regenerate dataflow: === "Python" - + ```py # previous code ... - + # Create IL basic blocks new_func.finalize() - + # Do dataflow, etc new_func.generate_ssa_form() - + # Assign to the context to apply modifications context.mlil = new_func ``` @@ -386,13 +386,13 @@ And finally, regenerate dataflow: ```C++ // previous code ... - + // Create IL basic blocks newFunc->Finalize(); - + // Do dataflow, etc newFunc->GenerateSSAForm(); - + // Assign to the context to apply modifications context->SetMediumLevelILFunction(newFunc); } @@ -408,15 +408,15 @@ Due to limitations currently in the Python bindings, IL mappings will only updat ```py # previous code ... - + # Need to commit this for mappings to work (as of 5.1 at least) context.mlil = new_func - + # Now we want to modify the function again... - + # The newly created function is now our "old function" old_func = new_func - + # All of the rest of this is the same as before new_func = MediumLevelILFunction(old_func.arch, low_level_il=context.llil) new_func.prepare_to_copy_function(old_func) @@ -426,15 +426,15 @@ Due to limitations currently in the Python bindings, IL mappings will only updat ```C++ // previous code ... - + // Need to commit this for mappings to work (as of 5.1 at least) context->SetMediumLevelILFunction(new_func); - + // Now we want to modify the function again... - + // The newly created function is now our "old function" oldFunc = newFunc; - + // All of the rest of this is the same as before Ref<MediumLevelILFunction> newFunc = new MediumLevelILFunction( oldFunc->GetArchitecture(), @@ -471,11 +471,11 @@ Simply call `*LevelILFunction.get_label_for_source_instruction` (C++: `*LevelILF new_func = MediumLevelILFunction(old_func.arch, low_level_il=context.llil) # Calling prepare_to_copy_function creates labels in new_func - # for all blocks in old_func, accessible via get_label_for_source_instruction + # for all blocks in old_func, accessible via get_label_for_source_instruction new_func.prepare_to_copy_function(old_func) for old_block in old_func.basic_blocks: - # Calling prepare_to_copy_block populates the labels created by + # Calling prepare_to_copy_block populates the labels created by # prepare_to_copy_function above. new_func.prepare_to_copy_block(old_block) @@ -494,10 +494,10 @@ Simply call `*LevelILFunction.get_label_for_source_instruction` (C++: `*LevelILF # Emit MLIL_GOTO with the label for our target instruction new_func.append(new_func.goto(label, ILSourceLocation.from_instruction(old_instr)), ILSourceLocation.from_instruction(old_instr)) continue - + # Otherwise, copy the instruction as-is new_func.append(old_instr.copy_to(new_func), ILSourceLocation.from_instruction(old_instr)) - + # Rest of function... new_func.finalize() new_func.generate_ssa_form() @@ -518,12 +518,12 @@ Simply call `*LevelILFunction.get_label_for_source_instruction` (C++: `*LevelILF ); // Calling PrepareToCopyFunction creates labels in newFunc - // for all blocks in oldFunc, accessible via GetLabelForSourceInstruction + // for all blocks in oldFunc, accessible via GetLabelForSourceInstruction newFunc->PrepareToCopyFunction(oldFunc); - + for (auto& oldBlock: oldFunc->GetBasicBlocks()) { - // Calling PrepareToCopyBlock populates the labels created by + // Calling PrepareToCopyBlock populates the labels created by // PrepareToCopyFunction above. newFunc->PrepareToCopyBlock(oldBlock); @@ -540,21 +540,22 @@ Simply call `*LevelILFunction.get_label_for_source_instruction` (C++: `*LevelILF // If the target is something else, see the section below. auto oldTarget = indexOfSomeInstructionAtStartOfBlockInOldFunc; BNMediumLevelILLabel* label = newFunc->GetLabelForSourceInstruction(oldTarget); - + // Emit MLIL_GOTO with the label for our target instruction newFunc->AddInstruction(newFunc->Goto(*label, oldInstr), oldInstr); continue; } - + // Otherwise, copy the instruction as-is newFunc->AddInstruction(oldInstr.CopyTo(newFunc), oldInstr); } } - + // Rest of function... newFunc->Finalize(); newFunc->GenerateSSAForm(); // ... + } ``` #### Jumping to New Blocks @@ -574,7 +575,7 @@ If you want to have a `*LIL_GOTO` instruction target the middle of an existing * 4. Mark the label for your target 5. Now emit your target instruction and the rest of the block -As mentioned previously, make sure to re-use the same `*LevelILLabel` object for your target in the call to `*LevelILFunction.goto`, `*LevelILFunction.mark_label`, and any other `*LIL_GOTO` instructions pointing at your target. +As mentioned previously, make sure to re-use the same `*LevelILLabel` object for your target in the call to `*LevelILFunction.goto`, `*LevelILFunction.mark_label`, and any other `*LIL_GOTO` instructions pointing at your target. ## Useful Tools @@ -642,9 +643,9 @@ Compare this to `current_il_instruction`, which only gives you the top-level ins ### IL Sidebar Included in the [BN Inspectors plugin](https://github.com/CouleeApps/bn_inspectors) is a sidebar that lists all IL Expressions and IL Instructions. -It shows you all the expressions' and instructions' indices in a big table, making it easier to determine where things are located. +It shows you all the expressions' and instructions' indices in a big table, making it easier to determine where things are located. It can be quite useful for determining the raw operand values of various expressions and finding dangling expressions that get lost after particularly messy calls to `replace_expr`. -It also shows what types the expressions have (MLIL and higher). +It also shows what types the expressions have (MLIL and higher).  @@ -661,15 +662,15 @@ current_function.request_debug_report("my_report") In your Activity, you can check for one of these requests and then, if detected, generate a Debug Report and present it to the user: === "Python" - + ```py def rewrite_action(context: AnalysisContext): report = None if context.function.check_for_debug_report("my_report"): report = ReportCollection() - + # ... do changes and build report - + if report is not None: show_report_collection("My Debug Report", report) ``` @@ -684,9 +685,9 @@ In your Activity, you can check for one of these requests and then, if detected, { report = new ReportCollection(); } - + // ... do changes and build report - + if (report) { ShowReportCollection("My Debug Report", report); @@ -697,13 +698,13 @@ In your Activity, you can check for one of these requests and then, if detected, It can be exceedingly helpful to put your entire modification script inside a `try` block and present the report even on failure: === "Python" - + ```py def rewrite_action(context: AnalysisContext): report = None if context.function.check_for_debug_report("my_report"): report = ReportCollection() - + try: # ... do changes and build report # and if this throws an exception, the report will be presented anyway, @@ -715,7 +716,7 @@ It can be exceedingly helpful to put your entire modification script inside a `t ``` === "C++" - + ```C++ void RewriteAction(Ref<AnalysisContext> context) { @@ -724,7 +725,7 @@ It can be exceedingly helpful to put your entire modification script inside a `t { report = new ReportCollection(); } - + try { // ... do changes and build report @@ -744,7 +745,7 @@ It can be exceedingly helpful to put your entire modification script inside a `t } // Rethrowing the exception here might abort the program (if it is uncaught) - // So you probably want to just eat the exception and print an angry message + // So you probably want to just eat the exception and print an angry message LogErrorWithStackTraceF("Got an exception"); // 5.2 and later // throw; } @@ -753,7 +754,7 @@ It can be exceedingly helpful to put your entire modification script inside a `t Then, you can build up the `ReportCollection` object by adding reports to it with the various Report APIs. -**Note:** If you are making function graphs using the various `create_graph` APIs, make sure to use `create_graph_immediate` as it will construct the graph from the function as it existed when called, instead of lazily once the view is rendered (otherwise you will only see the end state of the function). +**Note:** If you are making function graphs using the various `create_graph` APIs, make sure to use `create_graph_immediate` as it will construct the graph from the function as it existed when called, instead of lazily once the view is rendered (otherwise you will only see the end state of the function). === "Python" @@ -761,10 +762,10 @@ Then, you can build up the `ReportCollection` object by adding reports to it wit # For ReportCollection, create graph with the settings I want settings = DisassemblySettings() settings.set_option(DisassemblyOption.ShowAddress, True) - + # Create graph *immediate* from the function graph = old_func.create_graph_immediate(settings=settings) - + # And add it to my debug report report.append(FlowGraphReport("Initial", graph, context.view)) ``` @@ -778,7 +779,7 @@ Then, you can build up the `ReportCollection` object by adding reports to it wit // Create graph *immediate* from the function auto graph = oldFunc->CreateFunctionGraphImmediate(settings); - + // And add it to my debug report report->AddGraphReport(context->GetBinaryView(), "Initial", graph); ``` @@ -786,20 +787,20 @@ Then, you can build up the `ReportCollection` object by adding reports to it wit You can also use the various graph APIs to style the report and highlight nodes or instructions: === "Python" - + ```py graph = old_func.create_graph_immediate(settings=settings) - + # Go through the nodes in the graph and highlight some of them for node in graph.nodes: if some_condition: node.highlight = HighlightStandardColor.RedHighlightColor if other_condition: node.highlight = HighlightStandardColor.GreenHighlightColor - + # Update the graph with the highlighted node graph.replace(node_index, node) - + # And then you can add as normal report.append(FlowGraphReport("Graph with Highlights", graph, context.view)) ``` @@ -824,7 +825,7 @@ You can also use the various graph APIs to style the report and highlight nodes } auto graph = oldFunc->CreateFunctionGraphImmediate(settings); - + // Go through the nodes in the graph and highlight some of them auto nodes = graph->GetNodes(); for (size_t i = 0; i < nodes.size(); i++) @@ -841,7 +842,7 @@ You can also use the various graph APIs to style the report and highlight nodes // Update the graph with the highlighted node graph->ReplaceNode(i, node); } - + // And then you can add as normal report->AddGraphReport(context->GetBinaryView(), "Initial", graph); ``` @@ -852,31 +853,31 @@ You can also use the various graph APIs to style the report and highlight nodes Due to the nature of Workflows, a custom Debug Report is only able to be triggered if your Activity gets run. But what if you don't want to apply your changes yet? This might be because you suspect your changes are buggy, or so you can introspect the unchanged function while still seeing diagnostics from your changes. -Either way, sometimes it is helpful to have a variant of your Activity that does all of the logic _except_ for applying the updated analysis. +Either way, sometimes it is helpful to have a variant of your Activity that does all the logic _except_ for applying the updated analysis. There's no built-in solution for this, but you can model it pretty effectively by having two activities and a Dry Run switch. First, modify your Activity function with another parameter for if this is a dry run: === "Python" - + ```py def rewrite_action(context: AnalysisContext, dry_run: bool): # ... logic here - new_func = ... - + new_func = ... + # Only update analysis if this is NOT a dry run if not dry_run: context.mlil = new_func ``` === "C++" - + ```c++ void RewriteAction(Ref<AnalysisContext> context, bool dryRun) { // ... logic here - auto newFunc = ...; - + auto newFunc = ...; + // Only update analysis if this is NOT a dry run if (!dryRun) { @@ -888,11 +889,11 @@ First, modify your Activity function with another parameter for if this is a dry Then, when registering your Workflow Activity, make two copies of the Activity, having one be for the dry run: === "Python" - + ```py # Workflow setup boilerplate wf = Workflow("core.function.metaAnalysis").clone("core.function.metaAnalysis") - + # Define your activity twice, once as a "dry run" enabled by default wf.register_activity(Activity( configuration=json.dumps({ @@ -907,7 +908,7 @@ Then, when registering your Workflow Activity, make two copies of the Activity, }), action=lambda context: rewrite_action(context, True) # Dry run )) - + # ... and one that will actually apply changes, disabled by default wf.register_activity(Activity( configuration=json.dumps({ @@ -930,7 +931,7 @@ Then, when registering your Workflow Activity, make two copies of the Activity, ```c++ // Workflow setup boilerplate auto wf = Workflow::Instance("core.function.metaAnalysis")->Clone("core.function.metaAnalysis"); - + // Define your activity twice, once as a "dry run" enabled by default wf->RegisterActivity(new Activity(R"~( { @@ -946,7 +947,7 @@ Then, when registering your Workflow Activity, make two copies of the Activity, )~", [](Ref<AnalysisContext> context) { RewriteAction(context, true); // Dry run })); - + // ... and one that will actually apply changes, disabled by default wf->RegisterActivity(new Activity(R"~( { @@ -970,7 +971,7 @@ Finally, insert both actions into the workflow at the same place. Insert the dry run first, so that, even if the real action is enabled, the dry run still runs on the unchanged state. === "Python" - + ```py wf.insert_after("core.function.generateMediumLevelIL", [ "extension.my_extension.do_the_thing.dry_run", diff --git a/docs/dev/bnil-overview.md b/docs/dev/bnil-overview.md index fd56d5d9..3903519f 100644 --- a/docs/dev/bnil-overview.md +++ b/docs/dev/bnil-overview.md @@ -1,6 +1,6 @@ # Binary Ninja Intermediate Language: Overview -The Binary Ninja Intermediate Language (BNIL) is a semantic representation of the assembly language instructions for a native architecture in Binary Ninja. BNIL is actually a family of intermediate languages that work together to provide functionality at different abstraction layers. +The Binary Ninja Intermediate Language (BNIL) is a semantic representation of the assembly language instructions for a native architecture in Binary Ninja. BNIL is actually a family of intermediate languages that work together to provide functionality at different abstraction layers. BNIL is a [tree-based](https://raw.githubusercontent.com/withzombies/bnil-graph/master/images/graph.png), architecture-independent intermediate representation of machine code used throughout Binary Ninja. @@ -16,7 +16,7 @@ This short introduction is a very brief guide to BNIL with much more details in ## Summary -While the image above shows a bit of the logical relationships between each IL in terms of analysis, it's only an approximation. The actual analysis includes more edges than just those shown. That said, here's a short overview of some of the differences between each IL and when you might wish to use a given one: +While the image above shows a bit of the logical relationships between each IL in terms of analysis, it's only an approximation. The actual analysis includes more edges than just those shown. That said, here's a short overview of some differences between each IL and when you might wish to use a given one: - Lifted IL is intended to be a straight-forward translation from native instruction semantics to LLIL instructions. Note that Lifted IL and LLIL share the same instructions and are substantively similar with a few small differences. - Low Level IL (LLIL) is converted from Lifted IL, LLIL removes NOP instructions and folds flags into conditional instructions but otherwise contains the same operations as Lifted IL. @@ -29,7 +29,7 @@ While the image above shows a bit of the logical relationships between each IL i ## Reading IL -All of the various ILs (with the exception of the SSA forms) are intended to be easily human-readable and look much like pseudo-code. There is some shorthand notation that is used throughout the ILs, though, explained below: +All the various ILs (except the SSA forms) are intended to be easily human-readable and look much like pseudocode. There is some shorthand notation that is used throughout the ILs, though, explained below: ### Comparisons @@ -97,7 +97,7 @@ It represents the lower 32-bits of variable `rax_2`, sign-extended into a 64-bit ## Using the API with ILs -When you want to use the API to access BNIL instructions, here are a few tips that will help you with the task. First, if you want to learn what properties different instructions have, instead of manually using `dir()` or looking in the documentation ([1](https://docs.binary.ninja/dev/bnil-llil.html#the-instructions), [2](https://docs.binary.ninja/dev/bnil-mlil.html#the-instruction-set)) lists is to use the [BNIL Graph](https://github.com/Vector35/community-plugins#:~:text=BNIL%20Instruction%20Graph) plugin. Another very useful plugin is the [IL Hierarch](https://github.com/Vector35/community-plugins#:~:text=into%20Binary%20Ninja.-,ilhierarchy,-Fabian%20Freyer) plugin. This plugin is extremely useful for showing the _structure_ of IL instructions relative to one another. You can use several APIS ([1](https://api.binary.ninja/binaryninja.lowlevelil-module.html#binaryninja.lowlevelil.LowLevelILInstruction.show_llil_hierarchy), [2](https://api.binary.ninja/binaryninja.mediumlevelil-module.html#binaryninja.mediumlevelil.MediumLevelILInstruction.show_mlil_hierarchy), [3](https://api.binary.ninja/binaryninja.highlevelil-module.html#binaryninja.highlevelil.HighLevelILInstruction.show_hlil_hierarchy)) to see this overall structure, but the IL Hierarchy plugin lets you select a single IL instructions and see visually which categories of IL instructions it are in. +When you want to use the API to access BNIL instructions, here are a few tips that will help you with the task. First, if you want to learn what properties different instructions have, instead of manually using `dir()` or looking in the documentation ([1](https://docs.binary.ninja/dev/bnil-llil.html#the-instructions), [2](https://docs.binary.ninja/dev/bnil-mlil.html#the-instruction-set)) lists is to use the [BNIL Graph](https://github.com/Vector35/community-plugins#:~:text=BNIL%20Instruction%20Graph) plugin. Another very useful plugin is the [IL Hierarch](https://github.com/Vector35/community-plugins#:~:text=into%20Binary%20Ninja.-,ilhierarchy,-Fabian%20Freyer) plugin. This plugin is extremely useful for showing the _structure_ of IL instructions relative to one another. You can use several APIs ([1](https://api.binary.ninja/binaryninja.lowlevelil-module.html#binaryninja.lowlevelil.LowLevelILInstruction.show_llil_hierarchy), [2](https://api.binary.ninja/binaryninja.mediumlevelil-module.html#binaryninja.mediumlevelil.MediumLevelILInstruction.show_mlil_hierarchy), [3](https://api.binary.ninja/binaryninja.highlevelil-module.html#binaryninja.highlevelil.HighLevelILInstruction.show_hlil_hierarchy)) to see this overall structure, but the IL Hierarchy plugin lets you select a single IL instruction and see visually which categories of IL instructions it is in.  @@ -115,9 +115,9 @@ Here's what that instruction might look like when selected with the IL Hierarchy  -Be warned though! HLIL in particular is very tree-based. LLIL and MLIL are much safer to use the above paradigm of simply iterating through top-level instructions. +Be warned though! HLIL in particular is very tree-based. LLIL and MLIL are much safer to use the above paradigm of simply iterating through top-level instructions. -Make sure to also check out the specifics of each IL level for more details: [LLIL](bnil-llil.md), [MLIL](bnil-mlil.md), [HLIL](bnil-hlil.md) +Make sure to also check out the specifics of each IL level for more details: [LLIL](bnil-llil.md), [MLIL](bnil-mlil.md), [HLIL](bnil-hlil.md) ## Visitors @@ -139,7 +139,7 @@ The visitor receives 4 operands: ... match inst: ... case Arithmetic(right=Constant()): ... print(f"{inst.address:#x} {inst}") -... +... >>> current_hlil.root.visit(visitor) 0x4012a0 arg1 + 0x1404 0x4012b0 edx_1 u>> 3 diff --git a/docs/dev/concepts.md b/docs/dev/concepts.md index 376f485b..8e70dba7 100644 --- a/docs/dev/concepts.md +++ b/docs/dev/concepts.md @@ -16,7 +16,7 @@ When you are interacting in the UI with an executable file, you can access `bv` ???+ Info "Tip" Note the use of `bv` here as a shortcut to the currently open BinaryView. For other "magic" variables, see the [user guide](../guide/index.md#magic-console-variables) -If you want to start writing a plugin, most top-level methods will exist off of the BinaryView. Conceptually, you can think about the organization as a hierarchy starting with a BinaryView, then functions, then basic blocks, then instructions. There are of course lots of other ways to access parts of the binary but this is the most common organization. Check out the tab completion in the scripting console for `bv.get<TAB>` for example (a common prefix for many APIs): +If you want to start writing a plugin, most top-level methods will exist off of the BinaryView. Conceptually, you can think about the organization as a hierarchy starting with a BinaryView, then functions, then basic blocks, then instructions. There are of course lots of other ways to access parts of the binary, but this is the most common organization. Check out the tab completion in the scripting console for `bv.get<TAB>` for example (a common prefix for many APIs):  @@ -24,7 +24,7 @@ Some BinaryViews have parent views. The view used for decompilation includes mem ## REPL versus Scripts -When you're interacting with the Binary Ninja [scripting console](../guide/index.md#script-python-console), it's important to realize that every time you run a command, the UI is automatically going to update analysis. You can see this by even running a simply command like: +When you're interacting with the Binary Ninja [scripting console](../guide/index.md#script-python-console), it's important to realize that every time you run a command, the UI is automatically going to update analysis. You can see this by even running a simple command like: ``` print("test") @@ -36,17 +36,17 @@ To achieve the same results when writing a stand-alone plugin, you'd need to use Another difference in the REPL / scripting console is that the scripting console is not executing on the main thread. This means that if you wish to interact with the UI via QT or BN UI APIs from the console (for example, to [trigger an action via string](https://gist.github.com/psifertex/6fbc7532f536775194edd26290892ef7#file-trigger_actions-py)), you'd need to use [`mainthread.execute_on_main_thread_and_wait()`](https://api.binary.ninja/binaryninja.mainthread-module.html#binaryninja.mainthread.execute_on_main_thread_and_wait) or similar. - + ## Auto vs User -In the Binary Ninja API, there are often two parallel sets of functions with `_auto_` and `_user_` in their names. For example: [add_user_segment](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.add_user_segment) and [add_auto_segment](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.add_auto_segment). So what's the difference? Auto functions are those that are expected to be run _automatically_, every time the file is loaded. So for example, if you're writing a [custom file loader](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/nsf.py) that will parse a particular binary format, you would use `_auto_` functions because each time the file is opened your loader will be used. The results of auto functions are saved in a `.bndb` database, but will be cleared if re-running auto-analysis (which happens when updating the version of the database due to updating Binary Ninja in some instances). This is because it's expected that whatever produced them originally will again when the file is re-analyzed (you can use the [`analysis.database.suppressReanalysis`](../guide/settings.md#analysis.database.suppressReanalysis) setting to avoid this, but it's generally discouraged). This means that even if you're writing a plugin to make changes on a file during analysis, you likely want to use the `_user_` set of APIs so that the changes your plugin causes will separately be saved to the database as if they were done by a user. User actions are also added to the undo serialization and can be undone by the user which is not true of auto actions. +In the Binary Ninja API, there are often two parallel sets of functions with `_auto_` and `_user_` in their names. For example: [add_user_segment](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.add_user_segment) and [add_auto_segment](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.add_auto_segment). So what's the difference? Auto functions are those that are expected to be run _automatically_, every time the file is loaded. So for example, if you're writing a [custom file loader](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/nsf.py) that will parse a particular binary format, you would use `_auto_` functions because each time the file is opened your loader will be used. The results of auto functions are saved in a `.bndb` database, but will be cleared if re-running auto-analysis (which happens when updating the version of the database due to updating Binary Ninja in some instances). This is because it's expected that whatever produced them originally will again when the file is re-analyzed (you can use the [`analysis.database.suppressReanalysis`](../guide/settings.md#analysis.database.suppressReanalysis) setting to avoid this, but it's generally discouraged). This means that even if you're writing a plugin to make changes on a file during analysis, you likely want to use the `_user_` set of APIs so that the changes your plugin causes will separately be saved to the database as if they were done by a user. User actions are also added to the undo serialization and can be undone by the user which is not true of auto actions. ## Concepts for ILs ### Walking ILs -Because our ILs are tree-based, some naive plugins that walk IL looking for specific operations will miss instructions that are part of nested expressions. While there is generally less folding in [MLIL](https://docs.binary.ninja/dev/bnil-mlil.html), making it better target for simple loops like: +Because our ILs are tree-based, some naive plugins that walk IL looking for specific operations will miss instructions that are part of nested expressions. While there is generally less folding in [MLIL](https://docs.binary.ninja/dev/bnil-mlil.html), making it the better target for simple loops like: ```python for i in bv.mlil_instructions: @@ -54,7 +54,7 @@ for i in bv.mlil_instructions: print(i.params) ``` -it's still possible to miss instructions with this approach. Additionally, in HLIL it's even easier to miss instructions as there is significantly more nesting. Accordingly we created [`traverse`](https://api.binary.ninja/binaryninja.highlevelil-module.html#binaryninja.highlevelil.HighLevelILFunction.traverse) APIs to walk these tree-based ILs and match whatever property you're interested in. Here are several examples: +it's still possible to miss instructions with this approach. Additionally, in HLIL it's even easier to miss instructions as there is significantly more nesting. Accordingly, we created [`traverse`](https://api.binary.ninja/binaryninja.highlevelil-module.html#binaryninja.highlevelil.HighLevelILFunction.traverse) APIs to walk these tree-based ILs and match whatever property you're interested in. Here are several examples: ```python def find_strcpy(i, t) -> str: @@ -70,7 +70,7 @@ t = [ # Find the first call to a builtin: for result in current_hlil.traverse(find_strcpy, t): # Any logic should live here, not inside the callable which is just for - # matching. Because this is a generator, it can fail fast when used for + # matching. Because this is a generator, it can fail fast when used for # search! print(result) break @@ -153,11 +153,11 @@ It is easy to confuse ExpressionIndex and InstructionIndex properties in the API Our [BNIL Overview](bnil-overview.md) mentions Static Single Assignment (SSA) without explaining what it is. You can of course always check out [Wikipedia](https://en.wikipedia.org/wiki/Static_single-assignment_form), but here's a quick summary of what it is, why we expose multiple SSA forms of our ILs and how you can use it to improve your plugin writing with BNIL. -At it's simplest, SSA forms of an Intermediate Language or Intermediate Representation are those in which variables are read only. They can be set when created, but not modified. Instead of modifying them, when you make a change, a new "version" of the variable is created denoting that the value has changed in some way. While this certainly makes the form less readable to humans, this means it's actually really easy to walk back and see how a variable has been modified by simply looking at the definition site for a given variable or all possible versions before it. Any time the variable is modified, a new version of that variable will be created (you'll see `#` followed by a number indicating that this is a new version). +At it's simplest, SSA forms of an Intermediate Language or Intermediate Representation are those in which variables are read only. They can be set when created, but not modified. Instead of modifying them, when you make a change, a new "version" of the variable is created denoting that the value has changed in some way. While this certainly makes the form less readable to humans, this means it's actually really easy to walk back and see how a variable has been modified by simply looking at the definition site for a given variable or all possible versions before it. Any time the variable is modified, a new version of that variable will be created (you'll see `#` followed by a number indicating that this is a new version). But what if the code takes multiple paths and could have different values depending on the path that was taken? SSA forms use a `Φ` (pronounced either as "fee" or "fi" like "fly" depending on which mathematician or Greek speaker you ask!) function to solve this ambiguity. All a `Φ` function does is aggregate different versions of a variable when entering a basic block with multiple paths. So a basic block with two incoming edges where `eax` is modified might have something like: `eax#3 = Φ(eax#1, eax#2)` denoting that the new version of the variable could have come from either of those sources. -Binary Ninja uses this capability internally for its own value set analysis and constant dataflow propagation but plugins can also leverage this information to great effect. For example, want to find an uninitialized value? Simply look for an SSA variable being read from with a version of zero that isn't in the list of arguments to the function. Want to implement your own inter-procedural data-flow system? Binary Ninja does not for performance reasons, but in instances where you can prevent the state space explosion problem, you can build on top of the existing SSA forms to implement exactly this. A simple example might look for vulnerable function calls like printf() where the first argument is user-data. While most trivial cases of this type of flaw tend to be found quickly, it's often the case that subtler versions with functions that wrap functions that wrap functions that call a printf with user data are more tedious to identify. However, using an SSA-based script, it's super easy to see that, for example, the first parameter to a `printf` call originated in a calling function as the second parameter, and THAT function was called with input that came directly from some sort of user input. While one or two layers might be easy to check by hand with few cross-references, with a large embedded firmware, there might be hundreds or thousands of potential locations to check out, and a script using SSA can dramatically reduce the number of cases to investigate. +Binary Ninja uses this capability internally for its own value set analysis and constant dataflow propagation but plugins can also leverage this information to great effect. For example, want to find an uninitialized value? Simply look for an SSA variable being read from with a version of zero that isn't in the list of arguments to the function. Want to implement your own inter-procedural data-flow system? Binary Ninja does not for performance reasons, but in instances where you can prevent the state space explosion problem, you can build on top of the existing SSA forms to implement exactly this. A simple example might look for vulnerable function calls like `printf()` where the first argument is user-data. While most trivial cases of this type of flaw tend to be found quickly, it's often the case that subtler versions with functions that wrap functions that wrap functions that call a `printf` with user data are more tedious to identify. However, using an SSA-based script, it's super easy to see that, for example, the first parameter to a `printf` call originated in a calling function as the second parameter, and THAT function was called with input that came directly from some sort of user input. While one or two layers might be easy to check by hand with few cross-references, with a large embedded firmware, there might be hundreds or thousands of potential locations to check out, and a script using SSA can dramatically reduce the number of cases to investigate. ### When IL APIs Return None @@ -195,11 +195,11 @@ Binary Ninja utilizes two distinct data flow systems that are influenced by memo The distinction between readable and writable values affects how data flow analysis is performed: - **Readable Values:** Variables marked as readable are analyzed primarily for the flow of data without modification, aiding in tracking data dependencies. -- **Writable Values:** Writable variables are treated as mutable and thus no assumptions are made about the values being the same in a given function. +- **Writable Values:** Writable variables are treated as mutable and thus no assumptions are made about the values being the same in a given function. By overriding variable annotations or memory flags, you can alter Binary Ninja's assumptions about data: -- Marking a writable variable as **`const`** forces the analysis to treat its value as immutable, potentially simplifying the data flow but risking misinterpretation if the assumption is incorrect. +- Marking a writable variable as **`const`** forces the analysis to treat its value as immutable, potentially simplifying the data flow but risking misinterpretation if the assumption is incorrect. - Marking a constant variable as **`volatile`** forces the analysis to treat its value as mutable, safeguarding against data flow making assumptions about the value, this is particularly useful when dealing with data variables in sections marked with [ReadOnlyDataSectionSemantics](https://api.binary.ninja/binaryninja.enums-module.html#binaryninja.enums.SectionSemantics). ## Permissions Impact on Linear Sweep @@ -214,7 +214,7 @@ By modifying memory permissions, you can guide Binary Ninja's linear sweep analy ## UI Elements -There are several ways to create UI elements in Binary Ninja. The first is to use the simplified [interaction](https://api.binary.ninja/binaryninja.interaction-module.html) API which lets you make simple UI elements for use in GUI plugins in Binary Ninja. As an added bonus, they all have fallbacks that will work in headless console-based applications as well. Plugins that use these API include the [angr](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/angr_plugin.py) and [nampa](https://github.com/kenoph/nampa) plugins. +There are several ways to create UI elements in Binary Ninja. The first is to use the simplified [interaction](https://api.binary.ninja/binaryninja.interaction-module.html) API which lets you make simple UI elements for use in GUI plugins in Binary Ninja. As an added bonus, they all have fallbacks that will work in headless console-based applications as well. Plugins that use these APIs include the [angr](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/angr_plugin.py) and [nampa](https://github.com/kenoph/nampa) plugins. The second and more powerful (but more complicated) mechanism is to leverage the _binaryninjaui_ module. Additional documentation is forthcoming, but there are several examples ([1](https://github.com/Vector35/kaitai), [2](https://github.com/Vector35/snippets), [3](https://github.com/Vector35/binaryninja-api/tree/dev/python/examples/triage)), and most of the APIs are backed by the [documented C++ headers](https://api.binary.ninja/cpp). Additionally, the generated _binaryninjaui_ module is shipped with each build of binaryninja and the usual python `dir()` instructions are helpful for exploring its capabilities. @@ -226,21 +226,21 @@ When calling native UI methods, please make sure to use the [execute_on_main_thr One of the common questions asked of a binary analysis platform is "how big is a function?". This is a deceptively simple question without a simple answer. There are rather several equally valid definitions you could give for what is the size of a function: - - the total sum of all basic blocks? - - the highest virtual address in the function minus the lowest virtual address? - - the address of return instruction subtracted from the entry point + - The total sum of all basic blocks? + - The highest virtual address in the function minus the lowest virtual address? + - The address of return instruction subtracted from the entry point -Except that last one is a trick of course. Because not only can functions have multiple return instructions, but they may have multiple entry points (as is often the case with error handling). +Except that last one is a trick of course. Because not only can functions have multiple return instructions, but they may have multiple entry points (as is often the case with error handling). -Basic blocks have, by definition, a start, and an end. Basic Blocks can therefore have consistent sizes that all binary analysis tools would agree upon (though more formal analysis might stop basic blocks on call instructions while for convenience sake, most reverse engineering tools do not). +Basic blocks have, by definition, a start, and an end. Basic Blocks can therefore have consistent sizes that all binary analysis tools would agree upon (though more formal analysis might stop basic blocks on call instructions while for convenience's sake, most reverse engineering tools do not). -Summing up the basic blocks of a function is one way to produce a consistent size for a function, but how do you handle bytes that overlap standard function definitions, for example, via a tail call? Or via a mis-aligned jump where a byte is in two basic blocks? Different tools may resolve those ambiguous situations in different ways, so again, it is difficult to compare the "size" of any one binary analysis tool to another. +Summing up the basic blocks of a function is one way to produce a consistent size for a function, but how do you handle bytes that overlap standard function definitions, for example, via a tail call? Or via a misaligned jump where a byte is in two basic blocks? Different tools may resolve those ambiguous situations in different ways, so again, it is difficult to compare the "size" of any one binary analysis tool to another. In Binary Ninja, there is no explicit `.size` property of functions. Rather, you can choose to calculate it one of two ways: ``` function_size = current_function.total_bytes -# or +# or function_size = current_function.highest_address - current_function.lowest_address ``` @@ -248,14 +248,14 @@ Total bytes is similar to the first proposed definition above. It merely sums up ### When does a Function stop? -One reason that having an "end" might be useful in a function (as opposed to the `.highest_address` in Binary Ninja), would be to make it a property that controls the analysis for a function. Unlike some other systems where it's possible to define the start and end of a function, in Binary Ninja, you merely define the start and allow analysis to occur naturally. The end results when all basic blocks terminate either in: +One reason that having an "end" might be useful in a function (as opposed to the `.highest_address` in Binary Ninja), would be to make it a property that controls the analysis for a function. Unlike some other systems where it's possible to define the start and end of a function, in Binary Ninja, you merely define the start and allow analysis to occur naturally. The end results when all basic blocks terminate either in: - an invalid instruction - a return instruction - a call to a function marked as `__noreturn__` - a branch to a block already in the function - any other instruction such as an interrupt that by its definition stops analysis - + So how do you tell Binary Ninja how big a function is? The answer is you don't directly, but you can instead direct its analysis. For example, if a function is improperly not marked as a noreturn function, edit the function properties via the right-click menu and set the property and all calls to it will end analysis at that point in any callees. Likewise, you can do things like change memory permissions, patch in invalid instructions, [change an indirect branch's targets](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/jump_table.py), or use [UIDF](https://binary.ninja/2020/09/10/user-informed-dataflow.html) to influence analysis such that it ends where desired. @@ -264,7 +264,7 @@ Likewise, you can do things like change memory permissions, patch in invalid ins ### Unicode Support -If you're opening a file with non-ascii string encodings, make sure to use one of the "open with options" [methods](../guide/index.md#loading-files) and choose the appropriate code pages. These settings are under the `analysis` / `unicode` section of the settings dialog. If you're not sure which code page you should be enabling, consider using a library like [chardet](https://chardet.readthedocs.io/en/latest/usage.html). For example, here's a file both with and without the "CJK Unified Ideographs" block being enabled: +If you're opening a file with non-ASCII string encodings, make sure to use one of the "open with options" [methods](../guide/index.md#loading-files) and choose the appropriate code pages. These settings are under the `analysis` / `unicode` section of the settings dialog. If you're not sure which code page you should be enabling, consider using a library like [chardet](https://chardet.readthedocs.io/en/latest/usage.html). For example, here's a file both with and without the "CJK Unified Ideographs" block being enabled:  @@ -272,10 +272,10 @@ If you're opening a file with non-ascii string encodings, make sure to use one o ???+ Info "Tip" Note that there is a small bug and some string length calculations will be off with certain characters as shown above. - + ### Wide Strings in the UI -Many Windows applications use wide (UTF-16 or UTF-32) strings. By default these should be identified during initial string analysis as long as the default settings are still enabled. +Many Windows applications use wide (UTF-16 or UTF-32) strings. By default, these should be identified during initial string analysis as long as the default settings are still enabled.  diff --git a/docs/dev/flags.md b/docs/dev/flags.md index e87c6a2d..8474e73e 100644 --- a/docs/dev/flags.md +++ b/docs/dev/flags.md @@ -7,7 +7,7 @@ This guide is broken up into three sections. - Part 3 covers advanced topics like custom roles -## Binja Architecture Flags Part 1: The Basics +## Binary Ninja Architecture Flags Part 1: The Basics Think of flags as global boolean variables, set and cleared by observers who monitor the result of instructions from a distance: @@ -28,11 +28,11 @@ flag_v = ... Now the problem is that tons of instructions affect tons of flags, so a lot of noise is generated. -Binja tries to reduce the noise by displaying only the flags between producer and consumer that are relevant. For example, ADD affects many flags, but if none are read until ADC (add with carry) then only the carry flag is shown. +Binary Ninja tries to reduce the noise by displaying only the flags between producer and consumer that are relevant. For example, ADD affects many flags, but if none are read until ADC (add with carry) then only the carry flag is shown. -Writing effective flag code in architectures requires this understanding, and there's a bit of an art in "helping" Binja connect flag producers and consumers. +Writing effective flag code in architectures requires this understanding, and there's a bit of an art in "helping" Binary Ninja connect flag producers and consumers. -But this is about basics! So what are the basic ways in which an architecture author informs Binja of flags? The following is python architecture code for Z80. +But this is about basics! So what are the basic ways in which an architecture author informs Binary Ninja of flags? The following is python architecture code for Z80. ### 1) declare the flags @@ -53,7 +53,7 @@ That's simple, it's just a list of strings. 'c': FlagRole.CarryFlagRole } -Map each flag to a role from [api/python/enum.py](https://api.binary.ninja/_modules/binaryninja/enums.html) if possible. This informs Binja to generate IL for the basic, textbook behavior of a flag. For example, the `ZeroFlagRole` will generate IL that sets the flag when an arithmetic result is zero. +Map each flag to a role from [api/python/enum.py](https://api.binary.ninja/_modules/binaryninja/enums.html) if possible. This informs Binary Ninja to generate IL for the basic, textbook behavior of a flag. For example, the `ZeroFlagRole` will generate IL that sets the flag when an arithmetic result is zero. If a flag's behavior does not fit the textbook behavior, use `SpecialFlagRole` and in a future article we'll implement a callback for its custom IL. For example, `PV` here in Z80 acts as both a parity flag and an overflow flag. It's meaning at any given time depends on the last instruction that set it. @@ -77,13 +77,13 @@ Think of `flag_write_types` as custom named groups for your convenience. When yo In fact, the `flag` keyword parameter for the IL constructing functions does not accept individual flag names, it accepts only these group names! -## Binja Architecture Flags Part 2: Flag Producer/Consumer +## Binary Ninja Architecture Flags Part 2: Flag Producer/Consumer -In the last installment, we informed Binja of our architecture's flags by defining a list of flag names, a mapping from flag names to `flag roles`, and groups of flags commonly set together called `flag write types`. +In the last installment, we informed Binary Ninja of our architecture's flags by defining a list of flag names, a mapping from flag names to `flag roles`, and groups of flags commonly set together called `flag write types`. -Let's remind ourselves briefly that these are terms within an imagined protocol between architecture author and Binja. The protocol's purpose is to communicate to Binja what flags are present and how they behave. As we accumulate architectures that stretch the protocol's limits (eg: PowerPC with its flag banks), it's possible the protocol gets adjusted to more generally accommodate architectures. +Let's remind ourselves briefly that these are terms within an imagined protocol between architecture author and Binary Ninja. The protocol's purpose is to communicate to Binary Ninja what flags are present and how they behave. As we accumulate architectures that stretch the protocol's limits (eg: PowerPC with its flag banks), it's possible the protocol gets adjusted to more generally accommodate architectures. -In Part 1, we said Binja tries to reduce the noise in displayed IL by showing only the flags between producer and consumer that are relevant. Consider this simplified flags definition for Z80: +In Part 1, we said Binary Ninja tries to reduce the noise in displayed IL by showing only the flags between producer and consumer that are relevant. Consider this simplified flags definition for Z80: ```python flags = ['c'] @@ -92,7 +92,7 @@ flag_write_types = ['none', 'only_carry'] flags_written_by_flag_write_type = { 'none': [], 'only_carry': ['c'], } ``` -We have one flag named 'c', we tell Binja that it's the textbook carry flag, and we define a flag group called "only_carry" which sets just this flag. +We have one flag named 'c', we tell Binary Ninja that it's the textbook carry flag, and we define a flag group called "only_carry" which sets just this flag. At lifting time, we mark certain IL instructions as a **producer** using the `flags` keyword. Remember to pass the _group_ or _flag write type_ that contains `c`, **not** `c` itself: @@ -121,7 +121,7 @@ unsigned char add(unsigned char a, unsigned char b) Recap: the C function is named `add()` which will produce a Z80 instruction `ADD` (among others) which we lift to LLIL `ADD` that is a **producer** of the `c` flag. -Binja gives this LLIL, after compilation with [SDCC](http://sdcc.sourceforge.net): +Binary Ninja gives this LLIL, after compilation with [SDCC](http://sdcc.sourceforge.net): ``` _add: @@ -137,7 +137,7 @@ _add: WHERE'S THE CARRY!? -This is the lesson that was non-intuitive to me. Binja knows from the architecture-supplied _lifted IL_ that `ADD` sets `c`, but it doesn't produce any `c` setting _low level_ IL because it failed to detect anyone using `c`. +This is the lesson that was non-intuitive to me. Binary Ninja knows from the architecture-supplied _lifted IL_ that `ADD` sets `c`, but it doesn't produce any `c` setting _low level_ IL because it failed to detect anyone using `c`. #### Example B @@ -173,7 +173,7 @@ _add: 16 @ 00000225 <return> jump(pop) ``` -With `ADC` present, Binja detects the relationship, and produces the IL `flag:c = temp0.b + temp1.b u< temp0.b`. +With `ADC` present, Binary Ninja detects the relationship, and produces the IL `flag:c = temp0.b + temp1.b u< temp0.b`. This also means that you can mark instructions as producers of a group of many flags, and LLIL will contain the flag calculating code for those flags consumed further along. @@ -181,18 +181,18 @@ This also means that you can mark instructions as producers of a group of many f You, the architecture author: -* inform Binja of your architecture's flags by defining: +* inform Binary Ninja of your architecture's flags by defining: * flag names * flag "roles" which are just their textbook behavior, if they qualify * flag "write types" which groups of flags often set together * assign instructions as flag **producers** by passing `flag=` keyword parameter during lifting * assign instructions as flag **consumers** by passing IL flag expressions as operands during lifting -With the defined variables and lifted IL, Binja decides when to generate flag-affecting low level IL. +With the defined variables and lifted IL, Binary Ninja decides when to generate flag-affecting low level IL. -## Binja Architecture Flags Part 3: Flag Roles +## Binary Ninja Architecture Flags Part 3: Flag Roles -The previous sections said that flag roles were the textbook behavior of flags. If you can assign a flag to a role, there's a chance Binja will have that default behavior and you won't have to implement it yourself. For example, the carry flag is so common among architectures and its behavior is (to my knowledge) non-varying during addition, that we have it hardcoded into Binja: +The previous sections said that flag roles were the textbook behavior of flags. If you can assign a flag to a role, there's a chance Binary Ninja will have that default behavior and you won't have to implement it yourself. For example, the carry flag is so common among architectures and its behavior is (to my knowledge) non-varying during addition, that we have it hard-coded into Binary Ninja: ``` 5 @ 0000021b temp0.b = A @@ -201,9 +201,9 @@ The previous sections said that flag roles were the textbook behavior of flags. 8 @ 0000021b flag:c = temp0.b + temp1.b u< temp0.b <-- HERE ``` -The `c` flag was declared to have CarryFlagRole so the `flag:c = ...` was produced by Binja and the architecture author is not responsible. +The `c` flag was declared to have CarryFlagRole so the `flag:c = ...` was produced by Binary Ninja and the architecture author is not responsible. -The hardcoded flag code is in the `get_flag_write_low_level_il()` method of the Architecture class and (at the time of this writing) supports: +The hard-coded flag code is in the `get_flag_write_low_level_il()` method of the Architecture class and (at the time of this writing) supports: | | ADD | ADC | SUB | NEG | FSUB | OTHERS | | -------------------- | ---- | ---- | ---- | ---- | ---- | ------ | @@ -215,9 +215,9 @@ The hardcoded flag code is in the `get_flag_write_low_level_il()` method of the | UnorderedFlagRole | | | | | yes | | | ZeroFlagRole | yes | yes | yes | yes | yes | yes | -If a cell has "yes" it means that for the instruction at the column header, there's an built-in **attempt** at producing the flag in the row. It is not a guarantee that the flag semantics match your architecture's. +If a cell has "yes" it means that for the instruction at the column header, there's a built-in **attempt** at producing the flag in the row. It is not a guarantee that the flag semantics match your architecture's. -So for any instructions that are not ADD, ADC, SUB, NEG, or FSUB, you can try and see if Binja's built-in for calculating negative, positive, or zero flags are accurate for your architecture. If you want carry, ordered, overflow, or unordered, you will need to implement it yourself. +So for any instructions that are not ADD, ADC, SUB, NEG, or FSUB, you can try and see if Binary Ninja's built-in for calculating negative, positive, or zero flags are accurate for your architecture. If you want carry, ordered, overflow, or unordered, you will need to implement it yourself. What happens when you mark an instruction the producer of a certain flag, but no implementation exists? @@ -229,9 +229,9 @@ What happens when you mark an instruction the producer of a certain flag, but no Here, `sbb` is an `s` producer and a `pv` producer. -Flag `s` is mapped to FlagRole.NegativeSignRole which is "yes" in the table, so Binja emits the default LLIL to set it. +Flag `s` is mapped to FlagRole.NegativeSignRole which is "yes" in the table, so Binary Ninja emits the default LLIL to set it. -Flag `pv` is mapped to FlagRole.OverflowFlagRole which is not "yes", so Binja doesn't have an implementation and emits LLIL_UNIMPLEMENTED. +Flag `pv` is mapped to FlagRole.OverflowFlagRole which is not "yes", so Binary Ninja doesn't have an implementation and emits LLIL_UNIMPLEMENTED. If you need to define your own flag setting code, set the flag role to SpecialFlagRole and override `get_flag_write_low_level_il()`. There are two contraints you must work around: @@ -260,7 +260,7 @@ def get_flag_write_low_level_il(self, op, size, write_type, flag, operands, il): return il.not_expr(0, self.get_default_flag_write_low_level_il(op, size, FlagRole.CarryFlagRole, operands, il)) # Other operations use a normal carry flag return self.get_default_flag_write_low_level_il(op, size, FlagRole.CarryFlagRole, operands, il) - ... + ... ``` This is nice and convenient: the normal behavior is simply wrapped in a `not`. See [the source](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/nes.py) for the full code. diff --git a/docs/dev/index.md b/docs/dev/index.md index 7407d7bb..01c355a4 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -24,7 +24,7 @@ The Core API is designed to only be used as a shim from other languages and is n ### C++ API -The C++ API is what the Binary Ninja UI itself is built using so it's a robust and fully feature-complete interface to the core, however, it does not have the same level of detail in the documentation. +The C++ API is what the Binary Ninja UI itself uses, so it's a robust and fully feature-complete interface to the core, however, it does not have the same level of detail in the documentation. - [C++ Header](https://github.com/Vector35/binaryninja-api/blob/dev/binaryninjaapi.h) (along with the rest of the [repository](https://github.com/Vector35/binaryninja-api)) - [Build Instructions](https://github.com/Vector35/binaryninja-api#building) diff --git a/docs/dev/plugins.md b/docs/dev/plugins.md index 5e69fd88..7f0b1901 100644 --- a/docs/dev/plugins.md +++ b/docs/dev/plugins.md @@ -5,13 +5,13 @@ ### Creating the Plugin -First, take a look at some of the [example](https://github.com/Vector35/binaryninja-api/tree/dev/python/examples) plugins, or some of the [community](https://github.com/Vector35/community-plugins) plugins to get a feel for different APIs you might be interested in. Of course, the full [API](https://api.binary.ninja/) docs are online and available offline via the `Help`/`Open Python API Reference...`. +First, take a look at some [example](https://github.com/Vector35/binaryninja-api/tree/dev/python/examples) plugins, or some of the [community](https://github.com/Vector35/community-plugins) plugins to get a feel for different APIs you might be interested in. Of course, the full [API](https://api.binary.ninja/) docs are online and available offline via the `Help`/`Open Python API Reference...`. -To start, we suggest you download the [sample plugin](https://github.com/Vector35/sample_plugin) as a template since it contains all of the elements you're likely to need. +To start, we suggest you download the [sample plugin](https://github.com/Vector35/sample_plugin) as a template since it contains all the elements you're likely to need. - Begin by editing the `plugin.json` file - Next, update the `LICENSE` -- For small scripts, you can include all the code inside of `__init__.py`, though we recommend for most larger scripts that init just act as an initializer and call into functions organized appropriately in other files. +- For small scripts, you can include all the code inside `__init__.py`, though we recommend for larger scripts that `__init__.py` just act as an initializer and calls into functions organized appropriately in other files. - If you have python dependencies, create a [requirements.txt](https://pip.pypa.io/en/latest/cli/pip_freeze/) listing any python dependencies. ### Submitting to the Plugin Manager @@ -33,7 +33,7 @@ The [`add_repository`](https://api.binary.ninja/binaryninja.pluginmanager-module ### Testing -It's useful to be able to reload your plugin during testing. On the Commercial or Ultimate editions of Binary Ninja, this is easily accomplished with a stand-alone headless install using `import binaryninja` after [installing the API](https://github.com/Vector35/binaryninja-api/blob/dev/scripts/install_api.py). (install_api.py is included in each platforms respective [installation folder](../guide/index.md#binary-path)) +It's useful to be able to reload your plugin during testing. On the Commercial or Ultimate editions of Binary Ninja, this is easily accomplished with a stand-alone headless install using `import binaryninja` after [installing the API](https://github.com/Vector35/binaryninja-api/blob/dev/scripts/install_api.py). (install_api.py is included in each platform's respective [installation folder](../guide/index.md#binary-path)) Additionally, some plugin types like Architectures or BinaryViews are only loaded at launch and cannot be reloaded during a running session. @@ -45,7 +45,7 @@ import importlib importlib.reload(pluginname);pluginname.callbackmethod(bv) ``` -Then just `[UP] [ENTER]` to trigger the reload when the plugin has changed. +Then just `[UP] [ENTER]` to trigger a reload when the plugin has changed. ## Writing plugins using other IDEs (tab completion) @@ -83,14 +83,14 @@ Binary Ninja UI plugins should always `import binaryninjaui` before an `import P UI plugins can take many forms. Some, like [Snippets](https://github.com/vector35/snippets) create their own UI elements and interact via UIActions. Others extend the UI via existing UI elements such as [Triage](https://github.com/Vector35/binaryninja-api/tree/dev/python/examples/triage), [Kaitai](https://github.com/Vector35/kaitai), [hellosidebar](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/hellosidebar.py), or [helloglobalarea](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/helloglobalarea.py). -Many other [third-party](https://github.com/vector35/community-plugins/) plugins also implement UI based examples, such as [BNIL Graph](https://github.com/withzombies/bnil-graph) using the FlowGraph APIs. +Many other [third-party](https://github.com/vector35/community-plugins/) plugins also implement UI based examples, such as [BNIL Graph](https://github.com/withzombies/bnil-graph) using the FlowGraph APIs. Unfortunately, due to a PySide documentation generation issue, the best and most reliable documentation on the UI system is not in the regular [python](https://api.binary.ninja/) API docs, but in the [C++ documentation](https://api.binary.ninja/cpp/) which translates fairly cleanly to their python equivalent. ## Writing Native Plugins -Writing native plugins allows for higher performance code and lower level access to the Binary Ninja API, but comes with a couple more hurdles than Python. +Writing native plugins allows for higher performance code and lower level access to the Binary Ninja API, but comes with a couple more hurdles than Python. Notably, native plugins are built against a specific version of the API, cannot be hot-reloaded, and require more sophisticated build setups. ### Supported Toolchains @@ -112,16 +112,16 @@ although it is recommended to use the latest version. ### Project Setup -The first things to specify in your CMake file are a couple boilerplate options for building C++: +The first things to specify in your CMake file are a couple boilerplate options for building C++: # Pick whatever version you have cmake_minimum_required(VERSION 3.24) # Name your plugin project(TestPlugin CXX) - + set(CMAKE_CXX_STANDARD 20) - + # Unless you are writing a plugin that needs Qt's UI, specify this set(HEADLESS 1) @@ -135,14 +135,14 @@ and reference it directly in your plugin. If you're using git, this can be accom git submodule add https://github.com/Vector35/binaryninja-api.git binaryninjaapi cd binaryninjaapi # Pick the revision from api_REVISION.txt - git checkout 6466fba3341b2ea7dbfceeeebbc6c0322a5d8514 + git checkout 6466fba3341b2ea7dbfceeeebbc6c0322a5d8514 If you're not using git, you can clone the repository elsewhere: git clone https://github.com/Vector35/binaryninja-api.git binaryninjaapi - cd binaryninjaapi + cd binaryninjaapi # Pick the revision from api_REVISION.txt - git checkout 6466fba3341b2ea7dbfceeeebbc6c0322a5d8514 + git checkout 6466fba3341b2ea7dbfceeeebbc6c0322a5d8514 Now that you have the correct copy of the api, you need to point CMake at it and include it for use. Include something like the following in your CMake script and either add the path of your clone @@ -162,18 +162,18 @@ Be sure to create a shared library and link against the Binary Ninja api. Also, when you use `cmake install`. # Use whichever sources and plugin name you want - add_library(TestPlugin SHARED TestPlugin.cpp) - + add_library(TestPlugin SHARED TestPlugin.cpp) + # Link with Binary Ninja target_link_libraries(TestPlugin PUBLIC binaryninjaapi) - + # Tell `cmake --install` to copy your plugin to the plugins directory bn_install_plugin(TestPlugin) From there you can write the rest of your plugin's CMake configuration, including any other dependencies or options that you want. When you want to run your plugin, you can use `cmake --build` and `cmake --install` to compile and copy your plugin to your Binary Ninja plugins directory, or set up an IDE to do that for you. -You could also copy the plugin manually if you are using a different plugins directory location. +You could also copy the plugin manually if you are using a different plugins directory location. In the source code of your plugin, you will need to export some functions that Binary Ninja uses to load your plugin at runtime: @@ -182,7 +182,7 @@ at runtime: extern "C" { // Tells Binary Ninja which version of the API you compiled against BN_DECLARE_CORE_ABI_VERSION - + // Function run on plugin startup, do simple initialization here (Settings, BinaryViewTypes, etc) BINARYNINJAPLUGIN bool CorePluginInit() { @@ -206,7 +206,7 @@ class, and you should only ever handle Refs or bare pointers. When in doubt, fee ### Automated GitHub CI -Managing build infrastructure to build cross-platform native plugins can be a headache even for stables, let alone +Managing build infrastructure to build cross-platform native plugins can be a headache even for stables, let alone trying to track all dev releases. To help with that, we've published an [example plugin](https://github.com/Vector35/sample_plugin_cpp) that includes GitHub actions to build on MacOS, Linux, and Windows. Combining this with something like a [plugin loader](https://github.com/rikodot/binja_native_sigscan_loader) can simplify both publishing and using native plugins. @@ -225,10 +225,10 @@ you can amend your CMake file to make a UI plugin. You will need the following C # If you are using Qt MOC (i.e. use Q_OBJECT/Q_SIGNALS/Q_SLOTS) set(CMAKE_AUTOMOC ON) set(CMAKE_AUTORCC ON) - + # Locate Qt installation for linking find_package(Qt6 COMPONENTS Core Gui Widgets REQUIRED) - + # Add MOCS to your build add_library(TestPlugin SHARED library.cpp ${MOCS}) @@ -240,11 +240,11 @@ Then, in your plugin code, instead of using the exported functions for a core pl #include "binaryninjaapi.h" #include "uitypes.h" #include "uicontext.h" - + extern "C" { // Tells Binary Ninja which version of the API you compiled against BN_DECLARE_UI_ABI_VERSION - + // Function run on plugin startup, do simple initialization here (ViewTypes, SidebarWidgetTypes, etc) BINARYNINJAPLUGIN bool UIPluginInit() { @@ -252,7 +252,7 @@ Then, in your plugin code, instead of using the exported functions for a core pl } // (Optional) Function to add other plugin dependencies in case your plugin requires them - // Historically, these have never actually been used + // Historically, these have never actually been used BINARYNINJAPLUGIN void UIPluginDependencies() { // For example, if you require triage view to be loaded before your plugin @@ -277,16 +277,16 @@ are no cookie-cutter solutions to this problem, but there is a general strategy: Again, I'm going to point out [the debugger](https://github.com/vector35/debugger) as a fantastic example of how to implement this. Generally speaking, you will either need to write both sides of the FFI in a similar way, or you may -be able to find a library that does that for you. Possibly [libffi](https://sourceware.org/libffi/), although there +be able to find a library that does that for you. Possibly [libffi](https://sourceware.org/libffi/), although there aren't any examples of using it for Binary Ninja specifically. If you manage to get something working, let us know! -We would love to see more complex plugins with extensible behavior! +We would love to see more complex plugins with extensible behavior! ## IDE Setup ### CLion CLion is generally pretty good at handling CMake projects. Given the above CMake configuration, it can -automatically detect the plugin target and will compile and install correctly. Here are a a few steps to finish +automatically detect the plugin target and will compile and install correctly. Here are a few steps to finish setup for building and live debugging your plugin: 1. If you installed Binary Ninja somewhere other than the default, add an environment variable in your CMake Profile pointing at the installation, e.g.: `BN_INSTALL_DIR=/Applications/Binary Ninja.app` @@ -294,7 +294,7 @@ setup for building and live debugging your plugin: 3. In your Run Configuration's Before Launch steps, add an Install step. This will copy the updated version of your plugin before starting, so you don't have to run Install manually. 4. Set the Executable of your Run Configuration to point to the Binary Ninja executable. This allows you to compile your plugin and start Binary Ninja automatically. i. On macOS, you will need the full path to /Applications/Binary Ninja.app/Contents/MacOS/binaryninja -5. (Optionally) Add the `-e` flag to the Program Arguments to get error logs printed to your console +5. (Optionally) Add the `-e` flag to the Program Arguments to get error logs printed to your console 6. (Optionally) Add the `-e -d` flags to the Program Arguments to get debug logs printed to your console. This may slow down Binary Ninja (and CLion) due to the large volume of logs produced. 7. (Optionally) Add the `-l /tmp/bn_out.txt` flags to the Program Arguments so your logs also get printed to a text file when you inevitably fill up the Console buffer in CLion and want to see what happened. 8. (Optionally on macOS) Add the Environment Variables `MallocScribble=1` and `MallocPreScribble=1` to make memory errors easier to spot. @@ -317,7 +317,7 @@ You need to set up a task in `.vscode/tasks.json` to build and install your plug "detail": "CMake template install task", "options": { "environment": { - // You will need this if your Binary Ninja installation is not in the default location + // You will need this if your Binary Ninja installation is not in the default location "BN_INSTALL_DIR": "C:\\Users\\User\\AppData\\Local\\Vector35\\BinaryNinja", // You will need this if you are writing a UI plugin "PATH": "C:\\Users\\User\\Qt\\6.8.2\\msvc2019_64\\bin" @@ -377,6 +377,6 @@ Several native plugin examples exist: ## Contributing to Official Plugins -There are many many official plugins released as open source. Python ones are included in the [official plugin repository](https://github.com/vector35/official-plugins), [native architectures](https://github.com/vector35/?q=arch-&type=all&language=&sort=) are available on GitHub along with several others that are included with the default product such as the [debugger](https://github.com/Vector35/debugger), the [views](https://github.com/vector35/?q=view-&type=public&language=&sort=), [platforms](https://github.com/vector35/?q=platform&type=public&language=&sort=), and some [rust plugins](https://github.com/Vector35/binaryninja-api/tree/dev/rust/examples). +There are many official plugins released as open source. Python ones are included in the [official plugin repository](https://github.com/vector35/official-plugins), [native architectures](https://github.com/vector35/?q=arch-&type=all&language=&sort=) are available on GitHub along with several others that are included with the default product such as the [debugger](https://github.com/Vector35/debugger), the [views](https://github.com/vector35/?q=view-&type=public&language=&sort=), [platforms](https://github.com/vector35/?q=platform&type=public&language=&sort=), and some [rust plugins](https://github.com/Vector35/binaryninja-api/tree/dev/rust/examples). The first time you contribute, you'll be asked to sign a [CLA](https://gist.github.com/psifertex/a207c2e070f4e342554dc011e920b341) automatically by [cla-assistant](https://cla-assistant.io/). Further commits after the first should not require any changes. diff --git a/docs/dev/themes.md b/docs/dev/themes.md index 8f005b0e..1ca13bf1 100644 --- a/docs/dev/themes.md +++ b/docs/dev/themes.md @@ -117,7 +117,7 @@ Colors marked "*required*" must be specified. Unmarked colors will hold default 3. `numberColor` (*required*) - Used to color number literals (e.g. `0xf0`) 4. `codeSymbolColor` (*required*) - Used to color local function names (e.g. `sub_100003c50`) 5. `dataSymbolColor` (*required*) - Used to color data symbols (e.g. `data_100003e2c`) -6. `stackVariableColor` (*required*) - Used to color stack variables (e.g `var_8`) in disassembly and LLIL (the stack [does not exist](bnil-mlil.md) in MLIL and above) +6. `stackVariableColor` (*required*) - Used to color stack variables (e.g. `var_8`) in disassembly and LLIL (the stack [does not exist](bnil-mlil.md) in MLIL and above) 7. `importColor` (*required*) - Used to color imported function names (e.g. `printf`) 8. `annotationColor` (*required*) - Used to color annotations (e.g. hints), not shown in picture above 9. `commentColor` - Used to color code comments @@ -223,7 +223,7 @@ Both the graph background and individual graph nodes are actually painted as a g 1. `featureMapBaseColor` - Used to color the background 2. `featureMapNavLineColor` - Used to color the line(s) that represent where you are in the binary -3. `featureMapNavHighlightColor` - Used as a highlight outside of the navigation line(s) +3. `featureMapNavHighlightColor` - Used as a highlight outside the navigation line(s) 4. `featureMapDataVariableColor` - Used to highlight any area containing data variables 5. `featureMapAsciiStringColor` - Used to highlight any area containing ASCII strings 6. `featureMapUnicodeStringColor` - Used to highlight any area containing Unicode strings diff --git a/docs/dev/typelibraries.md b/docs/dev/typelibraries.md index 7302d967..5623e087 100644 --- a/docs/dev/typelibraries.md +++ b/docs/dev/typelibraries.md @@ -38,7 +38,7 @@ Therefore, without any alternative names, `libc.so.bntl` will not be loaded by B We recommend and use the following convention: -Type libraries should be named for the filename from which they were generated with the phrase ".bntl" added. When the source library contains additional minor and release number, like `libfoo.so.1.2.3` Binary Ninja would not load the resulting type library `libfoo.so.1.2.3.bntl` for an ELF requesting soname `libfoo.so.1`. Therefore the alternative names list should include the most specific version numbers, incrementally stripped down to the soname, and finally a linkname for good measure. +Type libraries should be named for the filename from which they were generated with the phrase ".bntl" added. When the source library contains additional minor and release number, like `libfoo.so.1.2.3` Binary Ninja would not load the resulting type library `libfoo.so.1.2.3.bntl` for an ELF requesting soname `libfoo.so.1`. Therefore, the alternative names list should include the most specific version numbers, incrementally stripped down to the soname, and finally a linkname for good measure. Example: @@ -285,4 +285,4 @@ When a binary is loaded and its external symbols is processed, the symbol names type library test.bntl found hit for _DoSuperComputation ``` -At this moment, there is no built in functionality to apply named objects to an existing Binary Ninja database. +At this moment, there is no built-in functionality to apply named objects to an existing Binary Ninja database. diff --git a/docs/dev/uidf.md b/docs/dev/uidf.md index 33c4c27f..817bbef7 100644 --- a/docs/dev/uidf.md +++ b/docs/dev/uidf.md @@ -6,11 +6,11 @@ Binary Ninja now implements User-Informed DataFlow (UIDF) to improve the static For the purpose of demonstration, we are going to use a simple [crackme](https://github.com/Vector35/uidf-example). -It is a fairly simple challenge but we can use it to demonstrate how UIDF can be used to progressively simplify the problem through branch elimination. First, we'll show how to use this new feature and then we'll discuss some of the design decisions that went into the implementation. +It is a fairly simple challenge, but we can use it to demonstrate how UIDF can be used to progressively simplify the problem through branch elimination. First, we'll show how to use this new feature, then we'll discuss some of the design decisions that went into the implementation. Let's start by looking at the `main` function. Note that the `rax_*` variables contain values returned from the computation of `check_*` functions. These values are used for determining whether control would flow to the satisfying or failure conditions of the crackme. -Let's first look at `rax_2`. We see that the conditional at Medium Level IL (MLIL) instruction 8 uses a field access into `rax` and checks its value against 0. If it evaluates to true, control-flow jumps to `0x004010a4` and outputs "Incorrect". However if it evaluates to false, we continue down the chain of conditionals which determine if the output is "Correct". To confirm our understanding and simplify the process further, we can inform the dataflow analysis engine with the value for `rax_2` and see how it impacts the control flow of the binary. +Let's first look at `rax_2`. We see that the conditional at Medium Level IL (MLIL) instruction 8 uses a field access into `rax` and checks its value against 0. If it evaluates to true, control-flow jumps to `0x004010a4` and outputs "Incorrect". However, if it evaluates to false, we continue down the chain of conditionals which determine if the output is "Correct". To confirm our understanding and simplify the process further, we can inform the dataflow analysis engine with the value for `rax_2` and see how it impacts the control flow of the binary. First, note the definition site of `rax_2` is instruction 7 and then right click there to see "Set User Variable Value...". Users can also bind the action to a hotkey or access it via the command palette. @@ -107,7 +107,7 @@ We believe that UIDF is great for experimenting around with during the initial p ### Jump-Table Example -Another example of using UIDF would be to construct jump-tables with known bounds. There are several other ways to achieve this same goal that it may be helpful to review. First, as our old [example plugin](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/jump_table.py#L81) demonstrates, you can use the [`set_user_indirect_branches`](https://api.binary.ninja/binaryninja.function-module.html#binaryninja.function.Function.set_user_indirect_branches) API, though this is probably the least easy method to use. Next, you can create an appropriately sized array of pointers at the base of a jump table reference, and the jump table recovery will automatically create all of the appropriate targets. +Another example of using UIDF would be to construct jump-tables with known bounds. There are several other ways to achieve this same goal that it may be helpful to review. First, as our old [example plugin](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/jump_table.py#L81) demonstrates, you can use the [`set_user_indirect_branches`](https://api.binary.ninja/binaryninja.function-module.html#binaryninja.function.Function.set_user_indirect_branches) API, though this is probably the least easy method to use. Next, you can create an appropriately sized array of pointers at the base of a jump table reference, and the jump table recovery will automatically create all the appropriate targets. Now, with the introduction of UIDF, there's another easy way to help inform the jump-table analysis. Consider the MLIL representation for an indirect jump as shown below: @@ -130,6 +130,6 @@ A jump table will be created with a base of `0x804891c` with 14 possible targets Static-only reverse engineering limits the ability to test hypotheses formulated during the initial analysis phase. To that end, analysts usually turn to emulation/debugging to experiment with a binary. This can often be time-consuming if the focus is on checking dependencies between program variables or for verifying the dataflow, or worse, may be impossible for some platforms that are difficult to dynamically analyze. With UIDF, we aim to provide our users with the ability to interact with the dataflow engine more richly and partly achieve the same results without leaving the flow graph or linear analysis window. UIDF could also aid in better understanding targets of indirect jumps or fixing jump tables. Research [[1](https://acmccs.github.io/papers/p347-shoshitaishviliA.pdf),[2](https://dl.acm.org/doi/fullHtml/10.1145/3290607.3313040)] shows that it is imperative that binary analysis tools adopt a more user-guided approach to solve RE/VR tasks and UIDF tries to be a step up that slope. We aim to continue work in this direction! -UIDF primarily operates on the MLIL layer. Binary Ninja performs constant propagation, resolves call parameters and lifts stack load/stores into a variables. However, the UI can apply a UIDF constraint on an HLIL variable or MLIL variable and apply it appropriately. This combined with the SSA form enables an effective dataflow analysis pipeline. MLIL also provides control over dead-code elimination and allows us to prevent removal of redundant instructions (eg. variable definitions for informed variables) and basic blocks (removed due to the resulting dataflow). Binary Ninja's tiered IL representation allows the higher layers to benefit from simplifying transformations at the lower layers. Through fixing jump tables or branch elimination, UIDF influences HLIL output as a user would expect without having drastic changes to the MLIL representation. +UIDF primarily operates on the MLIL layer. Binary Ninja performs constant propagation, resolves call parameters, and lifts stack load/stores into variables. However, the UI can apply a UIDF constraint on an HLIL variable or MLIL variable and apply it appropriately. This combined with the SSA form enables an effective dataflow analysis pipeline. MLIL also provides control over dead-code elimination and allows us to prevent removal of redundant instructions (e.g. variable definitions for informed variables) and basic blocks (removed due to the resulting dataflow). Binary Ninja's tiered IL representation allows the higher layers to benefit from simplifying transformations at the lower layers. Through fixing jump tables or branch elimination, UIDF influences HLIL output as a user would expect without having drastic changes to the MLIL representation. Looking towards the future, improvements to the dataflow solver engine would inherently increase the efficacy of UIDF. This could include support for more operations on the complex [`PossibleValueSet`](https://api.binary.ninja/binaryninja.variable-module.html#binaryninja.variable.PossibleValueSet) containers or the ability to play well with program structure such as loops. diff --git a/docs/dev/workflows.md b/docs/dev/workflows.md index d923dd05..61d6b77f 100644 --- a/docs/dev/workflows.md +++ b/docs/dev/workflows.md @@ -608,8 +608,8 @@ current_function.workflow.show_topology() These commands build upon the static view by incorporating real-time dynamic state information. It provides additional insights such as the current eligibility of activities, override states, and other runtime data. This enhanced view offers a more detailed and interactive perspective, making it invaluable for monitoring and debugging during execution. !!! note - Using the dynamic view, the eligibility of activities can be toggled interactively. Currently, this behavior is tied to the double-click event on an activity node. This feature allows you to quickly cycle through different eligibility states, enabling rapid testing and validation of activity configurations. + Using the dynamic view, the eligibility of activities can be toggled interactively. Currently, this behavior is tied to the double click event on an activity node. This feature allows you to quickly cycle through different eligibility states, enabling rapid testing and validation of activity configurations. ## Advanced Topics (Coming Soon) -This section covers advanced topics related to Workflows, Activities, and the Workflow Machine. It explores emerging concepts, best practices, and strategies for optimizing and extending the analysis pipeline.
\ No newline at end of file +This section covers advanced topics related to Workflows, Activities, and the Workflow Machine. It explores emerging concepts, best practices, and strategies for optimizing and extending the analysis pipeline. |