diff options
| author | Josh Ferrell <josh@vector35.com> | 2025-09-11 18:29:59 -0400 |
|---|---|---|
| committer | Josh Ferrell <josh@vector35.com> | 2025-09-11 19:09:28 -0400 |
| commit | 5f21429815bfa13bc167aff4a9785424675b82b6 (patch) | |
| tree | d13caac55b8ad269062929413e286bab44ad8429 /docs/dev/concepts.md | |
| parent | 2d20ed145b9135a7d3390acb3ac10a900d947f91 (diff) | |
Docs grammar and spelling fixes
Diffstat (limited to 'docs/dev/concepts.md')
| -rw-r--r-- | docs/dev/concepts.md | 48 |
1 files changed, 24 insertions, 24 deletions
diff --git a/docs/dev/concepts.md b/docs/dev/concepts.md index 376f485b..8e70dba7 100644 --- a/docs/dev/concepts.md +++ b/docs/dev/concepts.md @@ -16,7 +16,7 @@ When you are interacting in the UI with an executable file, you can access `bv` ???+ Info "Tip" Note the use of `bv` here as a shortcut to the currently open BinaryView. For other "magic" variables, see the [user guide](../guide/index.md#magic-console-variables) -If you want to start writing a plugin, most top-level methods will exist off of the BinaryView. Conceptually, you can think about the organization as a hierarchy starting with a BinaryView, then functions, then basic blocks, then instructions. There are of course lots of other ways to access parts of the binary but this is the most common organization. Check out the tab completion in the scripting console for `bv.get<TAB>` for example (a common prefix for many APIs): +If you want to start writing a plugin, most top-level methods will exist off of the BinaryView. Conceptually, you can think about the organization as a hierarchy starting with a BinaryView, then functions, then basic blocks, then instructions. There are of course lots of other ways to access parts of the binary, but this is the most common organization. Check out the tab completion in the scripting console for `bv.get<TAB>` for example (a common prefix for many APIs):  @@ -24,7 +24,7 @@ Some BinaryViews have parent views. The view used for decompilation includes mem ## REPL versus Scripts -When you're interacting with the Binary Ninja [scripting console](../guide/index.md#script-python-console), it's important to realize that every time you run a command, the UI is automatically going to update analysis. You can see this by even running a simply command like: +When you're interacting with the Binary Ninja [scripting console](../guide/index.md#script-python-console), it's important to realize that every time you run a command, the UI is automatically going to update analysis. You can see this by even running a simple command like: ``` print("test") @@ -36,17 +36,17 @@ To achieve the same results when writing a stand-alone plugin, you'd need to use Another difference in the REPL / scripting console is that the scripting console is not executing on the main thread. This means that if you wish to interact with the UI via QT or BN UI APIs from the console (for example, to [trigger an action via string](https://gist.github.com/psifertex/6fbc7532f536775194edd26290892ef7#file-trigger_actions-py)), you'd need to use [`mainthread.execute_on_main_thread_and_wait()`](https://api.binary.ninja/binaryninja.mainthread-module.html#binaryninja.mainthread.execute_on_main_thread_and_wait) or similar. - + ## Auto vs User -In the Binary Ninja API, there are often two parallel sets of functions with `_auto_` and `_user_` in their names. For example: [add_user_segment](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.add_user_segment) and [add_auto_segment](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.add_auto_segment). So what's the difference? Auto functions are those that are expected to be run _automatically_, every time the file is loaded. So for example, if you're writing a [custom file loader](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/nsf.py) that will parse a particular binary format, you would use `_auto_` functions because each time the file is opened your loader will be used. The results of auto functions are saved in a `.bndb` database, but will be cleared if re-running auto-analysis (which happens when updating the version of the database due to updating Binary Ninja in some instances). This is because it's expected that whatever produced them originally will again when the file is re-analyzed (you can use the [`analysis.database.suppressReanalysis`](../guide/settings.md#analysis.database.suppressReanalysis) setting to avoid this, but it's generally discouraged). This means that even if you're writing a plugin to make changes on a file during analysis, you likely want to use the `_user_` set of APIs so that the changes your plugin causes will separately be saved to the database as if they were done by a user. User actions are also added to the undo serialization and can be undone by the user which is not true of auto actions. +In the Binary Ninja API, there are often two parallel sets of functions with `_auto_` and `_user_` in their names. For example: [add_user_segment](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.add_user_segment) and [add_auto_segment](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.add_auto_segment). So what's the difference? Auto functions are those that are expected to be run _automatically_, every time the file is loaded. So for example, if you're writing a [custom file loader](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/nsf.py) that will parse a particular binary format, you would use `_auto_` functions because each time the file is opened your loader will be used. The results of auto functions are saved in a `.bndb` database, but will be cleared if re-running auto-analysis (which happens when updating the version of the database due to updating Binary Ninja in some instances). This is because it's expected that whatever produced them originally will again when the file is re-analyzed (you can use the [`analysis.database.suppressReanalysis`](../guide/settings.md#analysis.database.suppressReanalysis) setting to avoid this, but it's generally discouraged). This means that even if you're writing a plugin to make changes on a file during analysis, you likely want to use the `_user_` set of APIs so that the changes your plugin causes will separately be saved to the database as if they were done by a user. User actions are also added to the undo serialization and can be undone by the user which is not true of auto actions. ## Concepts for ILs ### Walking ILs -Because our ILs are tree-based, some naive plugins that walk IL looking for specific operations will miss instructions that are part of nested expressions. While there is generally less folding in [MLIL](https://docs.binary.ninja/dev/bnil-mlil.html), making it better target for simple loops like: +Because our ILs are tree-based, some naive plugins that walk IL looking for specific operations will miss instructions that are part of nested expressions. While there is generally less folding in [MLIL](https://docs.binary.ninja/dev/bnil-mlil.html), making it the better target for simple loops like: ```python for i in bv.mlil_instructions: @@ -54,7 +54,7 @@ for i in bv.mlil_instructions: print(i.params) ``` -it's still possible to miss instructions with this approach. Additionally, in HLIL it's even easier to miss instructions as there is significantly more nesting. Accordingly we created [`traverse`](https://api.binary.ninja/binaryninja.highlevelil-module.html#binaryninja.highlevelil.HighLevelILFunction.traverse) APIs to walk these tree-based ILs and match whatever property you're interested in. Here are several examples: +it's still possible to miss instructions with this approach. Additionally, in HLIL it's even easier to miss instructions as there is significantly more nesting. Accordingly, we created [`traverse`](https://api.binary.ninja/binaryninja.highlevelil-module.html#binaryninja.highlevelil.HighLevelILFunction.traverse) APIs to walk these tree-based ILs and match whatever property you're interested in. Here are several examples: ```python def find_strcpy(i, t) -> str: @@ -70,7 +70,7 @@ t = [ # Find the first call to a builtin: for result in current_hlil.traverse(find_strcpy, t): # Any logic should live here, not inside the callable which is just for - # matching. Because this is a generator, it can fail fast when used for + # matching. Because this is a generator, it can fail fast when used for # search! print(result) break @@ -153,11 +153,11 @@ It is easy to confuse ExpressionIndex and InstructionIndex properties in the API Our [BNIL Overview](bnil-overview.md) mentions Static Single Assignment (SSA) without explaining what it is. You can of course always check out [Wikipedia](https://en.wikipedia.org/wiki/Static_single-assignment_form), but here's a quick summary of what it is, why we expose multiple SSA forms of our ILs and how you can use it to improve your plugin writing with BNIL. -At it's simplest, SSA forms of an Intermediate Language or Intermediate Representation are those in which variables are read only. They can be set when created, but not modified. Instead of modifying them, when you make a change, a new "version" of the variable is created denoting that the value has changed in some way. While this certainly makes the form less readable to humans, this means it's actually really easy to walk back and see how a variable has been modified by simply looking at the definition site for a given variable or all possible versions before it. Any time the variable is modified, a new version of that variable will be created (you'll see `#` followed by a number indicating that this is a new version). +At it's simplest, SSA forms of an Intermediate Language or Intermediate Representation are those in which variables are read only. They can be set when created, but not modified. Instead of modifying them, when you make a change, a new "version" of the variable is created denoting that the value has changed in some way. While this certainly makes the form less readable to humans, this means it's actually really easy to walk back and see how a variable has been modified by simply looking at the definition site for a given variable or all possible versions before it. Any time the variable is modified, a new version of that variable will be created (you'll see `#` followed by a number indicating that this is a new version). But what if the code takes multiple paths and could have different values depending on the path that was taken? SSA forms use a `Φ` (pronounced either as "fee" or "fi" like "fly" depending on which mathematician or Greek speaker you ask!) function to solve this ambiguity. All a `Φ` function does is aggregate different versions of a variable when entering a basic block with multiple paths. So a basic block with two incoming edges where `eax` is modified might have something like: `eax#3 = Φ(eax#1, eax#2)` denoting that the new version of the variable could have come from either of those sources. -Binary Ninja uses this capability internally for its own value set analysis and constant dataflow propagation but plugins can also leverage this information to great effect. For example, want to find an uninitialized value? Simply look for an SSA variable being read from with a version of zero that isn't in the list of arguments to the function. Want to implement your own inter-procedural data-flow system? Binary Ninja does not for performance reasons, but in instances where you can prevent the state space explosion problem, you can build on top of the existing SSA forms to implement exactly this. A simple example might look for vulnerable function calls like printf() where the first argument is user-data. While most trivial cases of this type of flaw tend to be found quickly, it's often the case that subtler versions with functions that wrap functions that wrap functions that call a printf with user data are more tedious to identify. However, using an SSA-based script, it's super easy to see that, for example, the first parameter to a `printf` call originated in a calling function as the second parameter, and THAT function was called with input that came directly from some sort of user input. While one or two layers might be easy to check by hand with few cross-references, with a large embedded firmware, there might be hundreds or thousands of potential locations to check out, and a script using SSA can dramatically reduce the number of cases to investigate. +Binary Ninja uses this capability internally for its own value set analysis and constant dataflow propagation but plugins can also leverage this information to great effect. For example, want to find an uninitialized value? Simply look for an SSA variable being read from with a version of zero that isn't in the list of arguments to the function. Want to implement your own inter-procedural data-flow system? Binary Ninja does not for performance reasons, but in instances where you can prevent the state space explosion problem, you can build on top of the existing SSA forms to implement exactly this. A simple example might look for vulnerable function calls like `printf()` where the first argument is user-data. While most trivial cases of this type of flaw tend to be found quickly, it's often the case that subtler versions with functions that wrap functions that wrap functions that call a `printf` with user data are more tedious to identify. However, using an SSA-based script, it's super easy to see that, for example, the first parameter to a `printf` call originated in a calling function as the second parameter, and THAT function was called with input that came directly from some sort of user input. While one or two layers might be easy to check by hand with few cross-references, with a large embedded firmware, there might be hundreds or thousands of potential locations to check out, and a script using SSA can dramatically reduce the number of cases to investigate. ### When IL APIs Return None @@ -195,11 +195,11 @@ Binary Ninja utilizes two distinct data flow systems that are influenced by memo The distinction between readable and writable values affects how data flow analysis is performed: - **Readable Values:** Variables marked as readable are analyzed primarily for the flow of data without modification, aiding in tracking data dependencies. -- **Writable Values:** Writable variables are treated as mutable and thus no assumptions are made about the values being the same in a given function. +- **Writable Values:** Writable variables are treated as mutable and thus no assumptions are made about the values being the same in a given function. By overriding variable annotations or memory flags, you can alter Binary Ninja's assumptions about data: -- Marking a writable variable as **`const`** forces the analysis to treat its value as immutable, potentially simplifying the data flow but risking misinterpretation if the assumption is incorrect. +- Marking a writable variable as **`const`** forces the analysis to treat its value as immutable, potentially simplifying the data flow but risking misinterpretation if the assumption is incorrect. - Marking a constant variable as **`volatile`** forces the analysis to treat its value as mutable, safeguarding against data flow making assumptions about the value, this is particularly useful when dealing with data variables in sections marked with [ReadOnlyDataSectionSemantics](https://api.binary.ninja/binaryninja.enums-module.html#binaryninja.enums.SectionSemantics). ## Permissions Impact on Linear Sweep @@ -214,7 +214,7 @@ By modifying memory permissions, you can guide Binary Ninja's linear sweep analy ## UI Elements -There are several ways to create UI elements in Binary Ninja. The first is to use the simplified [interaction](https://api.binary.ninja/binaryninja.interaction-module.html) API which lets you make simple UI elements for use in GUI plugins in Binary Ninja. As an added bonus, they all have fallbacks that will work in headless console-based applications as well. Plugins that use these API include the [angr](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/angr_plugin.py) and [nampa](https://github.com/kenoph/nampa) plugins. +There are several ways to create UI elements in Binary Ninja. The first is to use the simplified [interaction](https://api.binary.ninja/binaryninja.interaction-module.html) API which lets you make simple UI elements for use in GUI plugins in Binary Ninja. As an added bonus, they all have fallbacks that will work in headless console-based applications as well. Plugins that use these APIs include the [angr](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/angr_plugin.py) and [nampa](https://github.com/kenoph/nampa) plugins. The second and more powerful (but more complicated) mechanism is to leverage the _binaryninjaui_ module. Additional documentation is forthcoming, but there are several examples ([1](https://github.com/Vector35/kaitai), [2](https://github.com/Vector35/snippets), [3](https://github.com/Vector35/binaryninja-api/tree/dev/python/examples/triage)), and most of the APIs are backed by the [documented C++ headers](https://api.binary.ninja/cpp). Additionally, the generated _binaryninjaui_ module is shipped with each build of binaryninja and the usual python `dir()` instructions are helpful for exploring its capabilities. @@ -226,21 +226,21 @@ When calling native UI methods, please make sure to use the [execute_on_main_thr One of the common questions asked of a binary analysis platform is "how big is a function?". This is a deceptively simple question without a simple answer. There are rather several equally valid definitions you could give for what is the size of a function: - - the total sum of all basic blocks? - - the highest virtual address in the function minus the lowest virtual address? - - the address of return instruction subtracted from the entry point + - The total sum of all basic blocks? + - The highest virtual address in the function minus the lowest virtual address? + - The address of return instruction subtracted from the entry point -Except that last one is a trick of course. Because not only can functions have multiple return instructions, but they may have multiple entry points (as is often the case with error handling). +Except that last one is a trick of course. Because not only can functions have multiple return instructions, but they may have multiple entry points (as is often the case with error handling). -Basic blocks have, by definition, a start, and an end. Basic Blocks can therefore have consistent sizes that all binary analysis tools would agree upon (though more formal analysis might stop basic blocks on call instructions while for convenience sake, most reverse engineering tools do not). +Basic blocks have, by definition, a start, and an end. Basic Blocks can therefore have consistent sizes that all binary analysis tools would agree upon (though more formal analysis might stop basic blocks on call instructions while for convenience's sake, most reverse engineering tools do not). -Summing up the basic blocks of a function is one way to produce a consistent size for a function, but how do you handle bytes that overlap standard function definitions, for example, via a tail call? Or via a mis-aligned jump where a byte is in two basic blocks? Different tools may resolve those ambiguous situations in different ways, so again, it is difficult to compare the "size" of any one binary analysis tool to another. +Summing up the basic blocks of a function is one way to produce a consistent size for a function, but how do you handle bytes that overlap standard function definitions, for example, via a tail call? Or via a misaligned jump where a byte is in two basic blocks? Different tools may resolve those ambiguous situations in different ways, so again, it is difficult to compare the "size" of any one binary analysis tool to another. In Binary Ninja, there is no explicit `.size` property of functions. Rather, you can choose to calculate it one of two ways: ``` function_size = current_function.total_bytes -# or +# or function_size = current_function.highest_address - current_function.lowest_address ``` @@ -248,14 +248,14 @@ Total bytes is similar to the first proposed definition above. It merely sums up ### When does a Function stop? -One reason that having an "end" might be useful in a function (as opposed to the `.highest_address` in Binary Ninja), would be to make it a property that controls the analysis for a function. Unlike some other systems where it's possible to define the start and end of a function, in Binary Ninja, you merely define the start and allow analysis to occur naturally. The end results when all basic blocks terminate either in: +One reason that having an "end" might be useful in a function (as opposed to the `.highest_address` in Binary Ninja), would be to make it a property that controls the analysis for a function. Unlike some other systems where it's possible to define the start and end of a function, in Binary Ninja, you merely define the start and allow analysis to occur naturally. The end results when all basic blocks terminate either in: - an invalid instruction - a return instruction - a call to a function marked as `__noreturn__` - a branch to a block already in the function - any other instruction such as an interrupt that by its definition stops analysis - + So how do you tell Binary Ninja how big a function is? The answer is you don't directly, but you can instead direct its analysis. For example, if a function is improperly not marked as a noreturn function, edit the function properties via the right-click menu and set the property and all calls to it will end analysis at that point in any callees. Likewise, you can do things like change memory permissions, patch in invalid instructions, [change an indirect branch's targets](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/jump_table.py), or use [UIDF](https://binary.ninja/2020/09/10/user-informed-dataflow.html) to influence analysis such that it ends where desired. @@ -264,7 +264,7 @@ Likewise, you can do things like change memory permissions, patch in invalid ins ### Unicode Support -If you're opening a file with non-ascii string encodings, make sure to use one of the "open with options" [methods](../guide/index.md#loading-files) and choose the appropriate code pages. These settings are under the `analysis` / `unicode` section of the settings dialog. If you're not sure which code page you should be enabling, consider using a library like [chardet](https://chardet.readthedocs.io/en/latest/usage.html). For example, here's a file both with and without the "CJK Unified Ideographs" block being enabled: +If you're opening a file with non-ASCII string encodings, make sure to use one of the "open with options" [methods](../guide/index.md#loading-files) and choose the appropriate code pages. These settings are under the `analysis` / `unicode` section of the settings dialog. If you're not sure which code page you should be enabling, consider using a library like [chardet](https://chardet.readthedocs.io/en/latest/usage.html). For example, here's a file both with and without the "CJK Unified Ideographs" block being enabled:  @@ -272,10 +272,10 @@ If you're opening a file with non-ascii string encodings, make sure to use one o ???+ Info "Tip" Note that there is a small bug and some string length calculations will be off with certain characters as shown above. - + ### Wide Strings in the UI -Many Windows applications use wide (UTF-16 or UTF-32) strings. By default these should be identified during initial string analysis as long as the default settings are still enabled. +Many Windows applications use wide (UTF-16 or UTF-32) strings. By default, these should be identified during initial string analysis as long as the default settings are still enabled.  |
