documentation overhaul

8 files changed, 109 insertions, 20 deletions

diff --git a/docs/dev/batch.md b/docs/dev/batch.md
index 679df21a..0f30749f 100644
--- a/docs/dev/batch.md
+++ b/docs/dev/batch.md
@@ -6,7 +6,7 @@ This document describes some general tips and tricks for effective batch process
 
 ## Install the API
 
-First, make sure to run the [install_api.py](https://github.com/Vector35/binaryninja-api/tree/dev/scripts) script. Note that the script is shipped with Binary Ninja already, just look in your [binary path](../getting-started.md#binary-path) inside of the `scripts` subfolder. Run it like:
+First, make sure to run the [install_api.py](https://github.com/Vector35/binaryninja-api/tree/dev/scripts) script. Note that the script is shipped with Binary Ninja already, just look in your [binary path](../guide/#binary-path) inside of the `scripts` subfolder. Run it like:
 
 ```
 python3 ~/binaryninja/scripts/install_api.py
diff --git a/docs/dev/concepts.md b/docs/dev/concepts.md
new file mode 100644
index 00000000..eae79854
--- /dev/null
+++ b/docs/dev/concepts.md
@@ -0,0 +1,23 @@
+# Important Concepts
+
+## Mapping between ILs
+
+ILs in general are critical to how Binary Ninja analyzes binaries and we have much more [in-depth](bnil-overview.md) documentation for BNIL (or Binary Ninja Intermediate Language -- the name given to the family of ILs that Binary Ninja uses). However, one important concept to summarize here is that the translation between each layer of IL is many-to-many. Going from disassembly to LLIL to MLIL can result in more or less instructions at each step. Additionally, at higher levels, data can be copied, moved around, etc. You can see this in action in the UI when you select a line of HLIL and many LLIL or disassembly instructions are highlighted.
+
+APIs that query these mappings are plural. So for example, while `current_hlil.llil` will give a single mapping, `current_hlil.llils` will return a list that may contain multiple mappings.
+
+![Mapping between ILs ><](/img/ilmapping.png "Mapping between ILs")
+
+## Operating on IL versus Native
+
+Generally speaking, scripts should operate on ILs. The available information far surpasses the native addresses and querying properties and using APIs almost always beats directly manipulating bytes. However, when it comes time to change the binary, there are some operations that can only be done at a simple virtual address. So for example, the [comment](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.set_comment_at) or [tag](https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.add_user_data_tag) APIs (among others) work off of native addressing irrespective of IL level.
+
+## Instruction Index vs Expression Index
+
+It is easy to confuse ExpressionIndex and InstructionIndex properties in the API. While they [are both integers](https://github.com/Vector35/binaryninja-api/blob/dev/python/highlevelil.py#L49-L50) they mean different things and it's important to keep them straight. The Instruction Index is a unique index for that given IL level for that given function. However, because BNIL is [tree-based](bnil-overview.md), when there are nested expresses the expression index may be needed. These indexes are also unique per-function and per-IL level, but they are _distinct_ from instruction indexes even though they may occasionally be similar since they each start at 0 for a given function!
+
+## UI Elements
+
+There are several ways to create UI elements in Binary Ninja. The first is to use the simplified [interaction](https://api.binary.ninja/binaryninja.interaction-module.html) API which lets you make simple UI elements for use in GUI plugins in Binary Ninja. As an added bonus, they all have fallbacks that will work in headless console-based applications as well. Plugins that use these API include the [angr](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/angr_plugin.py) and [nampa](https://github.com/kenoph/nampa) plugins.
+
+The second and more powerful (but more complicated) mechanism is to leverage the _binaryninjaui_ module. Additional documentation is forthcoming, but there are several examples ([1](https://github.com/Vector35/kaitai), [2](https://github.com/Vector35/snippets), [3](https://github.com/Vector35/binaryninja-api/tree/dev/python/examples/triage)), and most of the APIs are backed by the [documented C++ headers](https://api.binary.ninja/cpp). Additionally, the generated _binaryninjaui_ module is shipped with each build of binaryninja and the usual python `dir()` instructions are helpful for exploring its capabilities.
\ No newline at end of file
diff --git a/docs/dev/cookbook.md b/docs/dev/cookbook.md
new file mode 100644
index 00000000..465a5ebe
--- /dev/null
+++ b/docs/dev/cookbook.md
@@ -0,0 +1,68 @@
+# Cookbook
+
+One of the best ways to learn a complicated API is to simply find the right example! First, here's a huge number of other sources of larger programs if you don't find what you're looking for in the below simple examples:
+
+ - [Official Plugins](https://github.com/vector35/official-plugins): Official Plugins written and maintained by Vector 35
+ - [Community Plugins](https://github.com/vector35/community-plugins): Over 100 plugins contributed by the Binary Ninja community
+ - [Gist Collection](https://gist.github.com/psifertex/6fbc7532f536775194edd26290892ef7): Jordan's collection of python examples usually created for (or contributed by) customers
+ - [Offline examples](https://github.com/Vector35/binaryninja-api/tree/dev/python/examples): These examples are especially useful because they're included in your offline install as well, just look in the examples/python subfolder wherever Binary Ninja installed
+
+ That said, most of those examples tend to be more complex and so the following recipes are meant to be simple but useful building-blocks with which to learn useful techniques:
+
+## Recipies
+
+### Accessing cross references
+
+This recipe is useful for iterating over all of the HLIL cross-references of a given interesting function:
+
+```python
+for ref in current_function.caller_sites:
+	print(ref.hlil)
+```
+
+But what if you don't have that function yet?
+
+### Getting a function by name
+
+```python
+bv.get_functions_by_name('_start')
+```
+
+### Finding the function with the most bytes
+
+```python
+max(bv.functions, key=lambda x: x.total_bytes)
+```
+
+### Finding the most "connected" function
+
+As defined by having the highest sum of incoming and outgoing calls. Adjust accordingly.
+
+```python
+max(bv.functions, key=lambda x: len(x.callers + x.callees))
+```
+
+### Querying possible values of a function parameter
+
+Is that memcpy length a bit too big?
+
+```python
+for ref in current_function.caller_sites:
+	if isinstance(ref.hlil, Call) and len(ref.hlil.params) >= 3:
+		print(ref.hlil.params[2])
+		# For bonus points, query the range analysis
+```
+
+### Search for a good nop-slide?
+
+```python
+bv.find_next_data(0, b"\x90" * 10)
+```
+
+### Change a function's type signature
+
+Make sure to check out the much more in-depth [type guide](../guide/type.md#using-the-api) as well.
+
+```python
+current_function.function_type = Type.function(Type.void(), [])
+```
diff --git a/docs/dev/documentation.md b/docs/dev/documentation.md
index 16a1c9ee..b524d7fb 100644
--- a/docs/dev/documentation.md
+++ b/docs/dev/documentation.md
@@ -26,7 +26,7 @@ To contribute to the Binary Ninja documentation, first sign the [contribution li
 ## Changing
 Changing documentation for the API itself is fairly straight forward. Use [doxygen style comment blocks](https://www.stack.nl/~dimitri/doxygen/manual/docblocks.html) in C++ and C, and [restructured text blocks](http://thomas-cokelaer.info/tutorials/sphinx/docstring_python.html) for python for the source. The user documentation is located in the `api/docs/` folder and the API documentation is generated from the config in the `api/api-docs` folder.
 
-!!! Tip "Tip"
+???+ Warning "Tip"
     When updating user documentation, the `mkdocs serve` feature is particularly helpful.
 
 [contribution license agreement]: https://binary.ninja/cla.pdf
diff --git a/docs/dev/api.md b/docs/dev/index.md
index 8cc01543..d379de0d 100644
--- a/docs/dev/api.md
+++ b/docs/dev/index.md
@@ -1,9 +1,18 @@
 # Using the Binary Ninja API
 
+Welcome to the Binary Ninja API documentation. Much like the [User Manual](/guide/), some larger sections have been split off into their own sections on the left, while the table of contents for this documentation is on the right.
+
 ## Language Specific Bindings
 
 The Binary Ninja API is available through a [Core API](#core-api), through the [C++ API](#c-api), through a [Python API](#python-api), and a [Rust API](#rust-api).
 
+### Python API
+
+The most heavily documented of all of the APIs, the Python API serves as a useful documentation for the other APIs. Here's a list of the most important Python API documentation resources:
+
+ - [Writing Python Plugins](plugins.md)
+ - [Python API](https://api.binary.ninja/)
+ - [API Source](https://github.com/Vector35/binaryninja-api/tree/dev/python)
 
 ### Core API
 
@@ -19,24 +28,9 @@ The C++ API is what the Binary Ninja UI itself is built using so it's a robust a
  - [Build Instructions](https://github.com/Vector35/binaryninja-api#building)
  - [C++ / UI API Documentation](https://api.binary.ninja/cpp/)
 
-
-### Python API
-
-The most heavily documented of all of the APIs, the Python API serves as a useful documentation for the other APIs. Here's a list of the most important Python API documentation resources:
-
- - [Writing Python Plugins](plugins.md)
- - [Python API](https://api.binary.ninja/)
- - [API Source](https://github.com/Vector35/binaryninja-api/tree/dev/python)
-
 ### Rust API
 
-The Rust API is still experimental and lacks complete coverage for all core APIs. Instructions on using are available in:
+The Rust API is still experimental and lacks complete coverage for all core APIs. Documentation is available at:
 
  - [Rust API](https://github.com/Vector35/binaryninja-api/tree/dev/rust)
 
-
-## UI Elements
-
-There are several ways to create UI elements in Binary Ninja. The first is to use the simplified [interaction](https://api.binary.ninja/binaryninja.interaction-module.html) API which lets you make simple UI elements for use in GUI plugins in Binary Ninja. As an added bonus, they all have fallbacks that will work in headless console-based applications as well. Plugins that use these API include the [angr](https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/angr_plugin.py) and [nampa](https://github.com/kenoph/nampa) plugins.
-
-The second and more powerful (but more complicated) mechanism is to leverage the _binaryninjaui_ module. Additional documentation is forthcoming, but there are several examples ([1](https://github.com/Vector35/kaitai), [2](https://github.com/Vector35/snippets), [3](https://github.com/Vector35/binaryninja-api/tree/dev/python/examples/triage)), and most of the APIs are backed by the [documented C++ headers](https://api.binary.ninja/cpp). Additionally, the generated _binaryninjaui_ module is shipped with each build of binaryninja and the usual python `dir()` instructions are helpful for exploring its capabilities.
diff --git a/docs/dev/plugins.md b/docs/dev/plugins.md
index e1839fc4..981cb148 100644
--- a/docs/dev/plugins.md
+++ b/docs/dev/plugins.md
@@ -29,7 +29,7 @@ Once you've created your test repository, use the `pluginManager.unofficialName`
 The [`add_repository`](https://api.binary.ninja/binaryninja.pluginmanager-module.html#binaryninja.pluginmanager.RepositoryManager.add_repository) API can also be used to add the repository, though it [may require manual creation of the repository folder](https://github.com/Vector35/binaryninja-api/issues/2987).
 ## Testing
 
-It's useful to be able to reload your plugin during testing. On the Commercial edition of Binary Ninja, this is easily accomplished with a stand-alone headless install using `import binaryninja` after [installing the API](https://github.com/Vector35/binaryninja-api/blob/dev/scripts/install_api.py).  (install_api.py is included in each platforms respective [installation folder](../getting-started.md#binary-path))
+It's useful to be able to reload your plugin during testing. On the Commercial edition of Binary Ninja, this is easily accomplished with a stand-alone headless install using `import binaryninja` after [installing the API](https://github.com/Vector35/binaryninja-api/blob/dev/scripts/install_api.py).  (install_api.py is included in each platforms respective [installation folder](../guide/#binary-path))
 
 For other plugins, we recommend the following workflow from the scripting console which enables easy iteration and testing:
 
diff --git a/docs/dev/themes.md b/docs/dev/themes.md
index 37fedfc7..6c97ff6b 100644
--- a/docs/dev/themes.md
+++ b/docs/dev/themes.md
@@ -2,7 +2,7 @@
 
 User themes are loaded from JSON files (with the `.bntheme` extension) found in
 the `themes` or `community-themes` subdirectories of your [user
-folder](/getting-started.md#user-folder). The full path to these folders
+folder](../guide/index.md#user-folder). The full path to these folders
 effectively being the following:
 
 - macOS: `~/Library/Application Support/Binary Ninja/{themes,community-themes}`
diff --git a/docs/dev/workflows.md b/docs/dev/workflows.md
index 17e190dc..106f7087 100644
--- a/docs/dev/workflows.md
+++ b/docs/dev/workflows.md
@@ -110,6 +110,8 @@ A Workflow starts in the unregistered state from either creating a new empty Wor
 
 The Analysis Context provides access to the current analysis hierarchy along with APIs to query and inform new analysis information.
 
+<! ---
+
 ---
 # Execution Model
 ---
@@ -170,3 +172,5 @@ The Analysis Context provides access to the current analysis hierarchy along wit
 ---
 # Extended Analysis Descriptions
 ---
+
+--- >
\ No newline at end of file