Merge branch 'dev'

6 files changed, 226 insertions, 15 deletions

diff --git a/docs/about/open-source.md b/docs/about/open-source.md
index 7feb1b7a..9f385633 100644
--- a/docs/about/open-source.md
+++ b/docs/about/open-source.md
@@ -24,14 +24,17 @@ The previous tools are used in the generation of our documentation, but are not
     - [discount] ([discount license] - BSD)
     - [libcurl] ([libcurl license] - MIT/X derivative)
     - [libgit2] ([libgit2 license] - GPLv2 with linking exception)
+    - [libmspack] ([libmspack license] - LGPL, v2)
     - [llvm] ([llvm license] - BSD-style)
     - [lzf] ([lzf license] - BSD)
+    - [jemalloc] ([jemalloc license] - BSD)
     - [openssl] ([openssl license] - openssl license)
     - [sqlite] ([sqlite license] - public domain)
     - [zlib] ([zlib license] - zlib license)
 
 * Other
-    - [yasm] ([yasm license] - 2-clause BSD)
+    - [yasm] ([yasm license] - 2-clause BSD) used for assembling x86 and x64
+    - [capstone] ([capstone license] - 3-clause BSD) used in the PPC architecture module as an example of how to wrap an external disassembler
 
 * Upvector update library
     - [tomcrypt] ([tomcrypt license] - public domain)
@@ -54,6 +57,8 @@ Please note that we offer no support for running Binary Ninja with modified Qt l
 
 [Building Qt 5 from Git]: https://wiki.qt.io/Building-Qt-5-from-Git
 [Qt 5.6]: https://www.qt.io/qt-licensing-terms/
+[capstone]: https://github.com/aquynh/capstone
+[capstone license]: https://github.com/aquynh/capstone/blob/master/LICENSE.TXT
 [breathe license]: https://github.com/michaeljones/breathe/blob/master/LICENSE
 [breathe-rtd-theme license]: https://github.com/snide/sphinx_rtd_theme/blob/master/LICENSE
 [breathe-rtd-theme]: https://github.com/snide/sphinx_rtd_theme/
@@ -64,12 +69,16 @@ Please note that we offer no support for running Binary Ninja with modified Qt l
 [discount]: http://www.pell.portland.or.us/~orc/Code/discount/
 [doxygen license]: https://github.com/doxygen/doxygen/blob/master/LICENSE
 [doxygen]: http://www.stack.nl/~dimitri/doxygen/
-[libcurl license]: https://curl.haxx.se/docs/copyright.html 
 [libcurl]: https://curl.haxx.se/
-[libgit2 license]: https://github.com/libgit2/libgit2/blob/master/COPYING
+[libcurl license]: https://curl.haxx.se/docs/copyright.html 
 [libgit2]: https://libgit2.github.com/
-[llvm license]: http://llvm.org/releases/3.8.1/LICENSE.TXT
+[libgit2 license]: https://github.com/libgit2/libgit2/blob/master/COPYING
+[libmspack]: https://www.cabextract.org.uk/libmspack/
+[libmspack license]: https://www.cabextract.org.uk/libmspack/#license
 [llvm]: http://llvm.org/releases/3.8.1/
+[llvm license]: http://llvm.org/releases/3.8.1/LICENSE.TXT
+[jemalloc]: http://jemalloc.net/
+[jemalloc license]: https://github.com/jemalloc/jemalloc/blob/dev/COPYING
 [lzf license]: http://oldhome.schmorp.de/marc/liblzf.html
 [lzf]: http://oldhome.schmorp.de/marc/liblzf.html
 [mkdocs license]: https://github.com/mkdocs/mkdocs/blob/master/LICENSE
diff --git a/docs/docs.css b/docs/docs.css
index e06fa460..ebd56907 100644
--- a/docs/docs.css
+++ b/docs/docs.css
@@ -4,7 +4,12 @@ code {
 
 .admonition {
 	background: rgb(128, 198, 223);
-	color: #333;
+	color: #fff;
+}
+
+.tip {
+	background: rgb(110, 110, 110);
+	color: #fff;
 }
 
 img[alt$=">"] {
diff --git a/docs/getting-started.md b/docs/getting-started.md
index 27188dae..d91eb046 100644
--- a/docs/getting-started.md
+++ b/docs/getting-started.md
@@ -2,18 +2,45 @@
 
 Welcome to Binary Ninja. This introduction document is meant to quickly guide you over some of the most common uses of Binary Ninja.
 
-![license popup >](/images/license-popup.png "License Popup")
+## Directories
 
-## License
+Binary Ninja uses two main locations. The first is the install path of the binary itself and the second is the user folders for user-installed content.
 
-When you first run Binary Ninja, it will prompt you for your license key. You should have received your license key via email after your purchase. If not, please contact [support].
+### Binary Path
+
+Binaries are installed in the following locations by default:
+
+- OS X: `/Applications/Binary Ninja.app`
+- Windows: `C:\Program Files\Vector35\BinaryNinja`
+- Linux: Wherever you extract it! (No standard location)
+
+!!! Warning "Warning"
+    Do not put any user content in the install-path of Binary Ninja. The auto-update process of Binary Ninja may replace any files included in these folders. 
 
-Once the license key is installed, you can change it, back it up, or otherwise inspect it simply by looking in:
+### User Folder
+
+The base locations of user folders are: 
 
 - OS X: `~/Library/Application Support/Binary Ninja`
 - Linux: `~/.binaryninja`
 - Windows: `%APPDATA%\Binary Ninja`
 
+Contents of the user folder includes:
+
+- `settings.json`: Advanced settings (see [settings](#settings))
+- `lastrun`: A text file containing the directory of the last BinaryNinja binary path -- very useful for plugins to resolve the install locations in non-default settings or on linux.
+- `plugins/`: Folder containing all manually installed user plugins
+- `repositories/`: Folder containing files and plugins managed by the [Plugin Manager API](https://api.binary.ninja/binaryninja.pluginmanager-module.html)
+
+![license popup >](/images/license-popup.png "License Popup")
+
+## License
+
+When you first run Binary Ninja, it will prompt you for your license key. You should have received your license key via email after your purchase. If not, please contact [support].
+
+Once the license key is installed, you can change it, back it up, or otherwise inspect it simply by looking inside the base of the user folder for `license.dat`.
+
+
 ## Linux Setup
 
 Because linux install locations can vary widely, we do not assume a Binary Ninja has been installed in any particular folder on linux. Rather, you can simply run `binaryninja/scripts/linux-setup.sh` after extracting the zip and various file associations, icons, and other settings will be set up. Run it with `-h` to see the customization options.
@@ -200,6 +227,17 @@ Plugins can be installed by one of two methods. First, they can be manually inst
 
 Alternatively, plugins can be installed with the new [pluginmanager](https://api.binary.ninja/binaryninja.pluginmanager-module.html) API.
 
+For more detailed information, see the [plugin guide](/guide/plugins).
+
+## PDB Plugin
+
+Binary Ninja supports loading PDB files through the built in PDB plugin. When selected from the plugin menu it attempts to find where the corresponding PDB file is located using the following search order:
+
+1. Look for in the same directory as the opened file/bndb (e.g. If you ahve `c:\foo.exe` or `c:\foo.bndb` open the pdb plugin looks for `c:\foo.pdb`)
+2. Look in the local symbol store. This is the directory specified by the settings: `local-store-relative` or `local-store-absolute`. The format of this directory is `foo.pdb\<guid>\foo.pdb`.
+3. Attempt to connect and download the PDB from the list of symbol servers specified in setting `symbol-server-list`.
+4. Prompt the user for the pdb.
+
 ## Preferences/Updates
 
 ![preferences >](/images/preferences.png "Preferences")
@@ -208,10 +246,37 @@ Binary Ninja automatically updates itself by default. This functionality can be
 
 On windows, this is achieved through a separate launcher that loads first and replaces the installation before launching the new version. On OS X and Linux, the original installation is overwritten after the update occurs as these operating systems allow files to be replaced while running. The update on restart is thus immediate.
 
-Most preferences are fairly intuitive. There is no advanced preference system at this time, but it is [expected](https://github.com/Vector35/binaryninja-api/issues/126) to be added soon.
+## Settings
 
-## Getting Support
+Settings are stored in the _user_ directory in the file `settings.json`. Each top level object in this file is represents a different plugin.  As of build 860 the following settings are available:
+
+|Plugin | Setting                  | Type         | Default                                        | Description                                                                                   |
+|------:|-------------------------:|-------------:|-----------------------------------------------:|:----------------------------------------------------------------------------------------------|
+| ui    | activeContent            | boolean      | True                                           | Allow Binary Ninja to connect to the web to check for updates                                 |
+| ui    | colorblind               | boolean      | True                                           | Choose colors that are visible to those with red/green colorblind                             |
+| ui    | debug                    | boolean      | False                                          | Enable developer debugging features (Additional views: Lifted IL, and SSA forms)              |
+| pdb   | local-store-absolute     | string       | ""                                             | Absolute path specifying where the pdb symbol store exists on this machine, overrides relative path |
+| pdb   | local-store-relative     | string       | "symbols"                                      | Path *relative* to the binaryninja _user_ directory, sepcifying the pdb symbol store            |
+| pdb   | auto-download-pdb        | boolean      | True                                           | Automatically download pdb files from specified symbol servers                                |
+| pdb   | symbol-server-list       | list(string) | ["http://msdl.microsoft.com/download/symbols"] | List of servers to query for pdb symbols.                                                     |
 
-Vector 35 offers a number of ways to get Binary Ninja [support].
+Below is an example `settings.json` setting various options:
+```
+{
+	"ui" :
+	{
+		"activeContent" : false,
+		"colorblind" : false,
+		"debug" : true
+	}
+    "pdb" :
+    {
+        "local-store-absolute" : "C:\Symbols",
+        "local-store-relative" : "",
+        "symbol-server-list" : ["http://mysymbolserver.company.lan"]
+    }
+}
+```
+## Getting Support
 
-[support]: https://binary.ninja/support/
+Vector 35 offers a number of ways to get Binary Ninja [support](https://binary.ninja/support/).
diff --git a/docs/guide/plugins.md b/docs/guide/plugins.md
index e69de29b..9c67d44f 100644
--- a/docs/guide/plugins.md
+++ b/docs/guide/plugins.md
@@ -0,0 +1,102 @@
+# Plugins
+
+Plugins really show off the power of Binary Ninja. This guide should help give you an overview of both using and writing plugins.
+
+The most common Binary Ninja plugins are Python which we are covering here. That said, there are some C++ plugins which must be built for the appropriate native architecture and will usually include build instructions for each platform. Several [C++ examples] are included in the API repository. 
+
+## Using Plugins
+
+Plugins are loaded from the user's plugin folder: 
+
+- OS X: `~/Library/Application Support/Binary Ninja/plugins/`
+- Linux: `~/.binaryninja/plugins/`
+- Windows: `%APPDATA%\Binary Ninja\plugins`
+
+Note that plugins installed via the [PluginManager API] are installed in the `repositories` folder in the same path as the previous `plugin` folder listed above.  You should not need to manually touch anything in that folder, but should access them via the API instead. 
+
+### Manual installation
+
+You can manually install a plugin either by adding a folder which contains it (the plugin folder must contain an `__init__.py` at the top of the folder, or a python file can be included directly in the plugin folder though this is not recommended).
+
+Note, if manually cloning the [api repository](https://github.com/Vector35/binaryninja-api), make sure to:
+
+```
+git submodule update --init --recursive
+```
+
+after cloning or else the submodules will not actually be downloaded. 
+
+### Installing via the API
+
+Binary Ninja now offers a [PluginManager API] which can simplify the process of finding and installing plugins. From the console:
+
+```
+>>> mgr = RepositoryManager()
+>>> dir(mgr)
+['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'add_repository', 'check_for_updates', 'default_repository', 'disable_plugin', 'enable_plugin', 'handle', 'install_plugin', 'plugins', 'repositories', 'uninstall_plugin', 'update_plugin']
+>>> mgr.plugins
+{'default': [<binaryninja-bookmarks not-installed/disabled>, <binaryninja-msp430 not-installed/disabled>, <binaryninja-radare2 not-installed/disabled>, <binaryninja-spu not-installed/disabled>, <binja-avr not-installed/disabled>, <binja_smali not-installed/disabled>, <binjatron not-installed/disabled>, <binoculars not-installed/disabled>, <easypatch not-installed/disabled>, <liil installed/enabled>, <list_comments not-installed/disabled>, <x64dbgbinja not-installed/disabled>]}
+>>> mgr.install_plugin(easypatch)
+True
+>>> mgr.enable(easypatch)
+True
+```
+
+Then just restart, and your plugin will be loaded. 
+
+### Installing Prerequisites
+
+Because Windows ships with an embedded version of Python, if you want to install plugins inside that Python, you'll need to either adjust your `sys.path` to include the locations for the other libraries (making sure they're compatible with the built-in version), or else install them directly in the environment via:
+
+```
+import pip
+pip.main(['install', '--quiet', 'packagename'])
+```
+
+_--quiet is required to minimize some of the normal output of pip that doesn't work within the context of our scripting console_
+
+For both OS X and Linux, Binary Ninja can utilize the built in system Python so any installed packages should be available there via whatever typical mechanism you use.
+
+### Troubleshooting
+
+Troubleshooting many Binary Ninja problems is helped by enabling debug logs and logging the output to a file. Just launch Binary Ninja with 
+
+```
+/Applications/Binary\ Ninja.app/Contents/MacOS/binaryninja -d -l /tmp/bnlog.txt
+```
+
+And check `/tmp/bnlog.txt` when you're done. 
+
+## Writing Plugins
+
+First, take a look at some of the [example] plugins, or some of the [community] plugins to get a feel for different APIs you might be interested in. Of course, the full [API] docs are online and available offline via the `Help`/`Open API Reference...`.
+
+To start, we suggest you download the [sample plugin] as a template since it contains all of the elements you're likely to need.
+
+- Begin by editing the `plugin.json` file 
+- Next, update the `LICENSE`
+- For small scripts, you can include all the code inside of `__init__.py`, though we recommend for most larger scripts that init just act as an initializer and call into functions organized appropriately in other files.
+
+### UI Elements
+
+While it is possible to use Qt to directly create [UI enhancements] to Binary Ninja, we don't recommend it. First, there's a chance that we'll change UI platforms in the future (in particular because Qt's QWidget performance is actually getting worse with newer versions and they're trying to move everyone to QTQuick which might as well be Electron). Secondly, it is much more difficult for other users to install your plugin given the much more complicated dependencies and cross-platform headache of setup.
+
+The officially supported mechanism (until the 1.2 release which will include much more featureful UI API enhancements) are available from the [interaction API] and shown off in the [angr] and [nampa] plugins.
+
+### Testing
+
+It's useful to be able to reload your plugin during testing. On the Commercial edition of Binary Ninja, this is easily accomplished with a stand-alone headless install using `import binaryninja` after [installing the API].  (install_api.py is included in every install in the installation folder)
+
+For the Personal edition, we recommend simply commenting out the `register_` function normally used to register the plugin via whatever mechanism it uses and instead simply using the built-in Python console along with the python `reload` function to load new changes and test them by directly calling functions in the module. This work-around unfortunately is not supported for Binary View or Architecture plugins which unfortunately do require a restart to test if not running on Commercial. 
+
+[PluginManager API]: https://api.binary.ninja/binaryninja.pluginmanager-module.html
+[example]: https://github.com/Vector35/binaryninja-api/tree/dev/python/examples
+[community]: https://github.com/Vector35/community-plugins
+[C++ examples]: https://github.com/Vector35/binaryninja-api/tree/dev/examples
+[API]: https://api.binary.ninja/
+[sample plugin]: https://github.com/Vector35/sample_plugin
+[UI enhancements]: https://github.com/NOPDev/BinjaDock
+[interaction API]: https://api.binary.ninja/binaryninja.interaction-module.html
+[angr]: https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/angr_plugin.py
+[nampa]: https://github.com/kenoph/nampa
+[installing the API]: https://github.com/Vector35/binaryninja-api/blob/dev/scripts/install_api.py
diff --git a/docs/guide/troubleshooting.md b/docs/guide/troubleshooting.md
index efd9fab9..a640d47e 100644
--- a/docs/guide/troubleshooting.md
+++ b/docs/guide/troubleshooting.md
@@ -9,17 +9,31 @@
 
 ## Bug Reproduction
 Running Binary Ninja with debug logging will make your bug report more useful.
+
 ```
 ./binaryninja --debug --stderr-log
 ```
 
+Alternatively, it might be easier to save debug logs to a file instead:
+
+```
+./binaryninja -d -l logfile.txt
+```
+
+(note that both long and short-form of the command-line arguments are demonstrated in the above examples)
+
 ## Plugin Troubleshooting
 
-While third party plugins are not officially supported, there are a number of troubleshooting tips that can help identify the cause. The most importat is to enable debug logging as suggested in the previous section. This will often highlight problems with python paths or any other issues that prevent plugins from running.
+While third party plugins are not officially supported, there are a number of troubleshooting tips that can help identify the cause. The most important is to enable debug logging as suggested in the previous section. This will often highlight problems with python paths or any other issues that prevent plugins from running.
+
+Additionally, if you're having trouble running a plugin in headless mode (without a GUI calling directly into the core), make sure you'er running the Commercial version of Binary Ninja as the Student/Non-Commercial edition does not support headless processing.
+
+Next, if running a python plugin, make sure the python requirements are met by your existing installation. Note that on windows, the bundled python is used and python requirements should be installed either by manually copying the modules to the `plugins` [folder](/getting-started/#directories).
+
 
 ## License Problems
 
-- If experiencing problems with Windows UAC permissions during an update, the easiest fix is to completely un-install and [recover][recover] the latest installer and license. Preferences are saved outside the installation folder and are preserved, though you might want to remove your [license](/getting-started/index.html#license).
+- If experiencing problems with Windows UAC permissions during an update, the easiest fix is to completely un-install and [recover][recover] the latest installer and license. Preferences are saved outside the installation folder and are preserved, though you might want to remove your [license](/getting-started/#license).
 - If you need to change the email address on your license, contact [support].
 
 ## Linux
@@ -48,6 +62,19 @@ cd ~/binaryninja
 QT_PLUGIN_PATH=./qt ./binaryninja
 ```
 
+### Debian
+
+For Debian variants that (Kali, eg) don't match packages with Ubuntu LTS or the latest stable, the following might fix problems with libssl and libcrypto:
+
+```
+$ cd binaryninja/plugins
+$ ln -s libssl.so libssl.so.1.0.0
+$ ln -s libcrypto.so libcrypto.so.1.0.0
+```
+
+### Gentoo
+
+One Gentoo user [reported][issue672] a failed SSL certificate when trying to update. The solution was to copy over `/etc/ssl/certs/ca-certificates.crt` from another Linux distribution.
 
 ## API
 
@@ -60,3 +87,4 @@ QT_PLUGIN_PATH=./qt ./binaryninja
 [support]: https://binary.ninja/support.html
 [faq]: https://binary.ninja/faq.html
 [purchase]: https://binary.ninja/purchase.html
+[issue672]: https://github.com/Vector35/binaryninja-api/issues/672
diff --git a/docs/guide/type.md b/docs/guide/type.md
index e69de29b..65121864 100644
--- a/docs/guide/type.md
+++ b/docs/guide/type.md
@@ -0,0 +1,2 @@
+# Types and Structures
+