| Age | Commit message (Collapse) | Author |
|---|
|
How did we not have this before?
|
|
Also split out SegmentFlags and fix some UB with the section creation
|
|
|
|
|
|
|
|
MMappedFileAccessor::Open
Nothing guarantees that the `SharedCache` lives long enough.
|
|
|
|
There was a second BeginBulkModifySymbols after rebasing the new processing symbols commit
|
|
Closes Vector35/binaryninja-api#6430
|
|
Causing unneeded noise in the debug print
|
|
This was causing a lot of extra noise, where the function is kept just to ref for the tag name
|
|
This caused a crash when a user called function.tags() on a function with tags
|
|
This is _probably_ the fix? I am not actually sure I am pushing so the CI can figure it out for me :)
|
|
|
|
|
|
This allows us to construct a symbol without constructing a namespace, while also giving us the ability to specify raw, long, short names
|
|
Split out from https://github.com/WeiN76LQh/binaryninja-api/tree/process-local-symbols
|
|
Split out from https://github.com/WeiN76LQh/binaryninja-api/tree/process-local-symbols
|
|
|
|
GLOB_RECURSE has been cut down a little bit to make configuring times reasonable
|
|
|
|
|
|
The initial state is initialized during `PerformInitialLoad` and is
immutable after that point. This required some slight restructuring of
how information about memory regions is tracked as that was previously
modified as regions were loaded. Memory regions are now stored in a map
from their address range to the `MemoryRegion` object. This makes it
cheap to look them up by address which is a common operation.
The modified state consists of changes since the last save to the
`DSCView` / `ViewSpecificState`. This means it is no longer necessary to
copy any state when mutating a `SharedCache` instance for the first
time. Instead, its data structures start off empty and are populated as
images, sections, or symbol information is loaded.
The loaded state consists of all modified state that has since been
saved. It lives on the `ViewSpecificState`. Saving modified state
merges it into the the existing loaded state.
This pattern is carried over to the `Metadata` stored on the `DSCView`.
The initial state is stored under its own metadata key, and each
modified state is stored under a key with an incrementing number. This
means each save of the state only needs to serialize the state that
changed, rather than reserializing all of the state all of the time.
There are two huge benefits from these changes:
1. At no point does `SharedCache` have to copy its in memory state.
The basic copy-on-write approach introduced in #6129 reduced how
often these copies are made, but they're still frequent and very
expensive.
1. At no point does `SharedCache` have to re-serialize state to JSON
that it has already serialized. JSON serialization previously added
hundreds of milliseconds to any mutating operation on `SharedCache`.
As a result, this speeds up the initial load of the shared cache by
around 2x and loading of subsequent images improves by about the same.
One trade-off is that the serialization / deserialization logic is more
complicated. There are two reasons for this:
1. The state is now split across multiple metadata keys and needs to be
merged when it is loaded.
2. The in-memory representation uses pointers to identify memory regions.
These relationships have to be re-established after the JSON is
deserialized.
As a future direction it is worth considering whether the logic owned by
`SharedCache` could be split in a similar manner to the data. The
initial loading of the cache header, loading of images, and handling of
symbol information are all mostly independent and work on separate data.
If the logic were split into separate classes it would be easier to
reason about which data is valid when, and would easily permit
concurrent loading of multiple images from the shared library in a
thread-safe manner.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Also added actual test for base address detection
|
|
If we add new rust dependencies we should now be alerted when a new license type is added.
|
|
|
|
|
|
Binary Ninja now supports these formats in the core
|
|
|
|
|
|
functions
|
|
|
|
Fixes MSVC C++17 failing to compile, hopefully.
|
|
|
|
which had not been serialized.
We will just return early, this is fine as the previous code would effectively do the same thing.
|
|
These are from clang so there are still a bunch of warnings on GCC and MSVC
|
|
This apparently is not an issue on LLVM, but GCC and MSVC both seem to dislike this.
|
|
Potential nullptr deref on invalid view type (see the error log right after the logger init)
|
|
This is a meaningless commit, i'm throwing this in for when someone inevitably calls out the inconsistent "formatting" (what style is this again?)
|
|
A logic error in `FindSymbolAtAddrAndApplyToAddr` meant that a code path
that would attempt to re-use existing symbol definitions in the view
would still fall back to searching the exported symbol list.
`FindSymbolAtAddrAndApplyToAddr` is updated to return early if a symbol
already exists at the target location and the target location is the
same as the symbol location. Additionally, in the event a symbol was found
in the view at the symbol location, it is applied to the target location
and then the function returns.
Additionally, `WillMutateState` is moved after these initial checks so
it is only called in the codepaths that attempt to locate the symbol
amongst the exported symbols.
After these changes it is now possible to load QuartzCore from the iOS
14 shared cache in around 70 seconds. Previously it would run for hours
without the load completing.
|
|
Also add some other misc error reporting and nullptr check
|
|
This allows a consumer to bypass the alloc + free of the API NameSpace, removing the need for the consumer to manually construct a symbol for said behavior.
|
|
Previously, `MMappedFileAccessor::Open` attempted to impose a fixed limit
on the number of file accessors that were live at one time. This was
done because the default file descriptor limit on some platforms is
relatively low (256 on macOS). Given that recent iOS shared caches can
contain 60+ files it is easy to tie up a large percentage of this limit
by opening one or two shared caches.
This is problematic as if the limit imposed by `MMappedFileAccessor::Open`
is reached, the attempt to access the file will block waiting for
another file accessor to be closed. This can lead to a deadlock.
This commit makes three changes to this strategy:
1. It attempts to raise the file descriptor limit to 1024. Unix systems
support both soft and hard file descriptor limits. The soft file
descriptor limit is what is enforced, but a process can explicitly
raise the limit to any value below the hard limit if it wishes.
2. The fixed limit on open files accessors is changed to a soft limit.
`FileAccessorCache` is introduced to manage the caching of file
accessors. It provides a basic LRU cache. Whenever a new file is
opened, the cache of open accessors is pruned to stay below the
target limit (50% of the soft file descriptor limit). This limiting
is primarily done to allow files containing dirty pages to be
unmapped if they're no longer being used.
LRU isn't the optimal strategy for this, but it is simple to
implement and understand. Ideally there'd be some access time
component to the cache so files that haven't been accessed can be
released.
3. File accessors are cached even for sessions corresponding to views
that have been closed. The "Open Selection with Options..." context
menu hits this case as a view is created and destroyed as part of
populating the options dialog, and the real view is then created in
the same session.
The most significant benefit of this change is that the logic around
opening / closing file accessors is simpler and can no longer deadlock.
While working on this I noticed that `SharedCache` opened some files via
`MMappedFileAccessor::Open` without specifying a post-open operation to
apply slide information. Instead, it would explicitly apply the slide
information after opening the file. This works fine so long as the file
accessor limit is never hit. If it is hit and the accessor is closed,
the next time the file is opened it will not have any slide information
applied. This would lead to very confusing bugs.
|
