| Age | Commit message (Collapse) | Author |
|---|
|
Prior to this we would not have meaningful errors shown to the user when we fail to construct a CacheEntry from a file.
Apart of trying to make the opening of shared caches clearer.
|
|
Fixes https://github.com/Vector35/binaryninja-api/issues/6629
|
|
Allows us to display the user friendly name of the cache entry in the triage view
|
|
This improves symbol recovery drastically on newer shared caches
Related PR: https://github.com/Vector35/binaryninja-api/pull/6210
|
|
Implicit std::filesystem::path conversion oops
|
|
We already had separated this part out into the `CacheProcessor` but its more like a builder than anything, so this refactors that out into a `SharedCacheBuilder`.
- Separate out the `CacheProcessor` and simplify its implementation
- Move more implementation specific stuff to the shared cache view
- Prepare for more validation of the shared cache to detect irregular or incomplete shared cache information
- Deduplicate a lot of file vs. project file lookup stuff
- Stop copying all images when getting the number of images to log
- Allow user to select another directory to scan for shared cache files, might make this a warning instead to prevent users from relying on this behavior
We will likely follow this commit up soon with better open behavior, maybe open with options can specify a list of cache entry files? Shouldn't be too much effort aside from making the UI modal for that.
|
|
And also removed unneeded VMReader construction in ApplyRegion
|
|
|
|
Fixes the case of stub functions being applied with mangled names.
|
|
This made the macho objective-c processor aware of images and was passed around everywhere, instead we have an overridable section getter that gives the control over the section retrieval to the derived processor.
In the case of shared cache this means we can store the image to be processed and then use the header to locate the relevant sections.
Fixes: https://github.com/Vector35/binaryninja-api/issues/6594
|
|
This removes some section name collisions where a section name is in two separate image segments.
See https://github.com/Vector35/binaryninja-api/pull/6454 for more information.
Co-Authored-By: WeiN76LQh <WeiN76LQh@github.com>
|
|
Fixes: https://github.com/Vector35/binaryninja-api/issues/6571
|
|
This fixes a case where the remote file is downloaded and the shared cache view is selected, we _only_ work for file backed views, as our shared cache specific file accessors work on those. So this is not a loss in functionality.
In the future if we wanted to load shared caches without touching disk (yikes) than we would need to remove this restriction, or provide a way in the API for user to process their own cache entries (after view init).
|
|
This gives the user the ability to take a database with no original file path continue processing entries with the user supplied base file.
This also will set the original file path for future loads, we might want to make that optional. And the behavior with remote projects for that might be undesired.
|
|
Fixes https://github.com/Vector35/binaryninja-api/issues/6582
More considerations might need to be made for databases saved without the original file path. Likely solved through some shared cache specific metadata.
|
|
It is modified on a worker thread so if a user tries to call `GetSymbolWithName` after view init but before the processing on the worker thread finishes, there would have been issues!
|
|
Also swapped out CacheEntry::m_images to a vector as its never actually used as a lookup
|
|
slide info
This improves performance by ~10x with the only real issue being a copy of the cache entry existing for each weak ref.
Also fixes
- Some old TODO's that are no longer relevant
- Some function signatures not being const
- FileAccessorCache not evicting enough accessors to fit under an adjusted cache size
- Added a warning if we are over the global cache size in view initialization
- Logger name not having a `.` in `SlideInfoProcessor::SlideInfoProcessor`
- Re-added the accessor dirty check before applying slide info to fix https://github.com/Vector35/binaryninja-api/issues/6570
|
|
adding new images and regions
Within the user section creation we will assume that because a non-default section was added, we should mark all functions for reanalysis. From initial testing
this isn't an extreme slowdown, but with the addition of more images the number of queued up functions becomes much greater, even if those are unmarked immediately
and processing of those functions isn't slow it is unneeded and might lead to other non-obvious slowdowns.
To be clear, we do not need to queue up functions from already analyzed images to be analyzed again, there is no benefit, our workflow handles cross images calls already,
so those places will already be queued up, and if they aren't, that is the issue to solve, not this.
|
|
Fixes https://github.com/Vector35/binaryninja-api/issues/6561
- Also tightens the SharedCache class to move only, to prevent accidental copies.
- Also removes some extra copies in FFI when should pass by ref
- Also adds `get_symbol_with_name` to python API
The current named symbol map is populated in a worker thread spawned in the view init. This is because populating the map can take about 1 second.
If we are fine with another 1 second added to the view init time then we can add it serially but I don't think this way is _that_ bad, no analysis consults
this, however a user might add a workflow that would be racing this. So we need to add a mutex.
|
|
|
|
The logic for determining the class name of a category did not correctly
handle classes defined in other images in the shared cache. There were
two problems:
1. If the class is defined in another image that is already loaded,
`ObjCProcessor` has already renamed the symbol from `_OBJC_CLASS_$_`
to `cls_`. Both forms of symbol name are now handled.
2. If the class is defined in an image that is not yet loaded, no symbol
name is available. The category's class is now looked up in the
shared cache symbol table, and the symbol's name is parsed as if it
were an import symbol.
This fixes almost all cases of "Failed to determine base classname for
category" that I have come across.
Mason Reed: Fixed up to make objective-c processor always consult GetSymbol
|
|
|
|
address for the file offset
This would result in the virtual memory trying to access an unmapped region, failing to apply slide info
|
|
Just use a vector. In the future we can turn off the write log if the system has given us ample descriptors, and we can also add a page based cache.
|
|
|
|
|
|
|
|
Apparently the view metadata is not available until after view init
|
|
|
|
Stopped some stub calls resolving the target
|
|
removing it
|
|
|
|
|
|
In absence of a better name, this commit refactors the shared cache code.
|
|
This makes it so that we persist the symbols within the regular symbol list in storage
|
|
gracefully now
|
|
Slide info is parsed and applied on the main thread, below the
`SharedCache` constructor, when a shared cache is opened. Time spent
applying the slide is time that the main thread is blocked.
These changes eliminate some unnecessary overhead so what work remains
is dominated by kernel work (paging in data, copying pages when the
first modification is made).
The changes are:
1. Read slide pointers via `ReadULong` rather than the variable-length
`Read` method. The compiler isn't able to eliminate the call to
`memcpy` in the variable-length `Read` function, and the function
call overhead is noticeable given the small size of the read and
number of times it is called.
2. Remove the file member from `MappingInfo`. Slide info is applied for
a single file at a time so there's no reason to track the file it
belongs to. This removes unnecesssary reference counting on the
`std::shared_ptr` that holds the `MMappedFileAccessor`.
|
|
Still need to decide on how we want handle this, because we need multiple files we probably just want to prompt the user for a directory?
|
|
This warning will happen anytime you load an image with already processed regions.
|
|
Both the Macho and DSC views need to process Objective-C but have separate processor classes. It would appear that the DSC version was largely a copy and paste of the Macho view one, with some modifications. The majority of code overlaps between the 2 so it doesn't make sense to maintain 2 and copy and paste improvements/fixes between them.
This commit fixes that by creating a base Objective-C processor that contains the shared code. View specific code is implemented in the respective subclasses for the views. Although there is very little view specific code for each.
|
|
|
|
`MemoryRegion` now tracks the start address of the image to which it
belongs. `SharedCache::HeaderForAddress` uses the existing address range
map to quickly find the `MemoryRegion` for a given address, and from
there can look up the image via its start address.
Some extra logic is added to `CacheInfo::Load` to populate the image
start address on `MemoryRegion` when loading from metadata that predates
this change.
The result is that `HeaderForAddress` is no longer the most expensive
part of `FindSymbolAtAddrAndApplyToAddr`.
|
|
Use the LLVM demangler to demangle relevant symbols
|
|
|
|
Also we now initialize m_cacheInfo when calling `DeserializeFromRawView` the responsibility is on the caller to handle
|
|
version
This does not actually give the user control to exit early, that looks to need core changes.
|
|
to the regions being loaded
The logic before would default to adding a section unless it was from a region in the `regionsToLoad` argument and its header had already been initialized. However if a user does `Load Section by Address` then there's only one region in `regionsToLoad`. All sections not in that region would be automatically added, even if they had already been or the user hadn't requested them to be loaded.
|
|
|
|
Also fixes a BinaryViewRef leak
|
