From 1ffdd4da176f1990cdf0ff4d0079a3ccb4e51206 Mon Sep 17 00:00:00 2001 From: Mark Rowe Date: Thu, 30 Jan 2025 09:48:06 -0800 Subject: [SharedCache] Rework how file accessors are handled Previously, `MMappedFileAccessor::Open` attempted to impose a fixed limit on the number of file accessors that were live at one time. This was done because the default file descriptor limit on some platforms is relatively low (256 on macOS). Given that recent iOS shared caches can contain 60+ files it is easy to tie up a large percentage of this limit by opening one or two shared caches. This is problematic as if the limit imposed by `MMappedFileAccessor::Open` is reached, the attempt to access the file will block waiting for another file accessor to be closed. This can lead to a deadlock. This commit makes three changes to this strategy: 1. It attempts to raise the file descriptor limit to 1024. Unix systems support both soft and hard file descriptor limits. The soft file descriptor limit is what is enforced, but a process can explicitly raise the limit to any value below the hard limit if it wishes. 2. The fixed limit on open files accessors is changed to a soft limit. `FileAccessorCache` is introduced to manage the caching of file accessors. It provides a basic LRU cache. Whenever a new file is opened, the cache of open accessors is pruned to stay below the target limit (50% of the soft file descriptor limit). This limiting is primarily done to allow files containing dirty pages to be unmapped if they're no longer being used. LRU isn't the optimal strategy for this, but it is simple to implement and understand. Ideally there'd be some access time component to the cache so files that haven't been accessed can be released. 3. File accessors are cached even for sessions corresponding to views that have been closed. The "Open Selection with Options..." context menu hits this case as a view is created and destroyed as part of populating the options dialog, and the real view is then created in the same session. The most significant benefit of this change is that the logic around opening / closing file accessors is simpler and can no longer deadlock. While working on this I noticed that `SharedCache` opened some files via `MMappedFileAccessor::Open` without specifying a post-open operation to apply slide information. Instead, it would explicitly apply the slide information after opening the file. This works fine so long as the file accessor limit is never hit. If it is hit and the accessor is closed, the next time the file is opened it will not have any slide information applied. This would lead to very confusing bugs. --- view/sharedcache/core/SharedCache.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'view/sharedcache/core/SharedCache.h') diff --git a/view/sharedcache/core/SharedCache.h b/view/sharedcache/core/SharedCache.h index 1b3f8261..1ecfd885 100644 --- a/view/sharedcache/core/SharedCache.h +++ b/view/sharedcache/core/SharedCache.h @@ -588,13 +588,14 @@ namespace SharedCacheCore { void DeserializeFromRawView(); public: - std::shared_ptr GetVMMap(bool mapPages = true); + std::shared_ptr GetVMMap(); static SharedCache* GetFromDSCView(BinaryNinja::Ref dscView); static uint64_t FastGetBackingCacheCount(BinaryNinja::Ref dscView); bool SaveToDSCView(); void ParseAndApplySlideInfoForFile(std::shared_ptr file); + std::optional GetImageStart(std::string installName); std::optional HeaderForAddress(uint64_t); bool LoadImageWithInstallName(std::string installName, bool skipObjC); @@ -655,6 +656,9 @@ private: // Must be called before first access to `MutableState()` after the state // is loaded from the cache. Can safely be called multiple times. void WillMutateState(); + + std::shared_ptr MapFile(const std::string& path); + static std::shared_ptr MapFileWithoutApplyingSlide(const std::string& path); }; } -- cgit v1.3.1