summaryrefslogtreecommitdiff
path: root/plugins/idb_import
diff options
context:
space:
mode:
authorMason Reed <mason@vector35.com>2024-12-22 01:21:02 -0500
committerMason Reed <mason@vector35.com>2025-01-25 00:52:29 -0500
commit6a63c17853009ce7a0071458e39c140a09440230 (patch)
treead59ab3b356b0a2c60423e55f62c996891788788 /plugins/idb_import
parentbfa1d409b2d5e734804a5fdb4c68c90a93712f1f (diff)
Rust refactor
Moves a bunch of stuff out of src/types.rs that did not belong: - Confidence - Variable - Function specific stuff - Refactored InstructionInfo, see the msp430 and riscv examples. - Renamed Function::from_raw to Function::ref_from_raw and fixed places where the ref was incremented twice - Fixed FunctionRecognizer leaking functions (see above) - Fixed some apis incorrectly returning Result where Option is clearer - Started to move destructured types to the From trait for converting to an from ffi types, see Location for an example - Started to remove bad module level imports (importing std modules like mem all over the place) - Moved some wrapper handle types to named handle field (this improves readability), see CoreArchitecture for an example - Removed some unintuitive getters, this is bad practice for Rust code, just access the field directly, see DataVariable for an example - General code cleanup, purposely did not run rustfmt, that will be a single seperate commit More rust cleanup - Fixed invalid views being able to invoke UB when dealing with databases - Cleaned up some helper code in dwarf_import - Fixed inverted is_null checks causing crashes! Oops! More rust cleanup Still a WIP, I think branch info is still invalid, need to figure out the issue there. - Fixed some invalid Ref lifetimes when constructing indirectly, see Array<DataVariable> for example - Added some more comments - Removed some "magic" functions like MLIL Function::ssa_variables There are still a bunch of invalid lifetimes that aren't crashing us due to the usage of those API's not living long enough. But they ARE an issue. More rust cleanup Trying to comment more TODO's as I go along. - Renamed llil::Function to llil::LowLevelILFunction for clarity and consistency - Take base structures by ref in StructureBuilder::set_base_structures to prevent premature drops - Added more raw to wrapper conversions - Removed UB prone apis - Getting rid of more core module references, use std! - Removed improper Guard usage in array impls for wrapper types with no context - Untangling the UB of the Label api, still only half done :/ More rust cleanup - Misc formatting - Made Logger ref counted - Fixed leaking name of logger every time something was logged - Fixed the last (hopefully) of the unresolved labels - Simplified some code Fix leak in DebugInfo::AddType componentArray was never freed Add more HLIL related functions to rust More rust cleanup improve the CustomBinaryView init process Canonicalize path in `create_delete_empty` test Link core in rust When linking you must depend on the -sys crate. This is because linker arguments (what says where to find binaryninjacore) are NOT transitive. The top level application crate MUST provide it. For more information see: - https://github.com/rust-lang/cargo/issues/9554#issuecomment-857882964 - https://github.com/oxidecomputer/omicron/pull/225 Remove vendored pdb crate Use cargo to manage the git repo ref instead Fix misc rustdoc warnings Move actual plugins out of `rust/examples` and into `plugins` This is where all shipped public plugins that are not arch/view/platform/lang will be at from now on Originally they were in the rust workspace, meaning they all shared a Cargo.lock which is ill-advised. More rust cleanup - More clarification on plugin/executable requirements - Made examples actually rust examples - Add Display impl for InstructionTextToken - Renamed feature "noexports" to "no_exports" Move under_construction.png to assets directory This really did bother me Remove unneeded `extern crate bindgen` Replace nop'd log::info with println We don't register a compatible log sink so they will just get sent into the void Move inline tests into tests directory This is being done in the hopes of curbing the multi-thousand line files that will occur once we flesh out the tests Format rust code Update rust ci Still need to add support for running tests in ci More rust cleanup - Architecture id's are now typed accordingly - Fix some clippy lints - Make instruction index public in LLIL - Removed useless helper functions - LLIL expressions and instruction indexes are now typed accordingly Generate binaryninjacore-sys documentation This should show binaryninjacore-sys alongside binaryninja crate More rust cleanup - Remove lazy_static dependency - Remove hacky impl Debug for Type and just use the display impl - Add more debug impls - Reorder some top level namespace items - Add first type test Remove unneeded script helper in rust api More rust cleanup - Added main thread handler api - Register a headless main thread handler by default in initialization - Refactor QualifiedName to be properly owned - Loosened some type constraints on some apis involving QualifiedName - Fixed some apis that were crashing due to incorrect param types - Removed extern crate cruft for log crate - Simplified headless initialization using more wrapper apis - Fixed segments leaking because of no ref wrapper, see BinaryViewExt::segment_at - Added rstest to manage headless init in unit tests - Added some more unit tests - Refactored demangler api to be more ergonomic - Fixed minidump plugin not building More rust cleanup - Fixup usage of QualifiedName in plugins - Make QualifiedName more usable Implement rust TypeParser fix Platform TypeParser related functions separate User and System implementations of TypeParserResult Implement rust TypeContainer More rust cleanup - Hopefully fixed the rust.yml CI - Added TypePrinter API (this is still WIP and will crash) - Added TypeParser API - Added TypeContainer API - More work around QualifiedName apis Oh your suppose to do this Add workflow_dispatch trigger to rust.yml More rust fixes - Swapped some usage of raw 255 to MAX_CONFIDENCE, no one likes magic numbers - New InstructionTextToken API, complete with owned data, this still needs a lot of testing. - InstructionTextTokenKind describes a destructured InstructionTextToken, this should make token usage much clearer, some docs pending - Added some misc Default and Debug impls - Updated TypePrinter and architectures to use new InstructionTextToken API Misc formatting changes More rust cleanup - Fixed MANY memory leaks (most due to QualifiedName) - Made StructureBuilder more builder and less structure - Fixed CMakeLists.txt that were globbing entire api, resulting in 100 second slowdown on cmake generation - Added more Debug impl's - Added some more tests - Fixed TypeParserResult UB - Moved the From impls to blank impl for clarity, we have multiple different variants of core to rust for some structures, it should be explicit which one you are choosing. - PossibleValueSet should now be able to allocate so we can go from rust to core with those variants that require allocation - Misc doc code formatting Misc clippy lints and clippy CI Co-authored-by: Michael Krasnitski <michael.krasnitski@gmail.com> Fix typo in rust CI Misc rust formatting Fix misc typos and add typos to rust CI Add cargo workspace This will help tooling and external contributors get a map of the rust crates within binaryninja-api More rust cleanup - Format all rust plugins - Fix some tests that were out of date - Simplify WARP tests to only binaries, building object files from source is a pain - Link to core in all rust plugins - Fix some memory leaks - Update warp insta snapshots - Fix some misc clippy lints Run rust tests in CI This commit also coincides with the creation of the "testing" environment which exposes a BN_SERIAL secret for pulling a headless Binary Ninja Install missing wayland dependency in github CI Apparently its needed for linux file picker for the WARP integration Set the BINARYNINJADIR so rust can find binaryninjacore in CI The chances of this working are low Misc remove unused dependency Rust misc formatting fixes Improve initialization in rust headless scripts Provide sensible errors and validation to rust headless scripts, solves https://github.com/Vector35/binaryninja-api/issues/5796 Add BN_LICENSE environment variable to rust CI We pass the serial to download binary ninja, but we never provided the license for core initialization Fix typo More rust cleanup - Improved binary view initialization (see init_with_opts) - Allow floating license to free itself before initialization - Add initialization unit test - Add better Debug impls for some common types - Use Path api for opening binary views, this is not breaking as it uses the AsRef impl - Bump rayon dependency and constrain dependencies to x.x Update readme and include that directly in the rustdocs More rust documentation changes Add format comment to InitializationOptions::with_license Misc formatting and clippy lint allow More rust cleanup - Remove under_construction.png from the build.rs it has been removed - Use InstructionIndex in more places - Add missing PartialOrd and Ord impls for id types More rust cleanup - Make workflow cloning explicit - Add workflow tests - Add missing property string list getting for settings - Remove IntoActivityName (see https://github.com/Vector35/binaryninja-api/pull/6257) More rust cleanup This commit is half done Misc rust formatting More rust cleanup - Renamed common name conflictions (I will put my justification in the PR) - Fixed invalid instruction retrieval for LLIL - Added common aliases for llil function, instruction and expression types (see my comment in the PR) - Refactored the instruction retrieval for LLIL, MLIL and HLIL - Added instruction index types to MLIL and HLIL - Moved llil module to lowlevelil module (mlil and hlil will be moved as well) - Added preliminary LLIL unit testing Fix typos Misc clippy fixes More rust cleanup - Normalized modules - Split some code out into own files - Fixed some UB in type archive and projects - Improved API around type archives and projects substantially - Added ProgressExecutor abstraction for functions which have a progress callback - Improved background task documentation and added unit tests - Added worker thread api and unit tests - Moved some owned types to ref types, this is still not complete, but this is the path forward. - Add external location/library accessors to the binary view - Added some misc documentation - Replaced mod.rs with the module name source file Still need to normalize some paths and also update some documentation surrounding that change. Update some tests and examples Fix background task tests colliding We were creating multiple background tasks with the same progress text on multiple threads More rust cleanup - Fixed progress executor freeing itself after one iteration - Updated the last of the doc imports - Moved mainthread to main_thread - Made project creation and opening failable We could probably AsRef<ProgressExecutor> to get around the allocation and free inside the function bodies, not high priority as those functions are long running anyways. Move binary view initialization into function body for LLIL test Normalize test file names More rust cleanup - Updated README to clarify offline documentation - Refactored settings api - Added settings example to show dumping settings value and specific properties - Use the workspace to depend on binaryninja and binaryninjacore-sys - Remove binaryninjacore-sys as a workspace member (its not really required) Update workflow test to new settings api More rust cleanup - Rename Label to LowLevelILLabel - Update the functions label map automatically This fixed a major api blunder where the label map is returned as a reference and originally resulted in UB prone lifetime semantics. It was temporarily "fixed" with explicit label updates in the architecture implementation code. But that was less than ideal and was easy to mess up. Now the label map will be updated automatically as the location of labels is now tracked. Misc clippy lints More rust cleanup - Get rid of RawFunctionViewType - Add better Debug impl for Function More rust cleanup - Fixed the documentation icon using local files (thank you @mkrasnitski) - Fixed labels being updated and overwriting the label location used to update the label map More rust cleanup - Added unit tests for MLIL and HLIL - "Fixed" MLIL, LLIL, and HLIL having issues regarding Instruction vs Expression indexes - Renamed CallingConvention to CoreCallingConvention and removed architecture generic - Renamed CallingConventionBase to CallingConvention - Simplified calling convention code and fixed some bugs with implicit registers - Added impl Debug to MLIL and HLIL instructions Still need to at some point add an Expression to MLIL and HLIL. We also might want to look into having the Instruction kind just return the expression kind. Misc clippy lint More rust cleanup - Allow calling conventions to be registered for multiple architectures - Swapped a unreachable statement to an unimplemented statement More rust cleanup - Fixed the issue with PDB types, this has caused me an insane amount of grief - Fixed LLIL visit_tree missing LLIL_LOAD expressions - Added LLIL visitor test - Made all WARP file pickers use the rfd crate Use the dev branch of Binary Ninja in rust CI Misc rust fmt More rust cleanup - Refactored BinaryReader and BinaryWriter - Added some warnings to high_level_il and medium_level_il modules that they are unstable - Add BinaryReader and BinaryWriter tests - Changed BinaryView len to return u64 (that is what the core returns) - Misc formatting changes - Remove extern uses in lib.rs Add impl Debug for BinaryReader and BinaryWriter Turn off broken tests Add more info to the rust README.md More rust cleanup - Make EdgeStyle type not wrap raw - Regression tests for WARP will run on all bins in the out dir now impl rust Collaboration and Remote API Fix typo Update collaboration API Makes collaboration more in line with the refactor. Still a lot of work to do. Namely still need: - Proper errors - _with_opts functions - More ergonomic api - Better connection procedure - Updated documentation - A LOT of unit tests - An example - Typed id's for everything (i dont want BnString as the id!!!) - NEED to refactor the progress callbacks into the new progress api, but we should pull in some of the stuff the collab progress has - Elimination of apis that are dumb helpers Separate out the rust testing and use pull_request_target pull_request_target allows PR's to access the headless license, for this to be safe we need to prevent people from running the job. To prevent the job from being ran we add an environment requirement on testing that a reviewer must review the code and then manually approve it to run. More rust cleanup - Use GroupId instead of u64 - Use ProgressCallback in place of ProgressExecutor - Misc cleanup of FileMetadata - Add `save_to_path` and `save_to_accessor` to save modified binaries - Added binary_view unit tests - Added collaboration unit tests - Fixed a few issues with the collaboration apis - Renamed Command registration functions so that there is no import ambiguity - Split out RemoteUndoEntry - Collaboration apis now have a explicit `_with_progress` set of apis - Misc clippy lint fixes Fix some typos More rust cleanup - Add extra info to README.md - Refactor components api - Add components unit test Add testing and documentation to contributing section in README.md Fix misc doc comments
Diffstat (limited to 'plugins/idb_import')
-rw-r--r--plugins/idb_import/CMakeLists.txt91
-rw-r--r--plugins/idb_import/Cargo.toml15
-rw-r--r--plugins/idb_import/build.rs15
-rw-r--r--plugins/idb_import/src/addr_info.rs57
-rw-r--r--plugins/idb_import/src/lib.rs344
-rw-r--r--plugins/idb_import/src/types.rs700
6 files changed, 1222 insertions, 0 deletions
diff --git a/plugins/idb_import/CMakeLists.txt b/plugins/idb_import/CMakeLists.txt
new file mode 100644
index 00000000..6a6104d9
--- /dev/null
+++ b/plugins/idb_import/CMakeLists.txt
@@ -0,0 +1,91 @@
+cmake_minimum_required(VERSION 3.9 FATAL_ERROR)
+
+project(idb_import)
+
+file(GLOB_RECURSE PLUGIN_SOURCES CONFIGURE_DEPENDS
+ ${PROJECT_SOURCE_DIR}/Cargo.toml
+ ${PROJECT_SOURCE_DIR}/src/*.rs)
+
+file(GLOB_RECURSE API_SOURCES CONFIGURE_DEPENDS
+ ${PROJECT_SOURCE_DIR}/../../binaryninjacore.h
+ ${PROJECT_SOURCE_DIR}/../../rust/binaryninjacore-sys/build.rs
+ ${PROJECT_SOURCE_DIR}/../../rust/binaryninjacore-sys/Cargo.toml
+ ${PROJECT_SOURCE_DIR}/../../rust/binaryninjacore-sys/src/*
+ ${PROJECT_SOURCE_DIR}/../../rust/Cargo.toml
+ ${PROJECT_SOURCE_DIR}/../../rust/src/*.rs)
+
+if(CMAKE_BUILD_TYPE MATCHES Debug)
+ set(TARGET_DIR ${PROJECT_BINARY_DIR}/target/debug)
+ set(CARGO_OPTS --target-dir=${PROJECT_BINARY_DIR}/target)
+else()
+ set(TARGET_DIR ${PROJECT_BINARY_DIR}/target/release)
+ set(CARGO_OPTS --target-dir=${PROJECT_BINARY_DIR}/target --release)
+ set(OUTPUT_PDB_NAME ${CMAKE_SHARED_LIBRARY_PREFIX}idb_import.pdb)
+endif()
+
+set(OUTPUT_FILE ${CMAKE_STATIC_LIBRARY_PREFIX}idb_import${CMAKE_SHARED_LIBRARY_SUFFIX})
+set(PLUGIN_PATH ${TARGET_DIR}/${OUTPUT_FILE})
+
+add_custom_target(idb_import ALL DEPENDS ${PLUGIN_PATH})
+add_dependencies(idb_import binaryninjaapi)
+
+find_program(RUSTUP_PATH rustup REQUIRED HINTS ~/.cargo/bin)
+if(CARGO_API_VERSION)
+ set(RUSTUP_COMMAND ${RUSTUP_PATH} run ${CARGO_API_VERSION} cargo build)
+else()
+ set(RUSTUP_COMMAND ${RUSTUP_PATH} run ${CARGO_STABLE_VERSION} cargo build)
+endif()
+
+if(APPLE)
+ if(UNIVERSAL)
+ if(CMAKE_BUILD_TYPE MATCHES Debug)
+ set(AARCH64_LIB_PATH ${PROJECT_BINARY_DIR}/target/aarch64-apple-darwin/debug/${OUTPUT_FILE})
+ set(X86_64_LIB_PATH ${PROJECT_BINARY_DIR}/target/x86_64-apple-darwin/debug/${OUTPUT_FILE})
+ else()
+ set(AARCH64_LIB_PATH ${PROJECT_BINARY_DIR}/target/aarch64-apple-darwin/release/${OUTPUT_FILE})
+ set(X86_64_LIB_PATH ${PROJECT_BINARY_DIR}/target/x86_64-apple-darwin/release/${OUTPUT_FILE})
+ endif()
+
+ add_custom_command(
+ OUTPUT ${PLUGIN_PATH}
+ COMMAND ${CMAKE_COMMAND} -E env
+ MACOSX_DEPLOYMENT_TARGET=10.14 BINARYNINJADIR=${BN_CORE_OUTPUT_DIR}
+ ${RUSTUP_COMMAND} --target=aarch64-apple-darwin ${CARGO_OPTS}
+ COMMAND ${CMAKE_COMMAND} -E env
+ MACOSX_DEPLOYMENT_TARGET=10.14 BINARYNINJADIR=${BN_CORE_OUTPUT_DIR}
+ ${RUSTUP_COMMAND} --target=x86_64-apple-darwin ${CARGO_OPTS}
+ COMMAND mkdir -p ${TARGET_DIR}
+ COMMAND lipo -create ${AARCH64_LIB_PATH} ${X86_64_LIB_PATH} -output ${PLUGIN_PATH}
+ COMMAND ${CMAKE_COMMAND} -E copy ${PLUGIN_PATH} ${BN_CORE_PLUGIN_DIR}
+ WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
+ DEPENDS ${PLUGIN_SOURCES} ${API_SOURCES})
+ else()
+ if(CMAKE_BUILD_TYPE MATCHES Debug)
+ set(LIB_PATH ${PROJECT_BINARY_DIR}/target/debug/${OUTPUT_FILE})
+ else()
+ set(LIB_PATH ${PROJECT_BINARY_DIR}/target/release/${OUTPUT_FILE})
+ endif()
+
+ add_custom_command(
+ OUTPUT ${PLUGIN_PATH}
+ COMMAND ${CMAKE_COMMAND} -E env MACOSX_DEPLOYMENT_TARGET=10.14 BINARYNINJADIR=${BN_CORE_OUTPUT_DIR} ${RUSTUP_COMMAND} ${CARGO_OPTS}
+ COMMAND ${CMAKE_COMMAND} -E copy ${PLUGIN_PATH} ${BN_CORE_PLUGIN_DIR}
+ WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
+ DEPENDS ${PLUGIN_SOURCES} ${API_SOURCES})
+ endif()
+elseif(WIN32)
+ add_custom_command(
+ OUTPUT ${PLUGIN_PATH}
+ COMMAND ${CMAKE_COMMAND} -E env BINARYNINJADIR=${BN_CORE_OUTPUT_DIR} ${RUSTUP_COMMAND} ${CARGO_OPTS}
+ COMMAND ${CMAKE_COMMAND} -E copy ${PLUGIN_PATH} ${BN_CORE_PLUGIN_DIR}
+ COMMAND ${CMAKE_COMMAND} -E copy ${TARGET_DIR}/${OUTPUT_PDB_NAME} ${BN_CORE_PLUGIN_DIR}
+ WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
+ DEPENDS ${PLUGIN_SOURCES} ${API_SOURCES})
+else()
+ add_custom_command(
+ OUTPUT ${PLUGIN_PATH}
+ COMMAND ${CMAKE_COMMAND} -E env BINARYNINJADIR=${BN_CORE_OUTPUT_DIR} ${RUSTUP_COMMAND} ${CARGO_OPTS}
+ COMMAND ${CMAKE_COMMAND} -E copy ${PLUGIN_PATH} ${BN_CORE_PLUGIN_DIR}
+ WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
+ DEPENDS ${PLUGIN_SOURCES} ${API_SOURCES})
+endif() \ No newline at end of file
diff --git a/plugins/idb_import/Cargo.toml b/plugins/idb_import/Cargo.toml
new file mode 100644
index 00000000..66119081
--- /dev/null
+++ b/plugins/idb_import/Cargo.toml
@@ -0,0 +1,15 @@
+[package]
+name = "idb_import"
+version = "0.1.0"
+authors = ["Rubens Brandao <git@rubens.io>"]
+edition = "2021"
+
+[lib]
+crate-type = ["cdylib"]
+
+[dependencies]
+anyhow = { version = "1.0.86", features = ["backtrace"] }
+binaryninja.workspace = true
+binaryninjacore-sys.workspace = true
+idb-rs = { git = "https://github.com/Vector35/idb-rs", tag = "0.1.6" }
+log = "0.4" \ No newline at end of file
diff --git a/plugins/idb_import/build.rs b/plugins/idb_import/build.rs
new file mode 100644
index 00000000..ed6cec7d
--- /dev/null
+++ b/plugins/idb_import/build.rs
@@ -0,0 +1,15 @@
+fn main() {
+ let link_path = std::env::var_os("DEP_BINARYNINJACORE_PATH")
+ .expect("DEP_BINARYNINJACORE_PATH not specified");
+
+ println!("cargo::rustc-link-lib=dylib=binaryninjacore");
+ println!("cargo::rustc-link-search={}", link_path.to_str().unwrap());
+
+ #[cfg(not(target_os = "windows"))]
+ {
+ println!(
+ "cargo::rustc-link-arg=-Wl,-rpath,{0},-L{0}",
+ link_path.to_string_lossy()
+ );
+ }
+}
diff --git a/plugins/idb_import/src/addr_info.rs b/plugins/idb_import/src/addr_info.rs
new file mode 100644
index 00000000..8d1b1c1b
--- /dev/null
+++ b/plugins/idb_import/src/addr_info.rs
@@ -0,0 +1,57 @@
+use std::collections::HashMap;
+
+use anyhow::Result;
+
+use idb_rs::id0::ID0Section;
+use idb_rs::til;
+
+#[derive(Default)]
+pub struct AddrInfo<'a> {
+ // TODO does binja diferenciate comments types on the API?
+ pub comments: Vec<&'a [u8]>,
+ pub label: Option<&'a str>,
+ // TODO make this a ref
+ pub ty: Option<til::Type>,
+}
+
+pub fn get_info(id0: &ID0Section, version: u16) -> Result<HashMap<u64, AddrInfo<'_>>> {
+ let mut addr_info: HashMap<u64, AddrInfo> = HashMap::new();
+
+ // the old style comments, most likely empty on new versions
+ let old_comments = id0.functions_and_comments()?.filter_map(|fc| {
+ use idb_rs::id0::FunctionsAndComments::*;
+ match fc {
+ Err(e) => Some(Err(e)),
+ Ok(Comment { address, comment }) => Some(Ok((address, comment))),
+ Ok(Name | Function(_) | Unknown { .. }) => None,
+ }
+ });
+ for old_comment in old_comments {
+ let (addr, comment) = old_comment?;
+ let comment = comment.message();
+ addr_info.entry(addr).or_default().comments.push(comment);
+ }
+
+ // comments defined on the address information
+ for info in id0.address_info(version)? {
+ use idb_rs::id0::AddressInfo::*;
+ let (addr, info) = info?;
+ let entry = addr_info.entry(addr).or_default();
+ match info {
+ Comment(comments) => entry.comments.push(comments.message()),
+ Label(name) => {
+ if let Some(_old) = entry.label.replace(name) {
+ panic!("Duplicated label for an address should be impossible this is most likelly a programing error")
+ }
+ }
+ TilType(ty) => {
+ if let Some(_old) = entry.ty.replace(ty) {
+ panic!("Duplicated type for an address should be impossible this is most likelly a programing error")
+ }
+ }
+ Other { .. } => {}
+ }
+ }
+
+ Ok(addr_info)
+}
diff --git a/plugins/idb_import/src/lib.rs b/plugins/idb_import/src/lib.rs
new file mode 100644
index 00000000..ddd22ea5
--- /dev/null
+++ b/plugins/idb_import/src/lib.rs
@@ -0,0 +1,344 @@
+mod types;
+use types::*;
+mod addr_info;
+use addr_info::*;
+
+use binaryninja::binary_view::{BinaryView, BinaryViewBase, BinaryViewExt};
+use binaryninja::debuginfo::{
+ CustomDebugInfoParser, DebugFunctionInfo, DebugInfo, DebugInfoParser,
+};
+
+use idb_rs::id0::{ID0Section, IDBParam1, IDBParam2};
+use idb_rs::til::section::TILSection;
+use idb_rs::til::Type as TILType;
+
+use log::{error, trace, warn, LevelFilter};
+
+use anyhow::Result;
+use binaryninja::logger::Logger;
+
+struct IDBDebugInfoParser;
+impl CustomDebugInfoParser for IDBDebugInfoParser {
+ fn is_valid(&self, view: &BinaryView) -> bool {
+ if let Some(project_file) = view.file().project_file() {
+ project_file.name().as_str().ends_with(".i64")
+ || project_file.name().as_str().ends_with(".idb")
+ } else {
+ view.file().filename().as_str().ends_with(".i64")
+ || view.file().filename().as_str().ends_with(".idb")
+ }
+ }
+
+ fn parse_info(
+ &self,
+ debug_info: &mut DebugInfo,
+ bv: &BinaryView,
+ debug_file: &BinaryView,
+ progress: Box<dyn Fn(usize, usize) -> Result<(), ()>>,
+ ) -> bool {
+ match parse_idb_info(debug_info, bv, debug_file, progress) {
+ Ok(()) => true,
+ Err(error) => {
+ error!("Unable to parse IDB file: {error}");
+ false
+ }
+ }
+ }
+}
+
+struct TILDebugInfoParser;
+impl CustomDebugInfoParser for TILDebugInfoParser {
+ fn is_valid(&self, view: &BinaryView) -> bool {
+ if let Some(project_file) = view.file().project_file() {
+ project_file.name().as_str().ends_with(".til")
+ } else {
+ view.file().filename().as_str().ends_with(".til")
+ }
+ }
+
+ fn parse_info(
+ &self,
+ debug_info: &mut DebugInfo,
+ _bv: &BinaryView,
+ debug_file: &BinaryView,
+ progress: Box<dyn Fn(usize, usize) -> Result<(), ()>>,
+ ) -> bool {
+ match parse_til_info(debug_info, debug_file, progress) {
+ Ok(()) => true,
+ Err(error) => {
+ error!("Unable to parse TIL file: {error}");
+ false
+ }
+ }
+ }
+}
+
+struct BinaryViewReader<'a> {
+ bv: &'a BinaryView,
+ offset: u64,
+}
+impl std::io::Read for BinaryViewReader<'_> {
+ fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
+ if !self.bv.offset_valid(self.offset) {
+ return Err(std::io::Error::new(std::io::ErrorKind::UnexpectedEof, ""));
+ }
+ let len = self.bv.read(buf, self.offset);
+ self.offset += u64::try_from(len).unwrap();
+ Ok(len)
+ }
+}
+
+impl std::io::Seek for BinaryViewReader<'_> {
+ fn seek(&mut self, pos: std::io::SeekFrom) -> std::io::Result<u64> {
+ let new_offset = match pos {
+ std::io::SeekFrom::Start(offset) => Some(offset),
+ std::io::SeekFrom::End(end) => u64::try_from(self.bv.len())
+ .unwrap()
+ .checked_add_signed(end),
+ std::io::SeekFrom::Current(next) => self.offset.checked_add_signed(next),
+ };
+ let new_offset =
+ new_offset.ok_or_else(|| std::io::Error::new(std::io::ErrorKind::UnexpectedEof, ""))?;
+ if !self.bv.offset_valid(new_offset) {
+ return Err(std::io::Error::new(std::io::ErrorKind::UnexpectedEof, ""));
+ }
+ self.offset = new_offset;
+ Ok(new_offset)
+ }
+}
+
+fn parse_idb_info(
+ debug_info: &mut DebugInfo,
+ bv: &BinaryView,
+ debug_file: &BinaryView,
+ progress: Box<dyn Fn(usize, usize) -> Result<(), ()>>,
+) -> Result<()> {
+ trace!("Opening a IDB file");
+ let file = BinaryViewReader {
+ bv: debug_file,
+ offset: 0,
+ };
+ trace!("Parsing a IDB file");
+ let file = std::io::BufReader::new(file);
+ let mut parser = idb_rs::IDBParser::new(file)?;
+ if let Some(til_section) = parser.til_section_offset() {
+ trace!("Parsing the TIL section");
+ let til = parser.read_til_section(til_section)?;
+ // progress 0%-50%
+ import_til_section(debug_info, debug_file, &til, progress)?;
+ }
+
+ if let Some(id0_section) = parser.id0_section_offset() {
+ trace!("Parsing the ID0 section");
+ let id0 = parser.read_id0_section(id0_section)?;
+ // progress 50%-100%
+ parse_id0_section_info(debug_info, bv, debug_file, &id0)?;
+ }
+
+ Ok(())
+}
+
+fn parse_til_info(
+ debug_info: &mut DebugInfo,
+ debug_file: &BinaryView,
+ progress: Box<dyn Fn(usize, usize) -> Result<(), ()>>,
+) -> Result<()> {
+ trace!("Opening a TIL file");
+ let file = BinaryViewReader {
+ bv: debug_file,
+ offset: 0,
+ };
+ let file = std::io::BufReader::new(file);
+ trace!("Parsing the TIL section");
+ let til = TILSection::parse(file)?;
+ import_til_section(debug_info, debug_file, &til, progress)
+}
+
+pub fn import_til_section(
+ debug_info: &mut DebugInfo,
+ debug_file: &BinaryView,
+ til: &TILSection,
+ progress: impl Fn(usize, usize) -> Result<(), ()>,
+) -> Result<()> {
+ let types = types::translate_til_types(debug_file.default_arch().unwrap(), til, progress)?;
+
+ // print any errors
+ for ty in &types {
+ match &ty.ty {
+ TranslateTypeResult::NotYet => {
+ panic!(
+ "type could not be processed `{}`: {:#?}",
+ &String::from_utf8_lossy(&ty.name),
+ &ty.og_ty
+ );
+ }
+ TranslateTypeResult::Error(error) => {
+ error!(
+ "Unable to parse type `{}`: {error}",
+ &String::from_utf8_lossy(&ty.name)
+ );
+ }
+ TranslateTypeResult::PartiallyTranslated(_, error) => {
+ if let Some(error) = error {
+ error!(
+ "Unable to parse type `{}` correctly: {error}",
+ &String::from_utf8_lossy(&ty.name)
+ );
+ } else {
+ warn!(
+ "Type `{}` maybe not be fully translated",
+ &String::from_utf8_lossy(&ty.name)
+ );
+ }
+ }
+ TranslateTypeResult::Translated(_) => {}
+ };
+ }
+
+ // add all type to binary ninja
+ for ty in &types {
+ if let TranslateTypeResult::Translated(bn_ty)
+ | TranslateTypeResult::PartiallyTranslated(bn_ty, _) = &ty.ty
+ {
+ if !debug_info.add_type(String::from_utf8_lossy(&ty.name), bn_ty, &[/* TODO */]) {
+ error!(
+ "Unable to add type `{}`",
+ &String::from_utf8_lossy(&ty.name)
+ )
+ }
+ }
+ }
+
+ // add a second time to fix the references LOL
+ for ty in &types {
+ if let TranslateTypeResult::Translated(bn_ty)
+ | TranslateTypeResult::PartiallyTranslated(bn_ty, _) = &ty.ty
+ {
+ if !debug_info.add_type(String::from_utf8_lossy(&ty.name), bn_ty, &[/* TODO */]) {
+ error!(
+ "Unable to fix type `{}`",
+ &String::from_utf8_lossy(&ty.name)
+ )
+ }
+ }
+ }
+
+ Ok(())
+}
+
+fn parse_id0_section_info(
+ debug_info: &mut DebugInfo,
+ bv: &BinaryView,
+ debug_file: &BinaryView,
+ id0: &ID0Section,
+) -> Result<()> {
+ let version = match id0.ida_info()? {
+ idb_rs::id0::IDBParam::V1(IDBParam1 { version, .. })
+ | idb_rs::id0::IDBParam::V2(IDBParam2 { version, .. }) => version,
+ };
+
+ for (addr, info) in get_info(id0, version)? {
+ // just in case we change this struct in the future, this line will for us to review this code
+ // TODO merge this data with folder locations
+ let AddrInfo {
+ comments,
+ label,
+ ty,
+ } = info;
+ // TODO set comments to address here
+ for function in &bv.functions_containing(addr) {
+ function.set_comment_at(
+ addr,
+ String::from_utf8_lossy(&comments.join(&b"\n"[..])).to_string(),
+ );
+ }
+
+ let bnty = ty
+ .as_ref()
+ .and_then(|ty| match translate_ephemeral_type(debug_file, ty) {
+ TranslateTypeResult::Translated(result) => Some(result),
+ TranslateTypeResult::PartiallyTranslated(result, None) => {
+ warn!("Unable to fully translate the type at {addr:#x}");
+ Some(result)
+ }
+ TranslateTypeResult::NotYet => {
+ error!("Unable to translate the type at {addr:#x}");
+ None
+ }
+ TranslateTypeResult::PartiallyTranslated(_, Some(bn_type_error))
+ | TranslateTypeResult::Error(bn_type_error) => {
+ error!("Unable to translate the type at {addr:#x}: {bn_type_error}",);
+ None
+ }
+ });
+
+ match (label, &ty, bnty) {
+ (_, Some(TILType::Function(_)), bnty) => {
+ if bnty.is_none() {
+ error!("Unable to convert the function type at {addr:#x}",)
+ }
+ if !debug_info.add_function(DebugFunctionInfo::new(
+ None,
+ None,
+ label.map(str::to_string),
+ bnty,
+ Some(addr),
+ None,
+ vec![],
+ vec![],
+ )) {
+ error!("Unable to add the function at {addr:#x}")
+ }
+ }
+ (_, Some(_ty), Some(bnty)) => {
+ if !debug_info.add_data_variable(addr, &bnty, label, &[]) {
+ error!("Unable to add the type at {addr:#x}")
+ }
+ }
+ (_, Some(_ty), None) => {
+ // TODO types come from the TIL sections, can we make all types be just NamedTypes?
+ error!("Unable to convert type {addr:#x}");
+ // TODO how to add a label without a type associacted with it?
+ if let Some(name) = label {
+ if !debug_info.add_data_variable(
+ addr,
+ &binaryninja::types::Type::void(),
+ Some(name),
+ &[],
+ ) {
+ error!("Unable to add the label at {addr:#x}")
+ }
+ }
+ }
+ (Some(name), None, None) => {
+ // TODO how to add a label without a type associacted with it?
+ if !debug_info.add_data_variable(
+ addr,
+ &binaryninja::types::Type::void(),
+ Some(name),
+ &[],
+ ) {
+ error!("Unable to add the label at {addr:#x}")
+ }
+ }
+
+ // just comments at this address
+ (None, None, None) => {}
+
+ (_, None, Some(_)) => unreachable!(),
+ }
+ }
+
+ Ok(())
+}
+
+#[allow(non_snake_case)]
+#[no_mangle]
+pub extern "C" fn CorePluginInit() -> bool {
+ Logger::new("IDB Import")
+ .with_level(LevelFilter::Error)
+ .init();
+ DebugInfoParser::register("IDB Parser", IDBDebugInfoParser);
+ DebugInfoParser::register("TIL Parser", TILDebugInfoParser);
+ true
+}
diff --git a/plugins/idb_import/src/types.rs b/plugins/idb_import/src/types.rs
new file mode 100644
index 00000000..0a704241
--- /dev/null
+++ b/plugins/idb_import/src/types.rs
@@ -0,0 +1,700 @@
+use std::collections::HashMap;
+
+use anyhow::{anyhow, Result};
+use binaryninja::architecture::CoreArchitecture;
+use binaryninja::binary_view::{BinaryView, BinaryViewExt};
+use binaryninja::confidence::Conf;
+use binaryninja::rc::Ref;
+use binaryninja::types::{
+ EnumerationBuilder, FunctionParameter, MemberAccess, MemberScope, StructureBuilder,
+ StructureType, Type,
+};
+use idb_rs::til::{
+ array::Array as TILArray, function::Function as TILFunction, r#enum::Enum as TILEnum,
+ r#struct::Struct as TILStruct, r#struct::StructMember as TILStructMember, section::TILSection,
+ union::Union as TILUnion, TILTypeInfo, Type as TILType, Typedef as TILTypedef,
+};
+
+#[derive(Debug, Clone)]
+pub enum BnTypeError {
+ // TODO delete this and make this verification during the TIL/IDB parsing, translating the ordinal
+ // into a kind of type_idx
+ OrdinalNotFound(u32),
+ NameNotFound(String),
+
+ Typedef(Box<BnTypeError>),
+ Function(BnTypeFunctionError),
+ Array(Box<BnTypeError>),
+ Pointer(Box<BnTypeError>),
+ /// Error for members
+ Struct(Vec<(usize, BnTypeError)>),
+ Union(Vec<(usize, BnTypeError)>),
+}
+
+#[derive(Default, Debug, Clone)]
+pub struct BnTypeFunctionError {
+ pub ret: Option<Box<BnTypeError>>,
+ pub args: Vec<(usize, BnTypeError)>,
+}
+
+impl std::fmt::Display for BnTypeError {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ match self {
+ BnTypeError::OrdinalNotFound(i) => write!(f, "Reference to non existing Ordinal {i}"),
+ BnTypeError::NameNotFound(name) => write!(f, "Reference to non existing name {name}"),
+ BnTypeError::Typedef(error) => write!(f, "Typedef: {error}"),
+ BnTypeError::Function(BnTypeFunctionError { ret, args }) => {
+ if let Some(error) = ret {
+ write!(f, "Function return: {error} ")?;
+ }
+ for (i, error) in args {
+ write!(f, "Function argument {i}: {error} ")?;
+ }
+ Ok(())
+ }
+ BnTypeError::Array(error) => write!(f, "Array: {error}"),
+ BnTypeError::Struct(errors) => {
+ for (i, error) in errors {
+ write!(f, "Struct Member {i}: {error} ")?;
+ }
+ Ok(())
+ }
+ BnTypeError::Union(errors) => {
+ for (i, error) in errors {
+ write!(f, "Union Member {i}: {error} ")?;
+ }
+ Ok(())
+ }
+ BnTypeError::Pointer(error) => write!(f, "Pointer: {error}"),
+ }
+ }
+}
+
+#[derive(Default)]
+pub enum TranslateTypeResult {
+ #[default]
+ NotYet,
+ /// Unable to solve type, there is no point in trying again
+ Error(BnTypeError),
+ /// a type that is not final, but equivalent to the final type, if error, there is no
+ /// point in trying again
+ PartiallyTranslated(Ref<Type>, Option<BnTypeError>),
+ Translated(Ref<Type>),
+}
+
+impl From<Result<Ref<Type>, BnTypeError>> for TranslateTypeResult {
+ fn from(value: Result<Ref<Type>, BnTypeError>) -> Self {
+ match value {
+ Ok(ty) => Self::Translated(ty),
+ Err(error) => Self::Error(error),
+ }
+ }
+}
+
+pub struct TranslatesIDBType<'a> {
+ // sanitized name from IDB
+ pub name: Vec<u8>,
+ // the result, if converted
+ pub ty: TranslateTypeResult,
+ pub og_ty: &'a TILTypeInfo,
+ pub is_symbol: bool,
+}
+
+pub struct TranslateIDBTypes<'a, F: Fn(usize, usize) -> Result<(), ()>> {
+ pub arch: CoreArchitecture,
+ pub progress: F,
+ pub til: &'a TILSection,
+ // note it's mapped 1:1 with the same index from til types.chain(symbols)
+ pub types: Vec<TranslatesIDBType<'a>>,
+ // ordinals with index to types
+ pub types_by_ord: HashMap<u64, usize>,
+ // original names with index to types
+ pub types_by_name: HashMap<Vec<u8>, usize>,
+}
+
+impl<F: Fn(usize, usize) -> Result<(), ()>> TranslateIDBTypes<'_, F> {
+ fn find_typedef_by_ordinal(&self, ord: u64) -> Option<TranslateTypeResult> {
+ self.types_by_ord
+ .get(&ord)
+ .map(|idx| self.find_typedef(&self.types[*idx]))
+ }
+
+ fn find_typedef_by_name(&self, name: &[u8]) -> Option<TranslateTypeResult> {
+ if name.is_empty() {
+ // TODO this is my assumption, maybe an empty names Typedef means something else.
+ return Some(TranslateTypeResult::Translated(Type::void()));
+ }
+
+ if let Some(other_ty) = self
+ .types_by_name
+ .get(name)
+ .map(|idx| self.find_typedef(&self.types[*idx]))
+ {
+ return Some(other_ty);
+ }
+
+ // check for types that ar usually not defined directly
+ match name {
+ b"Unkown" | b"uint8_t" => Some(TranslateTypeResult::Translated(Type::int(1, false))),
+ b"IUnkown" | b"int8_t" => Some(TranslateTypeResult::Translated(Type::int(1, true))),
+ b"SHORT" | b"USHORT" => Some(TranslateTypeResult::Translated(Type::int(
+ self.til.size_short.get() as usize,
+ name == b"SHORT",
+ ))),
+ b"int16_t" => Some(TranslateTypeResult::Translated(Type::int(2, true))),
+ b"uint16_t" => Some(TranslateTypeResult::Translated(Type::int(2, false))),
+ b"int32_t" => Some(TranslateTypeResult::Translated(Type::int(4, true))),
+ b"uint32_t" => Some(TranslateTypeResult::Translated(Type::int(4, false))),
+ b"int64_t" => Some(TranslateTypeResult::Translated(Type::int(8, true))),
+ b"uint64_t" => Some(TranslateTypeResult::Translated(Type::int(8, false))),
+ b"int128_t" => Some(TranslateTypeResult::Translated(Type::int(16, true))),
+ b"uint128_t" => Some(TranslateTypeResult::Translated(Type::int(16, false))),
+ _ => None,
+ }
+ }
+
+ fn find_typedef(&self, ty: &TranslatesIDBType) -> TranslateTypeResult {
+ // only return a typedef, if it's solved, at least partially
+ match &ty.ty {
+ TranslateTypeResult::NotYet => TranslateTypeResult::NotYet,
+ TranslateTypeResult::Error(error) => {
+ TranslateTypeResult::Error(BnTypeError::Typedef(Box::new(error.to_owned())))
+ }
+ TranslateTypeResult::PartiallyTranslated(og_ty, error) => {
+ TranslateTypeResult::PartiallyTranslated(
+ Type::named_type_from_type(String::from_utf8_lossy(&ty.name), og_ty),
+ error
+ .as_ref()
+ .map(|x| BnTypeError::Typedef(Box::new(x.clone())))
+ .clone(),
+ )
+ }
+ TranslateTypeResult::Translated(og_ty) => TranslateTypeResult::Translated(
+ Type::named_type_from_type(String::from_utf8_lossy(&ty.name), og_ty),
+ ),
+ }
+ }
+
+ fn translate_pointer(&self, ty: &TILType) -> TranslateTypeResult {
+ match self.translate_type(ty) {
+ TranslateTypeResult::Translated(trans) => {
+ TranslateTypeResult::Translated(Type::pointer(&self.arch, &trans))
+ }
+ TranslateTypeResult::PartiallyTranslated(trans, error) => {
+ TranslateTypeResult::PartiallyTranslated(
+ Type::pointer(&self.arch, &trans),
+ error.map(|e| BnTypeError::Pointer(Box::new(e))),
+ )
+ }
+ TranslateTypeResult::Error(error) => TranslateTypeResult::PartiallyTranslated(
+ Type::pointer(&self.arch, &Type::void()),
+ Some(error),
+ ),
+ TranslateTypeResult::NotYet => TranslateTypeResult::PartiallyTranslated(
+ Type::pointer(&self.arch, &Type::void()),
+ None,
+ ),
+ }
+ }
+
+ fn translate_function(&self, fun: &TILFunction) -> TranslateTypeResult {
+ let mut is_partial = false;
+ let mut errors: BnTypeFunctionError = Default::default();
+ // funtions are always 0 len, so it's translated or partial(void)
+ let return_ty = match self.translate_type(&fun.ret) {
+ TranslateTypeResult::Translated(trans) => trans,
+ TranslateTypeResult::PartiallyTranslated(trans, error) => {
+ is_partial |= true;
+ errors.ret = error.map(Box::new);
+ trans
+ }
+ TranslateTypeResult::Error(error) => {
+ errors.ret = Some(Box::new(error));
+ return TranslateTypeResult::PartiallyTranslated(
+ Type::void(),
+ Some(BnTypeError::Function(errors)),
+ );
+ }
+ TranslateTypeResult::NotYet => {
+ return TranslateTypeResult::PartiallyTranslated(Type::void(), None)
+ }
+ };
+ let mut partial_error_args = vec![];
+ let mut bn_args = Vec::with_capacity(fun.args.len());
+ for (i, (arg_name, arg_type, _arg_loc)) in fun.args.iter().enumerate() {
+ let arg = match self.translate_type(arg_type) {
+ TranslateTypeResult::Translated(trans) => trans,
+ TranslateTypeResult::PartiallyTranslated(trans, error) => {
+ is_partial = true;
+ if let Some(error) = error {
+ errors.args.push((i, error));
+ }
+ trans
+ }
+ TranslateTypeResult::NotYet => {
+ return TranslateTypeResult::PartiallyTranslated(Type::void(), None)
+ }
+ TranslateTypeResult::Error(error) => {
+ partial_error_args.push((i, error));
+ return TranslateTypeResult::PartiallyTranslated(
+ Type::void(),
+ Some(BnTypeError::Function(errors)),
+ );
+ }
+ };
+ // TODO create location from `arg_loc`?
+ let loc = None;
+ let name = arg_name
+ .as_ref()
+ .map(|name| String::from_utf8_lossy(name).to_string())
+ .unwrap_or_else(|| format!("arg_{i}"));
+ bn_args.push(FunctionParameter::new(arg, name, loc));
+ }
+
+ let ty = Type::function(&return_ty, bn_args, false);
+ if is_partial {
+ let error = (errors.ret.is_some() || !errors.args.is_empty())
+ .then_some(BnTypeError::Function(errors));
+ TranslateTypeResult::PartiallyTranslated(ty, error)
+ } else {
+ assert!(errors.ret.is_none() && errors.args.is_empty());
+ TranslateTypeResult::Translated(ty)
+ }
+ }
+
+ fn translate_array(&self, array: &TILArray) -> TranslateTypeResult {
+ match self.translate_type(&array.elem_type) {
+ TranslateTypeResult::NotYet => TranslateTypeResult::NotYet,
+ TranslateTypeResult::Translated(ty) => {
+ TranslateTypeResult::Translated(Type::array(&ty, array.nelem.into()))
+ }
+ TranslateTypeResult::PartiallyTranslated(ty, error) => {
+ TranslateTypeResult::PartiallyTranslated(
+ Type::array(&ty, array.nelem.into()),
+ error.map(Box::new).map(BnTypeError::Array),
+ )
+ }
+ TranslateTypeResult::Error(error) => {
+ TranslateTypeResult::Error(BnTypeError::Array(Box::new(error)))
+ }
+ }
+ }
+
+ fn condensate_bitfields_from_struct(
+ &self,
+ offset: usize,
+ members_slice: &[TILStructMember],
+ struct_builder: &mut StructureBuilder,
+ ) {
+ if members_slice.is_empty() {
+ unreachable!()
+ }
+ let mut members = members_slice
+ .iter()
+ .map(|ty| match &ty.member_type {
+ TILType::Bitfield(b) => b,
+ _ => unreachable!(),
+ })
+ .enumerate();
+ let (_, first_field) = members.next().unwrap();
+ let mut current_field_bytes = first_field.nbytes;
+ let mut current_field_bits: u32 = first_field.width.into();
+ let mut start_idx = 0;
+
+ let mut create_field = |start_idx, i, bytes| {
+ let name = if start_idx == i - 1 {
+ let member: &TILStructMember = &members_slice[i - 1];
+ member
+ .name
+ .as_ref()
+ .map(|name| String::from_utf8_lossy(name).to_string())
+ .unwrap_or_else(|| format!("bitfield_{}", offset + start_idx))
+ } else {
+ format!("bitfield_{}_{}", offset + start_idx, offset + (i - 1))
+ };
+ let field = field_from_bytes(bytes);
+ struct_builder.append(&field, name, MemberAccess::NoAccess, MemberScope::NoScope);
+ };
+
+ for (i, member) in members {
+ // starting a new field
+ let max_bits = u32::try_from(current_field_bytes).unwrap() * 8;
+ // this bitfield start a a new field, or can't contain other bitfields
+ // finish the previous and start a new
+ if current_field_bytes != member.nbytes
+ || max_bits < current_field_bits + u32::from(member.width)
+ {
+ create_field(start_idx, i, current_field_bytes);
+ current_field_bytes = member.nbytes;
+ current_field_bits = 0;
+ start_idx = i;
+ }
+
+ // just add the current bitfield into the field
+ current_field_bits += u32::from(member.width);
+ }
+
+ if current_field_bits != 0 {
+ create_field(start_idx, members_slice.len(), current_field_bytes);
+ }
+ }
+
+ fn translate_struct(
+ &self,
+ members: &[TILStructMember],
+ effective_alignment: u16,
+ ) -> TranslateTypeResult {
+ if members.is_empty() {
+ // binary ninja crashes if you create an empty struct, because it divide by 0
+ return TranslateTypeResult::Translated(Type::void());
+ }
+ let mut is_partial = false;
+ let mut structure = StructureBuilder::new();
+ structure.alignment(effective_alignment.into());
+
+ let mut errors = vec![];
+ let mut first_bitfield_seq = None;
+ for (i, member) in members.iter().enumerate() {
+ match (&member.member_type, first_bitfield_seq) {
+ // accumulate the bitfield to be condensated
+ (TILType::Bitfield(_bit), None) => {
+ first_bitfield_seq = Some(i);
+ continue;
+ }
+ (TILType::Bitfield(_bit), Some(_)) => continue,
+
+ // condensate the bitfields into byte-wide fields
+ (_, Some(start_idx)) => {
+ first_bitfield_seq = None;
+ let members_bitrange = &members[start_idx..i];
+ self.condensate_bitfields_from_struct(
+ start_idx,
+ members_bitrange,
+ &mut structure,
+ );
+ }
+
+ (_, None) => {}
+ }
+
+ let mem = match self.translate_type(&member.member_type) {
+ TranslateTypeResult::Translated(ty) => ty,
+ TranslateTypeResult::PartiallyTranslated(partial_ty, error) => {
+ is_partial |= true;
+ if let Some(error) = error {
+ errors.push((i, error));
+ }
+ partial_ty
+ }
+ TranslateTypeResult::NotYet => return TranslateTypeResult::NotYet,
+ TranslateTypeResult::Error(error) => {
+ errors.push((i, error));
+ return TranslateTypeResult::Error(BnTypeError::Struct(errors));
+ }
+ };
+ let name = member
+ .name
+ .as_ref()
+ .map(|name| String::from_utf8_lossy(name).to_string())
+ .unwrap_or_else(|| format!("member_{i}"));
+ structure.append(&mem, name, MemberAccess::NoAccess, MemberScope::NoScope);
+ }
+ if let Some(start_idx) = first_bitfield_seq {
+ let members_bitrange = &members[start_idx..];
+ self.condensate_bitfields_from_struct(start_idx, members_bitrange, &mut structure);
+ }
+ let bn_ty = Type::structure(&structure.finalize());
+ if is_partial {
+ let partial_error = (!errors.is_empty()).then_some(BnTypeError::Struct(errors));
+ TranslateTypeResult::PartiallyTranslated(bn_ty, partial_error)
+ } else {
+ assert!(errors.is_empty());
+ TranslateTypeResult::Translated(bn_ty)
+ }
+ }
+
+ fn translate_union(
+ &self,
+ members: &[(Option<Vec<u8>>, TILType)],
+ _effective_alignment: u16,
+ ) -> TranslateTypeResult {
+ let mut is_partial = false;
+ let mut structure = StructureBuilder::new();
+ structure.structure_type(StructureType::UnionStructureType);
+ let mut errors = vec![];
+ for (i, (member_name, member_type)) in members.iter().enumerate() {
+ // bitfields can be translated into complete fields
+ let mem = match member_type {
+ TILType::Bitfield(field) => field_from_bytes(field.nbytes),
+ member_type => match self.translate_type(member_type) {
+ TranslateTypeResult::Translated(ty) => ty,
+ TranslateTypeResult::Error(error) => {
+ errors.push((i, error));
+ return TranslateTypeResult::Error(BnTypeError::Union(errors));
+ }
+ TranslateTypeResult::NotYet => return TranslateTypeResult::NotYet,
+ TranslateTypeResult::PartiallyTranslated(partial, error) => {
+ is_partial |= true;
+ if let Some(error) = error {
+ errors.push((i, error));
+ }
+ partial
+ }
+ },
+ };
+
+ let name = member_name
+ .as_ref()
+ .map(|name| String::from_utf8_lossy(name).to_string())
+ .unwrap_or_else(|| format!("member_{i}"));
+ structure.append(&mem, name, MemberAccess::NoAccess, MemberScope::NoScope);
+ }
+ let str_ref = structure.finalize();
+
+ let bn_ty = Type::structure(&str_ref);
+ if is_partial {
+ let partial_error = (!errors.is_empty()).then_some(BnTypeError::Struct(errors));
+ TranslateTypeResult::PartiallyTranslated(bn_ty, partial_error)
+ } else {
+ assert!(errors.is_empty());
+ TranslateTypeResult::Translated(bn_ty)
+ }
+ }
+
+ fn translate_enum(members: &[(Option<Vec<u8>>, u64)], bytesize: u64) -> Ref<Type> {
+ let eb = EnumerationBuilder::new();
+ for (i, (name, bytesize)) in members.iter().enumerate() {
+ let name = name
+ .as_ref()
+ .map(|name| String::from_utf8_lossy(name).to_string())
+ .unwrap_or_else(|| format!("member_{i}"));
+ eb.insert(name, *bytesize);
+ }
+ Type::enumeration(
+ &eb.finalize(),
+ // TODO: This looks bad, look at the comment in [`Type::width`].
+ (bytesize as usize).try_into().unwrap(),
+ Conf::new(false, 0),
+ )
+ }
+
+ fn translate_basic(mdata: &idb_rs::til::Basic) -> Ref<Type> {
+ match *mdata {
+ idb_rs::til::Basic::Void => Type::void(),
+ idb_rs::til::Basic::Unknown { bytes: 0 } => Type::void(),
+ idb_rs::til::Basic::Unknown { bytes } => Type::array(&Type::char(), bytes.into()),
+ idb_rs::til::Basic::Bool { bytes } if bytes.get() == 1 => Type::bool(),
+ // NOTE Binja don't have any representation for bool other then the default
+ idb_rs::til::Basic::Bool { bytes } => Type::int(bytes.get().into(), false),
+ idb_rs::til::Basic::Char => Type::char(),
+ // TODO what exacly is Segment Register?
+ idb_rs::til::Basic::SegReg => Type::char(),
+ idb_rs::til::Basic::Int { bytes, is_signed } => {
+ // default into signed
+ let is_signed = is_signed.as_ref().copied().unwrap_or(true);
+ Type::int(bytes.get().into(), is_signed)
+ }
+ idb_rs::til::Basic::Float { bytes } => Type::float(bytes.get().into()),
+ }
+ }
+
+ pub fn translate_type(&self, ty: &TILType) -> TranslateTypeResult {
+ match &ty {
+ // types that are always translatable
+ TILType::Basic(meta) => TranslateTypeResult::Translated(Self::translate_basic(meta)),
+ TILType::Bitfield(bit) => TranslateTypeResult::Translated(field_from_bytes(bit.nbytes)),
+ TILType::Enum(TILEnum::NonRef {
+ members, bytesize, ..
+ }) => TranslateTypeResult::Translated(Self::translate_enum(members, *bytesize)),
+ TILType::Typedef(TILTypedef::Ordinal(ord)) => self
+ .find_typedef_by_ordinal((*ord).into())
+ .unwrap_or_else(|| TranslateTypeResult::Error(BnTypeError::OrdinalNotFound(*ord))),
+ TILType::Typedef(TILTypedef::Name(name)) => {
+ self.find_typedef_by_name(name).unwrap_or_else(|| {
+ TranslateTypeResult::Error(BnTypeError::NameNotFound(
+ String::from_utf8_lossy(name).to_string(),
+ ))
+ })
+ }
+
+ // may not be translatable imediatly, but the size is known and can be
+ // updated after alBasicers are finished
+ TILType::Union(TILUnion::Ref { ref_type, .. })
+ | TILType::Struct(TILStruct::Ref { ref_type, .. })
+ | TILType::Enum(TILEnum::Ref { ref_type, .. }) => self.translate_pointer(ref_type),
+ TILType::Pointer(ty) => self.translate_pointer(&ty.typ),
+ TILType::Function(fun) => self.translate_function(fun),
+
+ // can only be partially solved if all fields are solved or partially solved
+ TILType::Array(array) => self.translate_array(array),
+ TILType::Struct(TILStruct::NonRef {
+ members,
+ effective_alignment,
+ ..
+ }) => self.translate_struct(members, *effective_alignment),
+ TILType::Union(TILUnion::NonRef {
+ members,
+ effective_alignment,
+ ..
+ }) => self.translate_union(members, *effective_alignment),
+ }
+ }
+}
+
+pub fn translate_ephemeral_type(debug_file: &BinaryView, ty: &TILType) -> TranslateTypeResult {
+ // in case we need to translate types
+ let translator = TranslateIDBTypes {
+ arch: debug_file.default_arch().unwrap(/* TODO */),
+ progress: |_, _| Ok(()),
+ // TODO it's unclear what to do here
+ til: &TILSection {
+ format: 12,
+ title: Vec::new(),
+ description: Vec::new(),
+ id: 0,
+ cm: 0,
+ def_align: 1,
+ symbols: vec![],
+ type_ordinal_alias: None,
+ types: vec![],
+ size_i: 4.try_into().unwrap(),
+ size_b: 1.try_into().unwrap(),
+ size_short: 2.try_into().unwrap(),
+ size_long: 4.try_into().unwrap(),
+ size_long_long: 8.try_into().unwrap(),
+ size_long_double: None,
+ macros: None,
+ is_universal: false,
+ },
+ types: vec![],
+ types_by_ord: HashMap::new(),
+ types_by_name: HashMap::new(),
+ };
+
+ translator.translate_type(ty)
+}
+
+pub fn translate_til_types(
+ arch: CoreArchitecture,
+ til: &TILSection,
+ progress: impl Fn(usize, usize) -> Result<(), ()>,
+) -> Result<Vec<TranslatesIDBType>> {
+ let total = til.symbols.len() + til.types.len();
+ let mut types = Vec::with_capacity(total);
+ let mut types_by_ord = HashMap::with_capacity(total);
+ let mut types_by_name = HashMap::with_capacity(total);
+ let all_types = til.types.iter().zip(core::iter::repeat(false));
+ // TODO: it's unclear how the demangle symbols and types names/ord, for now only parse types
+ //let all_types = all_types.chain(til.symbols.iter().zip(core::iter::repeat(true)));
+ for (i, (ty, is_symbol)) in all_types.enumerate() {
+ // TODO sanitized the input
+ // TODO find out how the namespaces used by TIL works
+ let name = ty.name.to_owned();
+ types.push(TranslatesIDBType {
+ name,
+ is_symbol,
+ og_ty: ty,
+ ty: TranslateTypeResult::NotYet,
+ });
+ if ty.ordinal != 0 && !is_symbol {
+ let dup1 = types_by_ord.insert(ty.ordinal, i);
+ if let Some(old) = dup1 {
+ let old_type = &types[old];
+ let new_type = types.last().unwrap();
+ // TODO error?
+ panic!(
+ "dup ord {}:{} {}:\n{:?}\n{:?}",
+ old_type.is_symbol,
+ new_type.is_symbol,
+ ty.ordinal,
+ &old_type.og_ty,
+ &new_type.og_ty,
+ )
+ }
+ }
+ if !ty.name.is_empty() {
+ let dup2 = types_by_name.insert(ty.name.to_owned(), i);
+ if let Some(old) = dup2 {
+ let old_type = &types[old];
+ let new_type = types.last().unwrap();
+ // TODO error?
+ panic!(
+ "dup name {}:{}: {}:\n{:?}\n{:?}",
+ old_type.is_symbol,
+ new_type.is_symbol,
+ &String::from_utf8_lossy(&ty.name),
+ &old_type.og_ty,
+ &new_type.og_ty,
+ )
+ }
+ }
+ }
+
+ let mut translator = TranslateIDBTypes {
+ arch,
+ progress,
+ til,
+ types,
+ types_by_ord,
+ types_by_name,
+ };
+ if (&translator.progress)(0, total).is_err() {
+ return Err(anyhow!("IDB import aborted"));
+ }
+
+ // solve types until there is nothing else that can be solved
+ loop {
+ // if something was solved, mark this variable as true
+ let mut did_something = false;
+ let mut num_translated = 0usize;
+ for i in 0..translator.types.len() {
+ match &translator.types[i].ty {
+ TranslateTypeResult::NotYet => {
+ let result = translator.translate_type(&translator.types[i].og_ty.tinfo);
+ did_something |= !matches!(&result, TranslateTypeResult::NotYet);
+ translator.types[i].ty = result;
+ }
+ TranslateTypeResult::PartiallyTranslated(_, None) => {
+ let result = translator.translate_type(&translator.types[i].og_ty.tinfo);
+ // don't allow regress, it can goes from PartiallyTranslated to any state other then NotYet
+ assert!(!matches!(&result, TranslateTypeResult::NotYet));
+ did_something |=
+ !matches!(&result, TranslateTypeResult::PartiallyTranslated(_, None));
+ translator.types[i].ty = result;
+ // don't need to add again they will be fixed on the loop below
+ }
+ // if an error was produced, there is no point in try again
+ TranslateTypeResult::PartiallyTranslated(_, Some(_)) => {}
+ // NOTE for now we are just accumulating errors, just try to translate the max number
+ // of types as possible
+ TranslateTypeResult::Error(_) => {}
+ // already translated, nothing do to here
+ TranslateTypeResult::Translated(_) => {}
+ }
+
+ // count the number of finished types
+ if let TranslateTypeResult::Translated(_) = &translator.types[i].ty {
+ num_translated += 1
+ }
+ }
+
+ if !did_something {
+ // means we acomplilshed nothing during this loop, there is no point in trying again
+ break;
+ }
+ if (&translator.progress)(num_translated, total).is_err() {
+ // error means the user aborted the progress
+ break;
+ }
+ }
+
+ Ok(translator.types)
+}
+
+fn field_from_bytes(bytes: i32) -> Ref<Type> {
+ match bytes {
+ 0 => unreachable!(),
+ num @ (1 | 2 | 4 | 8 | 16) => Type::int(num.try_into().unwrap(), false),
+ nelem => Type::array(&Type::char(), nelem.try_into().unwrap()),
+ }
+}