Problem with calling functions from a vtable
So I was trying to write a small wrapper lib for a dll and linking to the two exported functions works perfectly fine. So my current problem is that I get a vtable from this dll and I am also able to call the given functions of it, but this is only the case when the functions I am calling do not take any parameters. If I try to call a function of the vtable that takes any amount of parameters with the needed parameters the program ends up crashing due to an EXCEPTION_ACCESS_VIOLATION. The library is written in c++ so I was wondering if this could be the reason why I have problems, but afaik it shouldnt be. I cannot figure out what the problem is so I am asking here in hope that someone here know what my problem could be.
EDIT: My problem has been solved, my mistake was that I forgot to add the object itself as the first parameter to the function calls!
test.rs
extern crate gfxfilemanager;
use std::ptr::null_mut;
use std::os::raw::{c_char, c_int};
use std::ffi::{CStr, CString};
use gfxfilemanager::{IFileManager};
fn main() {
let mut object = IFileManager::new(1);//create the object that holds the vtable
unsafe {
use std::mem::transmute;
//the function signature of function0 is int Mode(void)
println!("{}", transmute::<*const (), extern "thiscall" fn() -> c_int>((*object).get_func(0) as *const ())());//this return 1, the value of IFileManager::new(1)
let name = CString::new("Test").unwrap();
let pw = CString::new("pw").unwrap();
//the function signature of function3 is int OpenContainer(const char *filename, const char* password, int mode)
let fnc = transmute::<*const (), extern "thiscall" fn(*const c_char, *const c_char) -> c_int>((*object).get_func(3) as *const ());
println!("{}", fnc(name.as_ptr(), pw.as_ptr()));//this crashes
}
}
lib.rs
use std::os::raw::{c_char, c_int};
#[repr(C)]
pub struct Vtable(::std::os::raw::c_void);
impl Vtable {
fn get_func(&self, index: isize) -> usize {
unsafe {
::std::ptr::read((self as *const _ as *const usize).offset(index))
}
}
}
#[repr(C)]
#[derive(Debug)]
pub struct IFileManager {
pub vtable: *const Vtable,
}
impl IFileManager {
pub fn new(mode: c_int) -> *mut IFileManager {
let mut obj = ::std::ptr::null_mut();
unsafe { GFXDllCreateObject(mode, &mut obj, 0x1007); }
obj
}
pub fn get_func(&self, index: isize) -> usize {
unsafe {
(*self.vtable).get_func(index)
}
}
}
impl Drop for IFileManager {
fn drop(&mut self) {
unsafe {
GFXDllReleaseObject(self);
}
}
}
#[link(name = "GFXFileManager")]
extern "stdcall" {
pub fn GFXDllCreateObject(mode: c_int, object: *mut *mut IFileManager, version: c_int) -> c_int;
pub fn GFXDllReleaseObject(object: *mut IFileManager) -> c_int;
}
Posted Jul 21, 2017, 9:49 PM UTC · edited