r/rust•
pmf

FFI

I have a question regarding FFI. I've used FFI in Python, Java (JNA and JNI) and LuaJIT, and LuaJIT's mechanism is just so much better than anything else (despite Lua being the language I have used the least before). I've created a gist here showing how to use the XInput library via LuaJIT's FFI.

In LuaJIT, you can just include the C header part, which very often works verbatim and even handles pack pragmas/atributes for structs correctly.

Why did Rust choose to handle FFI the way it does instead of allowing a similar import mechanism, given that it already includes the necessary LLVM tooling?

Posted Aug 28, 2015, 7:09 PM UTC

184
annodominirust•

LLVM doesn't include code for parsing C. What Rust uses is the LLVM backend that understands the various C calling conventions, while what you need for parsing C this way would be something like the Clang frontend.

As a matter of fact, there is the rust-bindgen project which does use the Clang frontend to parse C and generate Rust bindings.

For the most part, I think that the reason this is done the way it is is to keep the Rust compiler smaller and simpler. You don't need to provide a full C frontend as well; Rust just provides enough that you can manually write bindings, which is fine for simple APIs. For very large APIs, you can use third-party projects like rust-bindgen, without it having to be tied to the development cycle of the compiler, and without it complicating the basic language semantics and maintenance of the compiler.

15
tupshin•

Are you familiar with rust-bindgen?

While I like the simplicity of Lua's approach, I believ that bindgen can get you most of the way there. In particular, you might like the macro-based approach:

#![feature(phase)]
#[phase(plugin)] extern crate bindgen;

#[allow(dead_code, uppercase_variables, non_camel_case_types)]
mod mysql_bindings {
    bindgen!("/usr/include/mysql/mysql.h", match="mysql.h", link="mysql")
}

You can see an example of me wrapping the Cassandra C++ driver API in Rust here, consuming it here, and then writing a much more Rustic API here.

10
pmfOP•

Ah, the macro-based approach looks pretty much in line with how I can do things in LuaJIT!

Thanks!

3
ldpreload•

I haven't worked on many bindings, but of those that I have looked at, I haven't felt like having automatic reading of C headers would make things easier, for a few reasons:

  1. Rust doesn't have an equivalent of C unions (with no discriminant). The best you can do is cast one pointer/reference to another, but that doesn't help with alignment: there are things that do stuff like union { uint32_t x; struct { char a; char b; char c; char d; }; } with the assumption that the structure is aligned regardless of representation. So you need to handle this by hand anyway.
  2. Rust won't handle C macros, and a lot of interesting subtlety tends to be in C macros. The most useful thing an automatic parse of C headers could do (in my experience) is make sure that constants are correct, but that tends to risk pulling in arbitrarily-complicated macros.
  3. C APIs tend to lie about const, 'cause they can. A string literal in C has type char *, not const char *, despite it living in read-only memory, because C needs to be backwards-compatible with interfaces that literally predate const. Or they lie about const 'cause they have to, as a result of being on the wrong side of those interfaces.
  4. You have to do this on the target system anyway for correctness; you can't import the bindings once and do a translation step. For some languages (especially dynamic/interpreted ones), you can semi-reasonably expect your end-users to have a C compiler installed. That's less common with compiled binaries.
  5. In some cases, there are multiple choices about how to bind the raw C FFI function. One interesting approach I've seen is nix::sys::uio::IoVec, which augments the raw C struct iovec with a type parameter indicating whether it's an IO vector of read-only (const) or read-write data. The C structure just uses void * and promises not to write into those pointers when you call writev; see above about const.
  6. You need to care about ownership/lifetimes, so if you're exposing a safe interface, you need to write wrappers and pay close attention to pointer types anyway. (You also do in other languages, you can just get away with not paying attention much of the time and crash the rest of the time, while the Rust ethos is to get things right up front.)

In Python, for instance, projects have been moving from the built-in ctypes to cffi, which does let you read in C headers, but works by literally running a compiler when you call the module to figure out exactly what everything's alignments are. There is a mode where you can do that compilation before releasing a binary distribution, but it still involves asking a C compiler to help you build a Python program.

I've found that translating C structures by hand into Rust isn't too bad, and avoids other developers -- let alone end users -- needing to have bindgen (including LLVM development headers) installed and working on their machines. bindgen can handle the straightforward cases, but so can I. That said, I can see bindgen being very useful if you have a lot of API surface that you want bindings to and the bindings are largely mechanical; if you really don't need the human eye on the bindings, might as well make a computer do it.

BTW, if you're working on Windows bindings, you may want to get in touch with u/retep998 aka WindowsBunny, who's been doing a lot of Windows API binding work.

2