Source
r/rust•
stebalien

tempfile: temporary file library

A secure cross-platform temporary file library for rust. This library should be race free, reliable, and secure in the presence of pathological temporary file cleaners.

crates.io: https://crates.io/crates/tempfile

Example:

use tempfile::TempFile;
use std::io::{Write, Read, Seek, SeekFrom};

// Write
let mut tmpfile = TempFile::new().unwrap();
write!(tmpfile, "Hello World!").unwrap();

// Seek to start
tmpfile.seek(SeekFrom::Start(0)).unwrap();

// Read
let mut buf = String::new();
tmpfile.read_to_string(&mut buf).unwrap();
assert_eq!("abcde", buf);

Posted Apr 15, 2015, 3:26 AM UTC

2121
deadstone•

Out of curiosity, is there any advantage to using an actual file written to disk that gets deleted instead of some anonymous mmaped area?

6
daborossfern•

I've found it useful for writing tests which test things which depend on the file system.

7
simcop2387•

Sharing it with another process is easier. At least as long as you make sure that sharing happens inside the lifetime of the temp file.

6
[deleted]•

If this is an inter-process communications library, or intended to be, then:

  • how does another process discover the name of the temporary file
  • what locking mechanisms are recommended

Documentation:

  • fn shared(): s/guarentees/guarantees/
  • what happens when the structure goes out of scope, does the file get deleted?
3
simcop2387•

With this library I'm not sure at all. I hadn't looked at it yet but assumed there was a way to get the name it created, in which case you'd pass the name to the other process (either when running it, or via some other means). Usually what I use them for in other languages is a file that needs to be read by another process, e.g. an intermediary step in an image processing chain. It's not always possible or a good idea to keep them in memory if you're going to process more than you can fit (if they can fit, the kernel can keep them in caches more intelligently than you'll usually manage).

Usually with the way I've used them there's not really much need for locking since it's write once read-many, but if locking is needed using the flock() system call is all that's been needed. Usually they get treated as if they are immutable and if you make changes you'll just make a new temp file with a new lifetime/ownership.

I'd assume from the docs it'll get deleted when the structure goes out of scope. It says on exit but I suspect rust will be better about that because it knows the actual lifetime of the fd.

1
stebalienOP•

Paths:

There is no way to get the path. I may add a NamedTempFile struct (mirroring the python version) but one of the goals of this library was to be reliable in the presence of a pathological temporary file cleaner.

Deletion:

The file should be deleted when the structure goes out of scope. However, the is slightly stronger: On windows/linux >= 3.11, the file will be deleted by the time your program exists regardless of how it exits. On linux < 3.11 and other *nix operating systems, this guarantee only holds if the TempFile constructor finishes.

2
protestor•

You can create a temporary file, spawn another process passing the filename, and delete the file after this process exits. That's the usefulness of temporary files in languages like shell script anyway.

On Unix platforms, passing /dev/fd/xx as "filename" and writing to it with a pipe is usually better, unless the other process need to seek while reading. On Windows I'm not sure what's the alternative.

1
[deleted]•

spawn another process passing the filename

Specifically I'm asking of this crate how one obtains the filename of the temporary file?

1
stebalienOP•

I leave the locking mechanism up to the user. The only safe way to share the file with another process is to share by file handle/descriptor. This is why the TempFile object implements AsRawFd/AsRawHandle. Unfortunately, sharing the path itself is generally insecure/unsafe.

Thanks for the comments. I've updated the documentation.

1
stebalienOP•

I've added a NamedTempFile variant for when you absolutely need a path and are sure that a temporary file cleaner won't interfere.

1
Hauleth•

Some data is too big for RAM so it wouldn't work in that cases (i.e. you are fetching your program installer from net and it weights 10 GB). So there are cases for that.

3
[deleted]•

[deleted]

1
protestor•

But anonymous mmap in this case is just memory allocation, right? I suppose that jemalloc will call mmap when allocating large blocks.

1
RetraRoyale•

You can make things like pipes as temp files and have a bunch of unknown, separate processes write to them.

1
ssokolow•

How broad a scope are you going for with this crate?

Would you consider writing something along the lines of tempfile.mkdtemp from the Python standard library?

(To quote the start of the docstring, "Creates a temporary directory in the most secure manner possible.")

4
stebalienOP•

The tempdir crate already does this. However, one of the goals of this library is to be secure against pathological temporary file cleaners and that's difficult to impossible to do with temporary directories.

1
Manishearthrust · servo•

Is there any way to make a temp dir here?

3
stebalienOP•

See the tempdir crate.

2
Gubrial•

Is this a typo in v0.5.0?

#[inline(always)]
#[cfg(not(target_os = "linux"))]
pub fn create(dir: &Path) -> io::Result<File> {
    create_unix(path)
}

It's causing this compile error:

/Users/gabriel/.cargo/registry/src/github.com-1ecc6299db9ec823/tempfile-0.5.0/src/imp/unix.rs:45:17: 45:21 error: unresolved name `path`. Did you mean `dir`?
/Users/gabriel/.cargo/registry/src/github.com-1ecc6299db9ec823/tempfile-0.5.0/src/imp/unix.rs:45     create_unix(path)
                                                                                                                 ^~~~
1
stebalienOP•

Yes. Sorry, I don't have any non-linux unix machines to test on. It should be fixed in 0.5.1.

1
Gubrial•

Yeah, I know how it is. I don't mind testing for mac OSX if you like. For example I've submitted another compile issue on the github repo.

1
stebalienOP•

Thanks!

1