Potential security problem in compilation of Lua5.0-alpha

[lua+Steven.Murdoch@...]

> > io.tmpfile was added as an alternative solution to this problem.
> 
> io.tmpfile uses tmpfile, which is stated to be also insecure in:
> http://www.suse.com/us/private/support/howto/secprog/secprog3.html#tmpf
> However unlike tmpnam there is nothing in the manpages to indicate this so 
> perhaps it has been fixed. Probably a more definitive answer should be found,

I have tested tmpfile using strace on both Redhat Linux 7.2 and Debian
GNU/Linux Woody (3.0) and it does not suffer from the bug mentioned
above. Unless anyone can test other platforms and would say otherwise
I think than tmpfile and hence io.tmpfile is probably safe (at least
from this bug).

The key feature is that is passes the O_EXCL flag to the open call and
uses mode 600, for example from strace:

stat64("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...}) = 0
gettimeofday({1039218243, 63455}, NULL) = 0 getpid() = 24519
open("/tmp/tmpfsxOIRd", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
rmdir("/tmp/tmpfsxOIRd") = -1 ENOTDIR (Not a directory)
unlink("/tmp/tmpfsxOIRd") = 0

This ensures that the process which creates the temporary file is the
only user of the file (assuming the return value it checked and
properlay handled).

> otherwise mkstemp should be be used where available.

mkstemp is not standardised and behaves is as safe a way as
tmpfile. While previous versions of tmpfile have had vulnerabilities,
so has mkstemp. Debian now recommends using tmpfile and I would agree.
 
However os.tmpname is still insecure and since I can not see any
situations where it would be better using this, I think the safest
thing to do would be to remove it from the API, or at least disable it
by default and require a config file change to enable it (accompanied
by suitable warning messages).

This would have the advantage of removing the only warning message
from the compilation of Lua and more importanly encourage the writing
of secure code (one of the advantages of higher level languages like
Lua).

Steven Murdoch.