Mauro Iazzi

wiki

Ex API

As for the ExtensionProposal, I have some thoughts that came while reimplementing. It is useless to have two separate implementations of the API so I just put the code [here] so you can see it if you want. It is POSIX.1-2001 conforming, but i'm not sure of other standards (surely not previous POSIX).

The proposal is the API, not the implementation. My implementations are intended simply as reference implementations. So, you needn't say /re/implementing. :) I don't believe it is useless to have multiple implementations of the same API. -Mark

Provided you go on mantaining, your code probably is simply better than mine. Anyway the proposed changes are to the API... --mi

I implemented these as simple as possible, probably same way as Mark's:

 * getenv,
 * setenv,
 * environ,
 * chdir,
 * mkdir (setting perms to 0777, plus implied umask),
 * currentdir

Anyway the interface is the same so Mark's code is surely better. Some other functions are not even replicated as I did not need them.

"lock" works as Mark's but it can accept a mode of 'u' to unlock:

function ex.unlock = function (file, offset, length)
  return ex.lock(file, 'u', offset, length)
end

I don't see why ex can't allow a "u" mode string to the lock function. The reference implementations already allow it. In fact, there may as well be only the lock function. -Mark

Didn't know, so it's the same this too. --mi

I added a getid function:

  local uid, euid, gid, egid = ex.getid()

what about this, could be useful --mi

I had some problems with optimization, so I implemented "opendir", "closedir" and "readdir" which are just wrappers around the corresponding POSIX functions.

local d = assert(ex.opendir(name))
  for filename in ex.readdir, d do
    print(filename)
    --- you can even break out of the loop
    if (string.match(filename, pattern)) then break end
  end
ex.closedir(d)

Mark's ex.dir can be emulated by these ones like this:

function ex.dir (name)
  local d = assert(ex.opendir(name))
  return function ()
    local f = ex.readdir(d)
    if (f)==nil then ex.closedir(d) end
    return f
  end
end

for fn in ex.dir(name) do
  print(fn)
  -- you cannot break or dir will remain open until collected
end

I think this is quite more flexible, because you can iterate lots of times over a directory and can break out of the loop and close the dir, so that if you are searching a file in a *HUGE* dir and find it as first you can save much time. You can even reuse the directory using "rewinddir" I wrapped. The implementation is also very simple like that, and uses one C function for one Lua function.

local d = assert(ex.opendir(name))
local name = "first"
while name ~= "last" do
  ex.rewinddir(d)
  for filename in ex.readdir, d do
    print(filename)
    --- you can even break out of the loop
    if (filename == name) then 
      local f = assert(io.open(filename))
      name = f:read('*l')
      f:close()
      break
    end
  end
end
ex.closedir(d)

moreover readdir just returns the filename, but I implemented a fstat (and lstat) which returns some information on the file.

I exported stat and lstat. They too are just wrappers. I think it is simpler to divide functions like that and the names are more intuitive instead of having "dirent".

So, the reason that ex does not simply export an open/read/close style API for directory iteration is that the most common use case by far is iteration in a loop. If one really needs to partially iterate many, many directories (in which case collection may not be happening fast enough), then you can still do this with os.dir(), although it's not quite documented in the API. -Mark

Ok, I didn.t know. So I assume it would be ok to implement some of the API in Lua itself, is it? --mi

d,ds,di = assert(os.dir(name))
for entry in d,ds,di do
  if name==entry.name then
    break
  end
end
d.getmetatable().__gc(d)

I just fear that if this goes in the API requirement it would constrain a lot the implementation... I, for example, stick to the iterator function, opaque data model that is used for normal tables (for k, v in next, t do ... end). My iterator is a simple function and have no individual metatable, so I cannot provide a way to make this code work. On the other hand the API should actually specify a way to explicitly close the directory... I have no clue. --mi

That said, I don't think it's a bad thing to have opendir/readdir/closedir. It's also not bad to have stat and lstat. However, you still need to implement ex.dir and ex.dirent. dirent probably is a poor name however, but while stat may be more intuitive (at least to UNIX programmers), I don't want to confuse ex with POSIX. Do you have a better suggestion for the name of this function? -Mark

I agree, my naming was indeed strictly based on POSIX, as it was for me only. I think a ex.fileinfo would perfectly fit the Lua naming style. As by the fstat - lstat difference, do you have any idea? --mi

Please note that if what you really want is a POSIX API, then you should use lhf's POSIX wrapper http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/#lposix instead of ex. -Mark

''eheh, but it seems that he left the burden on this proposal, as I find following the link to 5.1 implementation: http://lua-users.org/lists/lua-l/2006-10/msg00499.html BTW I completely agree that this should not be just a wrapper over POSIX and endorse the name changing, for one. --mi''

Then "spawn": here's the big change. I defined it to be simple to implement, basically. It uses the arguments like this Lua code:

function lua_spawn(t)
  if type(t)=='string' then t = { t } end
  assert(type(t)=='table') -- "spawn must receive a table or a string"
  local argv = t.argv or t
  local argc = #argv
  local command = t.command or argv[1] or error'command not given'
  if not argv then argv = { command } end
  local env = t.env or ( type(t.env)==nil and ex.environ() or {} )
  if t.stdin then set_redirect_to(redirections, t.stdin) end
  if t.stderr then set_redirect_to(redirections, t.stderr) end
  if t.stdout then set_redirect_to(redirections, t.stdout) end
  return posix_spawn(command, argv, argc, env, redirections)
end

proc = ex.spawn"/bin/echo"
proc = ex.spawn{"/bin/echo", "hello", "world"}
proc = ex.spawn{command="/bin/echo", "echo", "hello", "world"} --big difference
proc = ex.spawn{argv = {"lua", "-e", 'print"Hello world\n"'}} -- equivalent to Mark's example
proc = ex.spawn{"lua", "-e", 'print"Hello world\n"'}
proc = ex.spawn{"lua", "-e", 'print"Hello world\n"', env=ex.environ()}
proc = ex.spawn{"/usr/bin/lua", "-e", 'print"Hello world\n"', env=false} -- this uses a void env
proc = ex.spawn{"/usr/bin/lua", "-e", 'print"Hello world\n"', env="equivalent"} -- this uses a void env
proc = ex.spawn{command='/usr/bin/vim', "ex", "ex.c"} -- starts vim in "ex" mode
proc = ex.spawn{command='/bin/busybox', argv = {"ls", "ex.c" }} -- ls ex.c
proc = ex.spawn{command='/bin/busybox', argv = {"cp", "ex.c", 'ex2.c'}} -- cp ex.c ex2.c, same binary
proc = ex.spawn{"echo", "hello", "world", env={ PATH="/bin" } }
proc = ex.spawn{"echo", "hello", "world", env={ "PATH=/bin" } } -- same as before

Why argv instead of args? -Mark

Because I'm stupid and did not remember your name when I wrote this. And never minded changing. This was meant for me only and I show it just as a test of how the API spec was interpreted by me. It's not a "reference implementation for changing your API" or something as pretending as that... Just plain code, writing which I thought some things. Most of which were simply wrong, you see :) --mi

The PID userdata can then be consumed by waitpid(pid), which is also a method:

ex.waitpid(pid)
pid:waitpid()

Why waitpid instead of wait? It should be fine to have wait() return both the exit code and a second return value with extra information. -Mark

I sticked to POSIX names, and wanted to make clear this does not work as yours. It returns the exit code as second parameter, so it's not compatible. If the other information is returned, it should be the second return value and not the first to adhere to your API --mi

This seems quite easy to use and mainly differs from Mark's in that env can be both the calling environ or voided, and that specifying BOTH command AND argv must be done with an additional argv[1] string. This is acceptable, I think, because if you require so much control over the spawn arguments you can double check that and probably would appreciate having the option to call the program by path with a different 0th argument (vim pays attention to that, for example)

I'm not sure what you mean when you say the environment can be "voided". Do you mean an empty environment, or an inherited one? ex already allows both. Also note that the ex API does in fact support zeroth arguments, with or without 'args':

''True for the env, I misread the proposal, I'm sorry. As for the zeroth argument I find a little counter-intuitive that the presence of a command field in the argument changes how the args list is interpreted. Specifically I compared:

os.spawn{"echo", "hello", "world"}
os.spawn{command="/bin/echo", [0]="echo", "hello", "world"}

os.spawn{command="/bin/echo", [0]="echo", "hello", "world"}
os.spawn{command="/usr/bin/vim", [0]="ex", "ex.c"}
os.spawn{command="/bin/busybox", args={[0]="ls", "ex.c"}}
os.spawn{command="/bin/busybox", args={[0]="cp", "ex.c", "ex2.c"}}

-Mark