Re: modules, require, magic

[Mark Hamburg <mark@...>]

Some clarity on these and why I think some of them are bad.

On Oct 17, 2011, at 11:37 PM, Mark Hamburg wrote:

> Let's run through what module does.
> 
> 1. It handles constructing module tables including the fields _NAME, _M, and _PACKAGE. This is useful if one cares about these fields, but I note that they don't appear in the pre-defined modules like table and string.

Straightforward. Not bad at all. I don't know how useful they are, but they basically just say that all module tables include these three fields. Or they do if they are created by the module function which means that lots of them don't which means that you probably can't count on them being present unless module becomes the only way to define modules include the standard modules.

> 2. It handles constructing submodules. I might question whether it makes sense to have socket.http available when socket is potentially just a stub table, but that's a more complicated matter.

Straightforward again (more or less) but only relevant if we also do #4 (add the modules to the global namespace). require "socket.http" returns a table of functions but this table is also accessible as socket.http via the global namespace because of #4. As another side-effect, if some code later does a require "socket", they can see socket.http as well but only if some earlier code has done a require "socket.http". That leads to the hidden dependencies discussion on #4.

> 3. It provides early registration for the module. This helps with mutually recursive requires, but it also means that require can return an incomplete module.

By sticking the table for the module into the package registry when we create the module, we allow for recursive and cyclic requires. The only downside is that it means that you can't count on the table you just got back from require to be completely filled in. This again could be viewed as a hidden dependency wherein the load sequence for a module works until it happens to end up in a load cycle and call a function that isn't yet defined/exported. On the other hand, without this early definition, the error handling for a cyclic load is also complicated though detectable with an appropriately implemented require function. Personally I tend to look to the Oberon system for inspiration and it got away without cyclic loads, but I know there are plenty of coding cases where code has to be contorted to avoid cyclic load, so having support for it may just be a wise engineering tradeoff.

> 4. It adds the module to the global namespace. This, in my opinion, is a bad thing because it creates hidden dependencies -- i.e., code can use a module that it never required simply because some other code required it earlier and it became available in the global namespace.

The problem here is that if module "foo" does a require "bar" which does a require "baz" and then module "foo" does a require "bleen", then bleen can see bar and baz without ever requiring them:

	foo
		require "bar"
			require "baz"
		require "bleen"
			-- bar and baz are now visible to bleen

But then we happen to rearrange or eliminate requires...

	foo
		require "bleen"

References to bar and baz from within bleen now stop working even though we changed none of bar, baz, or bleen. I view it as a bad thing when code gets broken without immediately obvious changes. Maybe unit tests catch this quickly. Even so, you'll be looking at a new failure in bleen when bleen hasn't been changed.

So, this is my big issue with adding globals and with the recursive submodules system: It creates fragile code because it makes code accessible without the use of require if it happens to have been previously required.

Now, there have been some suggestions that an opposition to this behavior should in its logical conclusion all oppose having any of the standard functions accessible through the globals. As I responded to that message, I can certainly see a case for that. However, the argument here neither supports nor undermines that case because since those functions are always present, there is no hidden dependency problem.

> 5. It mucks with the environment to make it "easy" to export functions. But then to compensate for this, it offers package.seeall which results in a module that reveals a lot of globals in its table which have nothing to do with the module -- i.e., it pollutes the API for the module.

Bascially, the issues here has to do with whether it bothers one that if module foo uses the module function with package.seeall then references such as foo.print and foo.pairs and foo.math.sin are all valid. If this is deemed too ugly, then you don't want to use package.seeall, but in that case the fact that module changes where global lookups get directed becomes a rather significant pain for a lot of code that relies on standard functions. Now, if we said that the standard global environment is empty and we just have a require function, then maybe we just wouldn't need package.seeall. But we haven't gone there.

As noted elsewhere, we could construct a module function that essentially kept the published module table separate from the environment for the module code and resolve this issue that way but doing so would make the global accesses within the module more expensive.

The net for me is that to make module a useful and clean construct, it really needs to operate in a world where we could write something like:

	module "foo"

	import io, table, pairs

	-- code goes here

Then, instead of (or in addition to) _ENV, we have import and module as predefined locals.

Mark

P.S. I don't know how to deal with nested module names on a general basis, but import could also allow

	import msin = math.sin, mcos = math.cos

The question would be what locals would get if one just wrote:

	import math.sin, math.cos