require+localizing=frustrating (was Re: Idea for a new kind of require)

[Jay Carlson <nop@...>]

On Aug 25, 2012, at 2:07 PM, Doug Currie wrote:

> On Aug 25, 2012, at 1:21 PM, Rob Hoelz <rob@hoelz.ro> wrote:
>> For interesting parties, I have started development
> 
> An option to force re-load of the module would be nice. During development, I sometimes find it useful to set the package.loaded to false so the module is re-loaded, but I usually have to confirm what exactly has be be cleared to avoid using the already-loaded/cached version(s). 

Although the modern all-local model for imports has saved much frustration, it's kinda nasty about two things:

1) It fails "Don't Repeat Yourself", and fails it in programs two steps beyond "hello, world". local ml,sqlite=require"ml",require"sqlite". This is not a pretty thing to show off to beginners. The alternative is ml=require"ml", and we seem to be trying to discourage that. Or as proposed, go back to a require-replacement which sets _G.ml.

2a) Interactive development of modules. Single-module reload is hopeless. Everybody has a local pointer to your table, and they'll be using the old table. This is fixable by starting all modules with:

  local mymodule = {}
  local modulename = select(1,...)
  if modulename then
      mymodule = packages.loaded[modulename] or {}
      for k in next,mymodule,nil do mymodule[k] = nil end
  end
  function mymodule.write() end

which is kinda like what the deprecated module() did. Then you just need a require() which can be overridden to ignore packages.loaded.

2b) As things moved from _G into modules (like table.*), a lot of code got big lists of abbreviations at the top, in the name of efficiency, conciseness, and immunity to _G poisoning. local tinsert=table.insert. If anybody has stashed a local copy of mymodule.write, it doesn't matter if you updated the mymodule table in-place.

2c) Random closures floating around will have lexically localized old versions of mymodule.write.

Issue #2 seems to be telling me "get a real debugger if you don't want to restart on change". I don't find that very satisfactory, but the alternatives seem worse. Being able to mutate the procedure part of closures is a classic debug-and-go trick, but...yuck.

I think much of both #1 and #2 is a desire to use short names for references--and not take the efficiency hit of traversing _ENV.packages.loaded.mymodule.write in an inner loop. If modules had an aliases block which could be triggered on change-of-environment, that'd cover most of it. I dunno, hoisting inner declarations into that block is the logical next step, maybe.[1] 

alias do
   import mymodule -- not quoted; is a variable declaration
   tinsert=table.insert
   in mymodule alias read, merge end
end
for i =1,1e3 do
  alias write=mymodule.write
  for j=1,1e6 do write() end
end

could be rewritten into 

local mymodule, read, merge, tinsert, gensym$4321
local function __init()
  mymodule=require"mymodule"
  tinsert=table.insert
  read = mymodule.read
  merge = mymodule.merge
  gensym$4321=mymodule.write
end
__init()
for i=1,1e3 do
  for j=1,1e6 do gensym$4321() end
end

A whole bunch of trickery seems like it would benefit from a general hoist-to-top-chunk syntax, allowing relatively constant expressions to be declared in a lexical scope close to use, but not evaluated every time the loop goes 'round. (This is similar to C function-context "static" declarations.) But that's pretty radical, and I think I'd want to try it in metalua or something for a while before endorsing it.

I don't think this is "as simple as possible but no simpler," but probably unlike most people, don't-repeat-yourself has been driving me nutty lately.[2] This is yet another lexical environment problem--if you think it's a problem at all. Nobody is going to chop off your head for using _ENV.

Jay

[1]: The long form would be something like "alias function init()" but that's enough Christmas tree syntax for one day. Oh, and inner aliases could be hoisted to the first surrounding context with an alias block...wait, that means adding an alias block at level 2 can screw an alias four pages later and 9 levels deep. Bleh.

[2]: I went on a Project Lombok rampage last year and deleted a few thousand lines of Java getFoo()/setFoo() accessors. That commit had a very satisfying diffstat.