[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: To all rock maintainers, take 3 (now with x64 supports)
- From: Sean Conner <sean@...>
- Date: Wed, 4 Nov 2015 20:56:35 -0500
It was thus said that the Great Stefano once stated:
> On 4 November 2015 at 20:00, Sean Conner <sean@conman.org> wrote:
> >
> > While I could store both luuid's "uuid" and uuid's "uuid" modules in the
> > same LEM, I don't have a way to specify which one to use. That *is*
> > something I would need to work on. But as it stands, the LEM format can
> > easily handle different operating systems, different architectures and
> > different Lua versions (look closely---you'll see I have an LPeg 0.12 module
> > for Lua 5.1 and for Lua 5.2).
>
> That does not solve the issues I raised:
> What about file conflicts across different packages (LEMs)?
The original intent of my project was to distribute and run a Lua
application directly from a LEM file. That proved to be ... optimistic at
best [1] (I was able to run LuaRocks directly from a LEM file, but LuaRocks
is pure Lua, and I had to hack into io.open(), dofile() and loadfile()).
But assuming I drop that requirement and instead just unpack everything onto
disk.
> How to perform updates when one package nest another one?
> How to keep the complexity manageable while still achieving
> rollback-on-failure safety?
> Ecc ecc
You face those same problems. Lua doesn't handle module versions at all
(I don't mean between Lua versions, I mean version 1.3 and 1.4 of "foobar")
and if there's a fundamental incompatability, the best you can do is "I'm
sorry, Dave. I can't let you do that."
But aside from that---my system had operating system, architecture, Lua
version (minimum and maximum the module could run under) and module version.
So, running through ULua (I downloaded it from
http://www.scilua.org/ulua.html) as a LEM module (hypothetically speaking
here), here's how I would handle things. First, a listing of the LEM (not
everything is included as I didn't want to spend too much time trying to
track down what everything does, but this should get the point across):
Linux x86 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
Linux x86 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
Linux x86-64 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
Linux x86-64 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
Darwin x86 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
Darwin x86 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
Darwin x86-64 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
Darwin x64-64 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
Windows x86 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
Windows x86 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
Windows x86-64 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
Windows x86-64 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/cURL
MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/cURL.safe
MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/cURL.utils
MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/cURL.impl.cURL
MIT/X11 Lua 5.1 5.1 1.0-beta6 MODULES/pkg.init
MIT/X11 1.6.3-103 DOCS/lfs/doc.css
MIT/X11 1.6.3-103 DOCS/lfs/license.html
MIT/X11 1.6.3-103 DOCS/lfs/index.html
MIT/X11 1.6.3-103 DOCS/lfs/examples.html
MIT/X11 1.6.3-103 DOCS/lfs/luafilesystem.png
MIT/X11 1.6.3-103 DOCS/lfs/manual.html
MIT/X11 1.0-beta6 DOCS/pkg.init/README.md
MIT/X11 1.0-beta6 DOCS/pkg.init/LICENSE
MIT/X11 1.6.3-103 TESTS/lfs/test.lua
MIT/X11 BIN/upkg
MIT/X11 BIN/upkg.cmd
Linux x86 BIN/zip
Linux x86 BIN/unzip
Windows x86 BIN/zip
Windows x86 BIN/unzip
Windows x86 BIN/chmod
Linux x86 MIT/X11 7.42.1 LIB/libcurl
Linux x86-64 MIT/X11 7.42.1 LIB/libcurl
Windows x86 LIB/libintl3
Windows x86 LIB/libiconv2
Windows x86 MIT/X11 7.42.1 LIB/libcurl
Windows x86-64 MIT/X11 7.42.1 LIB/libcurl
FILE/upkg
As I'm running Linux x86, I can ignore anything for Linux x86-64, Darwin
(how OS-X identifies itself) and Windows. Files without an operating
system/architecture designation can be used on anything.
First up, lfs. I already have lfs version 1.6.3 installed on my system,
so I can skip that module. I do *not* have curl currently installed, so I
can copy that module out (keeping track of versions like LuaRocks does)
somewhere into my Lua CPATH (let's make that a configuration option of LEM).
I don't currently include dependency information (but I'll skip that for
now). I can extract the module and as a safety check, I can run 'ldd' over
it to make sure I can run it:
[spc]lucy:/tmp/foo/ulua>ldd ./lcurl/0_3_1+103/Linux/x86/-lcurl.so
./lcurl/0_3_1+103/Linux/x86/-lcurl.so: /lib/tls/libc.so.6: version LIBC_2.4' not found (required by ./lcurl/0_3_1+103/Linux/x86/-lcurl.so)
libcurl.so.4 => not found
libc.so.6 => /lib/tls/libc.so.6 (0x00111000)
/lib/ld-linux.so.2 (0x00b76000)
Failure, but I haven't gotten to installing it yet, so nothing damaged
(yes, I am running an older Linux distribution, no I have no plans of
upgrading any time soon as it works---the last time I upgraded a system
around here it was ... painful [2]). And even if that succeeded, I don't
have libcurl4. But I think that's why you include the library, so let's
carry on as if nothing happened. The Lua based cURL modules can be copied
appropriately, as well as pkg.init. I'll skip over the DOCS and TESTS
"directories" for now.
"upkg" and "upkg.cmd" don't exist in my path, so we can copy those to a
directory in my path that I control (most likely $HOME/bin) [3]. Having the
upkg.cmd won't hurt as I won't be using it.
zip and unzip exist in my path, so we can skip those. Ah, there's
libcurl we could install. And like the curl module, we can pretest this
thing:
[spc]lucy:/tmp/foo/ulua>ldd ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so
./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so: /lib/tls/libc.so.6: version LIBC_2.11' not found (required by ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so)
./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so: /lib/tls/libc.so.6: version LIBC_2.7' not found (required by ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so)
./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so: /lib/tls/libc.so.6: version LIBC_2.4' not found (required by ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so)
librt.so.1 => /lib/tls/librt.so.1 (0x002c8000)
libc.so.6 => /lib/tls/libc.so.6 (0x006f9000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0x008fa000)
/lib/ld-linux.so.2 (0x00b76000)
Hmm ... failure (I did mention I was running an older system here, and I'm
probably not your target audience for ULua). That's the only dependency we
can't fulfill (at least, given that I don't have curl 0.3.1 installed).
Backing out is fine---we can remove curl 0.3.1 as I never had it installed
to begin with. Same with pkg.init. I can delete ukpg and ukpg.cmd since
they were just installed.
But assuming I already had cURL installed (current version is 0.3.2 by the
way). That should be okay, according to semversion, so I could have skipped
MODULES/curl and (had I the dependency information) skip libcurl as well.
Then pkg.init and upkg would be installed and everything would be fine.
Other things to keep in mind---what if I installed a module prior to using
ULua? I have modules installed from source, as well as several installed
via LuaRocks. That's something to keep in mind. A minimum---try including
a module prior to installing it to see if I might already have one
installed. If so, and I can't determine the version (because it lacks a
_VERSION or VERSION or _version or version field) then I can either error
out or ask the user---something along those lines.
How to handle the case where I might be installing a module that breaks a
previously installed module? Error out and inform the user. Maybe offer to
install everything in a sandboxed location. If everything has always been
installed via LEMs (hypothetically speaking) perhaps attempt to do a "fake"
upgrade first to see if it's possible not to break anything and either say
"yes, you can upgrade and not break anything" or "sorry, we can't upgrade
because foo will break."
Keeping this all in mind, I don't see where the issue of nested packages.
> > I propose to you---what would you do if someone would like to use package
> > manager to install json4lua? Or a different example---someone needs
> > numerical integration and thus, would like to use math-rungekutta but can't,
> > because it "conflicts" with math-evol.
>
> The thing is, right now math-evol is taking over math, which is
> clearly bad (we already discussed about that).
> Again, a problem that would be solved by having project name = module name.
> Then, hopefully, something different from math would have been used.
Technically, it's taking over 'Math', and both modules are written by the
same person and are released under a common namespace module (hmmm ...
sounds familiar), named because it's a port of similarly named Perl modules
(Math::RangeKutta, Math::Evol, etc.). The author isn't doing anything wrong
here.
> > Final parting words---if "conflicting modules" is too complex to handle,
> > then please, remove "org" from your project.
>
> I welcome your ides on how to handle it.
> Not in the sense of storing them in a package. I am talking about the
> package management logic here.
Treat the module names as an opaque string and stop trying to shoehorn
everything into what appears to be a top-level module. Lua does this
internally (after require "org.conman.table", require "pl", require
"socket"):
> show(package.loaded)
string table: 0x9acb618
org.conman.table table: 0x9add188
package table: 0x9acba50
_G table: 0x9aca450
os table: 0x9acd3c8
table table: 0x9acb100
math table: 0x9acdf90
coroutine table: 0x9acb718
pl.utils table: 0x9ae3328
debug table: 0x9ace990
pl true
io table: 0x9accbc0
socket.core table: 0x9aeea48
pl.import_into function: 0x9ae29b0
socket table: 0x9aeea48
org.conman.net table: 0x9ae3a28
pl.compat table: 0x9ae9718
Just because "org.conman.table" is stored on disk as
"org/conman/table.lua" doesn't necessarily mean "org" is the top level
module here. It's "org.conman.table". Yes, this might complicate your
package manager some, but I was able to deal with a ZIP file with duplicate
filenames!
> When I get around to implement rocks aggregation (meaning all your
> rocks will end up under a single org package), you'll be free to opt
> out if you want to.
ORG IS NOT THE NAME OF MY MODULES! HOW MANY TIMES DO I HAVE TO SHOUT THAT
BEFORE YOU UNDERSTAND?!
-spc (Wait a second! Can't LuaRocks handle binary distributions as well?)
[1] While I could load Lua modules written in Lua directly from the ZIP
file, those written in C could not. It was a limitation of dlopen()
(the Unix API to load a shared object) and it wouldn't surprise me
if there wasn't a similar situation under Windows. And while I
could hook onto the Lua functions that open a file for reading, I
never did get a clear idea of how to handle opening a file for
writing.
[2] http://boston.conman.org/2015/10/01-18
[3] upkg.cmd *could* probably have a Windows operating system associated
with it. "upkg" itself is for Unix. I suppose I could include a
"unix" operating system label for such a condition.
- References:
- To all rock maintainers, take 3 (now with x64 supports), Stefano
- Re: To all rock maintainers, take 3 (now with x64 supports), Sean Conner
- Re: To all rock maintainers, take 3 (now with x64 supports), Stefano
- Re: To all rock maintainers, take 3 (now with x64 supports), Sean Conner
- Re: To all rock maintainers, take 3 (now with x64 supports), Stefano
