Re: Minimum size Lua without floats (e. g. for CORTEX-M0 cores)

On Sun, May 16, 2021 at 4:16 PM Philippe Verdy <verdyp@gmail.com> wrote:

Remember that old home computers had their BIOS/device handlers, basic filesystem and BASIC implemented entirely in a 32KB ROM and were still working within 16KB of RAM, leaving 16KB for the devices and about 4KB for display. I don't think that core Lua is more complex than BASIC. and with more decent home computer, you even had a graphical interface (e.g. Atari ST, using Motorola 32-bit instructions and which also did not have any FPU in its 512KB RAM model; I don't remember the size of the ROM, but it was roughtly the same as your controller; it had a separate controller only for the display and a few I/O ports: parallel, serial/MIDI; it implemented a basic GEM desktop and an graphical API similar to GDI in Windows; and its OS was also single-threaded in one process just like MSDOS or the first 16-bit versions of Windows, using only collaborative fibers/coroutines and no preemption at all; there was preemption only in the kernel for device handler interrupts: timers and I/O port queues; there was also a simple DMA controler for floppy/harddisk, and for the graphic accelerator, an early GPU working within a single frame buffer for some simple graphic primitives). The FPU emulation was quite decent and supported double precision entirely in software. such emulations are still in use today in many low-power controlers, or in some network interfaces and various audio devices.

The ARM libraries contain lot of existing implementations of a basic <math.h> API for C.

The only real limitation is that Lua uses dynamic memory (not just for. tables/objects but also for strings). For strings you may want to change the representation using compression as well. You can look at existing implementations for example made for Postscript, Forth (stack based interpreters). And you can save lot of code in the ROM if you don't implement any form of JIT compilation and code expansion meant only for performance.

But given that your controller will work at much higher frequencies today, you can use data compression and also compact the code using a basic loop, avoiding unrolling existing loops in the ARM code (i.e. compile not for performance but for code size). given the limits of your ROM, I think you still have ample space to embed floatting point support (at least 32-bit float).

And your controller does not need a full features filesystem: there are many simple filesystems that can work in ROM (compressed) and with RAM cache (decompressed): FAT16 requires very small amount of code, and does not necessarily require managing an additional bus for its storage interface which can work directly in ROM and RAM.

As well there are tons of existing command-line tools in Linux that are single-threaded and never fork/yield, and can compute things without using any FPU instruction (e.g. for working with BCD or higher precision than those few IEEE formats built in FPUs, or implemented in audio/video codecs, or for signal processing).

If you restrict too much the usability of Lua, it's very likely that your desire to use Lua on your device will not make it very useful if it cannot reuse most code written in Lua, whbich highly depend on tables, metatables, and coroutines (and very often as well at least 32-bit floats and 32-bit integers).

If you really want to be very compact, consider using Forth or Postscript (but note that both also use associative tables: dictionnaries, most of them keyed by strings)... If you drop essential things from Lua core, don't call this "Lua" even if it uses a similar syntax, because you won't use the same kind of Lua software.

Le dim. 16 mai 2021 à 15:15, Flyer31 Test <flyer31@googlemail.com> a écrit :
Hi Phillippe,
thank you for your remarks about PC history.

But my "desired limit" would be the STM32G071, this has BELOW 256kB ROM, it only has 128kB. And I cannot use all for Lua ... I also need ROM space for my own software (min. about 20kB) and also for the Lua script to run (also min. 10-20kB)... so the "optimum max space request" for Lua would then reduce to 128kB - 40kB= 88kB, at least a "lucky Chinese number" :).

RAM limit of 36kB also is another story... .

(please understand that these ROM/RAM values in such Microcontroller systems are NOT extendable... you have no way to add any further ROM or RAM there (unless you want to loose main controller system advantages... - a main advantage is that it should be VERY small... )).

On Sun, May 16, 2021 at 9:40 AM Philippe Verdy <verdyp@gmail.com> wrote:
Stripping the support for tables would not work. This would no longer be Lua at all. Once you have tables, assing metatables is trivial (and they are important for many Lua applications and its core semantics).

Coroutines may be removed however, but implementing them is really simple (notably if your C library supports setjmp/longjmp, however even if it does not, it just requires very few assembler instructions: remember that coroutines are NOT threads, there's NO preemption at all, they can be just "fibers", all running in a single thread, in a single process, and not requiring any integration of interrupt handlers, this can even be supported in the Lua interpreter in its simple processing loop; you will not need to save many information and manage critical sections for context swtiches, you don't need to preserve complex register states including flags, just allow saving/reloading the stack pointer register).

And there are lot of math libraries in C available that can perform floating point calculation using standard function calls (not requiring any binary instruction parsing and emulation) based only on support for integers (in those libraries you may want to trade about the precision to support; fixed-point is probably not a good option (it is just an adding in some C libraries for generally used for supporting currency amounts, or in some signal processing where the precision is predetermined, such as in audio/video codecs or for some graphics rendering, but you can do all that using only integers, and most C/C++ implementations do not provide any native fixed-point numeric type).

The basic minimum support for C is that every platform supports at least the "float" type ("double" may be just an alias, not increasing the precision or range), and generally this is the 32-bit IEEE (may be it is not supporting all IEEE rounidng modes, and some trigonometric or exponential/logarithms allow for approximations with more than a few twips: some "fast" libraries could compute for example only 16-bit of precision for a sinus). Your math library would typically support only very few functions: sin/cos/tan and their inverses, log and exp (plus probably a faster log2 function which is very simple to implement).

A Cortex-M0 processor is still much more modern than old 8-bit processors that had full support of C basic types and math libraries, and they run at faster frequencies and much more memory. Can you remember the old personal computers of the 80's: just a few kilobytes of RAM, a single core, and capping at a few Megahertz (e.g. the original IBM PC, where it was still really simple to implement a preempting multithreading environment using just plain Pascal and a few inline assembler instructions to setup an interrupt handler): your Cortex-M0 system is much more capable.

And the Lua core source is very simple and porting the basic required set should not require more than a few days of development (you already can find good C compilers for it, even if you need to use cross-compilation on an even more modern platform to create the Lua engine that you'll install on you M0 system, even if the M0 does not compile it itself).

So the good question is not about stripping down Lua itself, but selecting its support libraries that you'll link to your Lua engine: it is in those libraries that you may want to strip down some code to just link what is required and matches your goal. Lua itself is very small (and already largely below your 256KB ROM limit). Do you remember that the original IBM PC could implement the BIOS and BASIC using a 16KB-ROM only, and support for various IO devices, a basic filesystem, and some basic 2D graphics?

Le dim. 16 mai 2021 à 06:36, Flyer31 Test <flyer31@googlemail.com> a écrit :
Hi,
some assumptions really have short life... Last week in by "lua-resume/lua-yieldk post" I proclaimed clearly, that I would not think of using Lua in integer only CortexM0 devices as STM32G0 series... but meanwhile I "licked Lua blood". And the recent STM32G0B controllers really at least for QFN32/5x5mm housing size allow > 100kB RAM (STM32G0B has 256 or 512k ROM/ 144kB - a nice possible way to go for small Lua things would then possibly also support already 128kB ROM / 36kB RAM of STM32G07 in the cute WCLSP28 2.5x2.5mm housing).

The main restriction for the core Cortex M0+ is:
- no floating point
- no Divide command
But they are all M0 PLUS(the + mainly means that they have 32bit multiply command :) ).

The missing Divide command can somehow be "accepted/ignored", as divide for integers can be done also very efficiently by compiler in typically 30-100 core cycles (and even core-managed device will in most times, like in Cortex-M4, be NO 1-cycle command, but need 10-30 cycles, at least for 3 or /64bit integer divide...).

But the missing floating point means, that if I want to run Lua on such a controller, it should be "free of floating point" (if you allow floating point, this will typically "eat controller resources excessively", as floating point math will be is about 1000 times slower than fix point math, and floating point support also will add about 30kB ROM need, and I assume that for a "small Lua application" floating point can be somehow "mainly avoided").

But somehow for the "lazy Lua user" it would important to be able to use some "sort of floating point numbers" anyway... .

I am quite sure that my "standard Lua users" for such a "stripped system" would be happy, if e. g. the float numbers have fix resolution of 1 permille, so 3 digits after point... and then I would propose to use in Lua Fix-Point math with 20 bit "before digit" and 10 bit "after digit" (so then max. number 1E6, and fix resolution of 0.001), and then 1 bit for sign and 1 bit for NAN/INF signaling.

Optimally nice would be, if the "pre-digit"/"after-digit" counts could be somehow configured in Lua conf file, as 14 digits will be required for resolution of 1E4 and 16 digits for "nearly 1E5", very well 14 pre/16 after, or 16 pre/14 after digits would make sense (or 10 pre/ 20 after for ppm resolution fix point math numbers).

So my dream for a "stripped Lua" for such Cortex-M0+ systems would be the following:
- float calculation replaced by fix point math with fix pre/after digits
- ministring lib (no pattern search ... mainly only support of printf and basic number/integer/string conversions)
- minimath lib (no cos/sin... possibly also no exp/log)
- no table support (at least no meta support)
- no coroutines (except please C support for yieldk and resume to allow cooperative "Lua-multithreading" by C support)
- only basic libraries (no file-io, no os,...).

To all who already have used "stripped down Lua systems" already 2 questions (I assume restricting to "basic libs" is very straight forwrad, so no question about this):
- Is it quite easy to strip support of tables and coroutines? (or is e. g. Lua code itself using "table handling" so dominantly, that this would not make sense).
- Is it quite easy/feasable to replace float math by such a "fix point" integer math?

... if somebody has done something similar like this already, I would be curious about the minimum ROM/RAM sizes which would be required by Lua after such "basic stripping down"