Re: Premake vs. CMake

["Alexander Gladysh" <agladysh@...>]

> > > How large is your codebase?  What does your project do?

> > We're using premake to build our crossplatform (Linux x86 / Linux x64
> > / Windows / OS X) set of... well, complex TCP-services.

> > Our project is under heavy development, and right now consist of
> > approximately 40K lines of C++, 40K lines of Lua and a few 3rd party
> > Lua modules which are built by custom-written premake scripts as well.
> > There are approximately 15 .so and executable files in total for each
> > of two build targets (so far, release and debug).

> > Our premake scripts are approximately 500 lines in total.

I should note also that we're able control (more or less) the
environment there the project would be built from source (this is
relevant for Linux target of course). So no hard requirements for
being able to be built out-of-the-box on every randomly setup crazy
Linux distro machine.

> Ok, so in the scheme of things, a small-to-medium sized project.

Definitely so (while we expect it to grow to full medium size at least :-) ).

> I'm
> currently translating a 400MB codebase from GNU Autoconf + GMake -->
> CMake.  Translating the configure.in alone results in a 10K LOC
> CMakeLists.txt, and the source code tree has thousands of Makefile.in
> files to translate as well.  Recently I think CMake has been proving
> that it can handle large scale build projects, for instance KDE uses
> it.

That is good. But I think such huge projects enforce quite different
requirements to the build tool. In my opinion, it is not necessary for
the same build tool to adequately well meet requirements for both huge
and small projects. More on that below.

> Usually when I see the "we prefer premake" sentiment, or some
> such for another build tool, it's a small project.  I've been
> interested in what advantages these tools offer, whether for small
> projects, or otherwise.

The main advantage for us is that premake uses full-blown mainstream
language for its scripts. Python-based SCons is equally good here --
its just that Lua is closer to our hearts.

> I'm also interested in people's perceptions
> about the advantages, and how that squares with design, maintenance,
> and scale-up over the long haul.

Being able to scale up is a very important feature. But it is not
necessary to be able to scale up for the whole range of project sizes,
say, from 1KLOC to 10 MLOC+.  As your project grows in size, you would
most likely hit some hidden limits in the logic of your build process
organization. Not the natural limits of the build tool, but in how you
organized whole process. This is especially true when you suddenly
growing out the project size limit you ever experienced before. When
you hit such barrier, you have to reorganize your build. Sometimes you
have to rewrite your build scripts from scratch even if you continue
to use the same build tool. It is natural to change build tool during
one of such rewrites.

Each such redesign point could be very painful, I know, so one must
avoid it at all costs. But it is too hard anyway to organize build
process (not tool!) capable for 1MLOC for users to work with 10KLOC
projects comfortably. And vice versa, then you design build process
for estimated 50 KLOC, and suddenly grow ten times heavier, you would
definitely hit something considerable you have not considered at the
start.

For huge projects, the main requirement for the build system is, as I
see it, well, being able to build the project efficiently, while being
able to maintain the build with as little pain as possible. If one is
able to do that -- he is happy. If there is a single way in the world
to do it -- OK, no problem.

But for smaller projects -- where efficiency is not so crucial -- it
is more important to have good maintainability, to be as flexible as
needed to be able to express things in the way that is comfortable for
*our* specific task and to accommodate to *our* favorite habits. The
smaller need to efficiency -- the larger desire to have such features.

(OK, I admit, I do not know for sure what happens after 500K+
boundary; also I might sound naïve... but this is what my previous
experience tells me.)

The point is -- may be it is better to have two excellent tools for
each side of project size scale than one single average tool 'that
fits them all'?

As a side note: I do not want to say that, for example, premake is
excellent for small projects while CMake is perfect for huge ones or
vice-versa. Also I strongly believe that Lua is quite capable to serve
as an efficient solution for a build tool (not necessary premake),
targeted at huge projects. Effective in terms of speed, reliability
and usability and so on.

> For instance, many people are of the
> opinion that a mainstream programming language is beneficial to a
> build system.  Have you seen that actually pay off for you in some
> tangible way?  Or was it about comfort level, feeling that you didn't
> have to learn something new, and it didn't turn out to matter in
> practice?

It is not that we're afraid to invest time into learning something new
-- if it is worth it. But general (not related to CMake in particular)
experience with custom programming languages suggests that they tend
to be... well, of lower quality (either underdeveloped, or buggy, or
too singleminded about ways to solve problems etc.). Aside from
complexity of the language design problem, authors rarely have enough
time to develop well both the language and the logic of the
application itself -- so languages tend to suffer. Also, user base for
such languages are usually smaller, thus reducing amount of the
possible solutions for the problem -- and lowering chances that we
would be able to do things in the ways comfortable for our specific
case.

On the other hand, Lua -- the language we passionately love -- highly
increases adoption chances of any build system capable of production
grade performance.

As for the features... All three seriously considered build systems
(CMake, SCons and premake) can do roughly the same amount of
*absolutely necessary* tricks we require to build our projects.

By the way, there is one more thing that affected our decision --
extensibility / maintainability of the build tool itself. If we
stumbled upon a bug or in sudden need of some mandatory  trick build
system does not have -- we must be able to solve the problem quickly.
The fact that premake uses familiar technology is a great plus here.

Alexander.