Project

General

Profile

Task #1031

Autoinstall FFTW

Added by Roland Schulz over 6 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
build system
Target version:
Difficulty:
uncategorized
Close

Description

Given that a lot of installation problems are related to FFTW and FFTW is the only required library (for good performnance) it seems it makes sense to auto-install FFTW if it isn't available.

Associated revisions

Revision 3b21fcb9 (diff)
Added by Christoph Junghans over 6 years ago

Add option to internally build FFTW if selected

Fixes #1031

Change-Id: If9d7cee68f4ced26e931b9a7c5595df47185bb66

History

#1 Updated by Mark Abraham over 6 years ago

Agreed. It's GPL, so we can do it. FFTW 3.2.2 is 3.7MB compressed, which shouldn't break the bank. We will still support linking to an external FFTW, so won't distribute FFTW source in the git repo. A small amount of CMake logic will be required to let a developer know that FFTW isn't found and to either make it available or put the tarball in the dependencies directory.

#2 Updated by Roland Schulz over 6 years ago

I think distributing FFTW with our tarballs is a good idea. It's GPL, so we can do it. FFTW 3.2.2 is 3.7MB compressed, which shouldn't break the bank. We will still support linking to an external FFTW, so won't distribute FFTW source in the git repo. A small amount of CMake logic will be required to let a developer know that FFTW isn't found and to either make it available or put the tarball in the dependencies directory.

I don't think we should distribute FFTW with Gromacs. Erik wants to change the license to LGPL so the GPL is a problem. Also I think it is unnecessary to bundle. cmake has a build in function to auto-download FFTW if required.

#3 Updated by Mark Abraham over 6 years ago

LGPL is only a problem if we want version 3, if I read http://www.gnu.org/licenses/gpl-faq.html#AllCompatibility correctly.

If we don't bundle, we need to make provision for people who build on hosts that can't download during the build process. I also think it is bad policy to assume we can download software from a cmake execution, so there still has to be user participation in the process somewhere. We also have to be careful that we don't download repeatedly, for some reason.

#4 Updated by Roland Schulz over 6 years ago

Mark Abraham wrote:

LGPL is only a problem if we want version 3, if I read http://www.gnu.org/licenses/gpl-faq.html#AllCompatibility correctly.

They are compatible. That's true. But the bundle is under GPL not LGPL. But if we would have a download both with and without FFTW that probably would be OK because someone who wants to use it under LPLG could use the version without FFTW bundled.

If we don't bundle, we need to make provision for people who build on hosts that can't download during the build process. I also think it is bad policy to assume we can download software from a cmake execution, so there still has to be user participation in the process somewhere. We also have to be careful that we don't download repeatedly, for some reason.

If a version is already installed it should always be the default. That also means manually installing will continue to work. So I don't think it is important that the auto-build works for everyone (e.g. hosts without internet). Obviously for required features we should aim for making it work for 100% of people. But for optional or increased usability features like this, I think it makes more sense to only aim for most users. That saves us time and complexity so that we have less followup problems and have more time, we can use to do add more features which help the majority.

#5 Updated by Peter Kasson over 6 years ago

Confirming here that we want to distribute Gromacs under LGPL rather than GPL. Therefore bundling software that places the bundle under GPL is a no-no for official Gromacs distributions. If anyone wants to do third-party bundles, that's fine, but we shouldn't be doing it with the official distro. As I noted on Gerrit, Erik suggests that we request permission directly from the FFTW developers to do an LGPL bundle with Gromacs. That would solve the issue.

Failing that, there's always MKL...

#6 Updated by Mark Abraham over 6 years ago

The authors of a work are entitled to distribute under any licenses they want. AFAIK that means they can distribute under both GPL and LGPL (but those distributions might differ, e.g. by whether they bundle GPL code with them).

So, we could release an LGPL'ed distribution that does not bundle FFTW (which presumably meets some purpose which I'd like to find out!), and release a GPL'ed version that could make our users lives easy by building a bundled FFTW for them that will link cleanly and take advantage of what we've detected about their hardware.

However, if we can get a license for FFTW under LGPL, then we have the option of just releasing under LGPL. I will contact FFTW and ask. That would make our lives simpler than releasing twice, but I'd like to know why LGPL is a good idea, since changing nothing at all is simpler still!

#7 Updated by Roland Schulz over 6 years ago

What exactly do you want to ask the FFTW authors? GPL and LGPL is already compatible (see e.g. http://en.wikipedia.org/wiki/GNU_Lesser_General_Public_License right info box). And I hardly doubt that they'll give it to us under LGPL because that would mean everyone could use it under LGPL.

#8 Updated by Mark Abraham over 6 years ago

Compatible, yes, but http://www.gnu.org/licenses/gpl-faq.html#AllCompatibility shows that if we were to bundle the GPLv2+ FFTW with GROMACS then we have to convey that bundle under GPLv2 or higher. That would mean providing GROMACS under two different licenses, if we want to bundle FFTW, and also want an LGPL version available to suit some external need.

As you say, I don't expect them to agree to LGPL FFTW, since that should allow us to distribute LGPL FFTW. They do permit other licenses for a fee, but presumably we'll not take that option if it's more than a nominal amount.

#9 Updated by Roland Schulz over 6 years ago

I still don't see how even bundling is a problem. Yes the bundle would be under GPLv2. But that would not mean we need to provide Gromacs under two licenses. http://www.gnu.org/licenses/lgpl.html section 2b): " you may convey a copy of the modified version [] under the GNU GPL, with none of the additional permissions of this License applicable to that copy." So after we switch, everyone (including ourselves) can still redistribute under GPL, if they so which. As long as we still provide an option to get the LGPL version we still achieved the goal to have a LGPL version. Thus it is not a problem to have binaries compiled with FFTW on our homepage after the switch.

#10 Updated by Mark Abraham over 6 years ago

Summarizing discussion elsewhere:

1) Official GROMACS distributions will only occur under LGPL to suit the requirements of our funding source

2) If we use a CMake feature that can install FFTW for the user (but is off by default), that does not itself distribute GPL code, or cause a GPL library to be distributed automatically, but rather permits the user to choose to receive and install GPL code. Even if you interpret the function of such code as GROMACS distributing GPL code, we are expressly permitted to do that under clause 1 of the GPLv2 under which FFTW is licensed so long as we satisfy some copyright and warranty conditions (which we do by causing their tarball to be used intact, and can augment this by noting that this FFTW software will be licensed to the user under the GPL, not the LGPL)

3) I have emailed FFTW and enquired about an LGPL license for FFTW, which would give us the option of bundling that version of FFTW with LGPL GROMACS. Whether we actually want to do so remains an open question.

4) Fixing a particular FFTW version is inconvenient, even if we let the user change the version we install for them, but we are choosing to accept some burden to make the software more user-friendly and effective for people who don't know much about installing complex software. A trip to the FFTW webpage every time we release seems not so big a deal to me.

#11 Updated by Mark Abraham over 6 years ago

Roland Schulz wrote:

I still don't see how even bundling is a problem. Yes the bundle would be under GPLv2. But that would not mean we need to provide Gromacs under two licenses. http://www.gnu.org/licenses/lgpl.html section 2b): " you may convey a copy of the modified version [] under the GNU GPL, with none of the additional permissions of this License applicable to that copy." So after we switch, everyone (including ourselves) can still redistribute under GPL, if they so which. As long as we still provide an option to get the LGPL version we still achieved the goal to have a LGPL version. Thus it is not a problem to have binaries compiled with FFTW on our homepage after the switch.

That clause of the LGPL conveys rights to the user of the LGPL software GROMACS, and so is not relevant to whether we can bundle FFTW with GROMACS.

The GPLv2 (or higher) applies to FFTW and works derived from it, and its 2b states "You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License." Here the "Program" is FFTW and that license constrains the distribution of the derived work of "GROMACS bundling FFTW".

Anyone who gets an LGPL copy of GROMACS can still make and distribute such a GPL bundle that if they want to do that, but we are going to be safe and not do so using the time and resources provided on the requirement that we distribute under LGPL.

#12 Updated by Roland Schulz over 6 years ago

Regarding Gromacs binaires:

(Somewhat of topic, but it probably gets confusing if they discussion is in even more places.)

I put quite a bit of effort into making it easily possible to generate binaries of Gromacs so I would hate to see that time wasted. So I'm not happy at all, that it is being decided that we can't distribute binaries with FFTW just to be safe, after the work has been done. (Of course the author have any rights the users have too)

#13 Updated by Mark Abraham over 6 years ago

Matteo Frigo of FFTW got back to me. As expected, they have no plans or intention to release FFTW under LGPL. So we will not bundle FFTW in our source distributions.

#14 Updated by Mark Abraham over 6 years ago

Roland Schulz wrote:

Regarding Gromacs binaires:

(Somewhat of topic, but it probably gets confusing if they discussion is in even more places.)

I put quite a bit of effort into making it easily possible to generate binaries of Gromacs so I would hate to see that time wasted. So I'm not happy at all, that it is being decided that we can't distribute binaries with FFTW just to be safe, after the work has been done. (Of course the author have any rights the users have too)

According to my reading of the GPL (section of part 2 of the GPL below), a GROMACS binary statically linked to FFTW is a derivative work of FFTW (defined in section 0) and thus must be distributed under the GPL.

These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it.

Perhaps one could argue that a GROMACS binary dynamically linked with FFTW is not a derivative work, so long as we use our own header file, and so distribute such a binary under the LGPL. I'm not qualified to give a legal opinion on that, and while I hate to see useful work wasted, think that the GROMACS project as a whole is better served by not running the risk of legal or funding issues. In the absence of compelling evidence that it is legal to do something, I think we should avoid doing it.

In part, the GPL is designed to give free software precisely this advantage - software using it must stay free.

One alternative is to obtain a license to distribute binaries compiled with Intel MKL. They charge for a license "per developer" (which is a bit hard to define for a distributed free software project). I can't see us being keen to pay more than a nominal amount, however. My experience of linking with MKL has not been a happy one, as the linking command changes with the hardware, and I am not sure how reasonable it is for us to distribute binaries in that context. Either we perhaps need to support a lot of linking constructs, or perhaps need to accept that people will get bad performance. I'd rather put the effort into making it easy to build GROMACS to get good performance.

#15 Updated by Roland Schulz over 6 years ago

Mark Abraham wrote:

According to my reading of the GPL (section of part 2 of the GPL below), a GROMACS binary statically linked to FFTW is a derivative work of FFTW (defined in section 0) and thus must be distributed under the GPL.

I agree.

[] while I hate to see useful work wasted, think that the GROMACS project as a whole is better served by not running the risk of legal or funding issues. In the absence of compelling evidence that it is legal to do something, I think we should avoid doing it.

What exactly is the potential problem? The derived work (the binary) is effective under GPL. But this doesn't mean that Gromacs is under GPL it is still under LGPL but the derived work is under GPL. This is allowed because the licenses are compatible. Do you agree so far?
Now to the funding issue: Does it exclude that anything is offered for download ALSO under GPL? Or only that it is exclusively under GPL? Because the binaries wouldn't exclude anyone. They would just add another option of how to download Gromacs. And this other option (the binaries) would be something which is under GPL. But it wouldn't effect at all the source download which would still be under LGPL. In that sense it doesn't restrict anyone. The source download is as before. The binary download is added but only usable by people who don't mind to use it under GPL. For people who want to use it under LGPL nothing would change. They would continue to only have the source download option.

#16 Updated by Mark Abraham over 6 years ago

Roland Schulz wrote:

Mark Abraham wrote:

According to my reading of the GPL (section of part 2 of the GPL below), a GROMACS binary statically linked to FFTW is a derivative work of FFTW (defined in section 0) and thus must be distributed under the GPL.

I agree.

[] while I hate to see useful work wasted, think that the GROMACS project as a whole is better served by not running the risk of legal or funding issues. In the absence of compelling evidence that it is legal to do something, I think we should avoid doing it.

What exactly is the potential problem? The derived work (the binary) is effective under GPL. But this doesn't mean that Gromacs is under GPL it is still under LGPL but the derived work is under GPL. This is allowed because the licenses are compatible. Do you agree so far?

It means we are making GROMACS available under both GPL and LGPL, and if that was clearly permissible then I presume Erik would be having us do that. I have not seen any details in writing, however.

Now to the funding issue: Does it exclude that anything is offered for download ALSO under GPL? Or only that it is exclusively under GPL? Because the binaries wouldn't exclude anyone. They would just add another option of how to download Gromacs. And this other option (the binaries) would be something which is under GPL. But it wouldn't effect at all the source download which would still be under LGPL. In that sense it doesn't restrict anyone. The source download is as before. The binary download is added but only usable by people who don't mind to use it under GPL. For people who want to use it under LGPL nothing would change. They would continue to only have the source download option.

I don't know the wording. I can ask Erik for a copy of the conditions of the funding, but even if I get it, I'm not going to wade through a lot of legalese in the near future :) If it says "software must be released under the LGPL" without clearly embracing the possibility of multiple licensing, then it is not reasonable to jeopardise the future of GROMACS development on the possible future interpretation of some bureaucrat who doesn't care about GROMACS or free software, to suit the convenience of people who find it hard to compile really good software for which they have paid nothing.

#17 Updated by Roland Schulz over 6 years ago

I fully agree that you shouldn't waste a lot of time on it. But if you could very quickly check whether the funding text clause is obvious or write an short email to an appropriate legal department (either of the university or the funding agency), this would be very much appreciated.

It would be a big pain to have to setup the whole Jenkins thing again here, just to be able to do offer this binary download option from an ORNL website (and thus avoid the funding issue).

#18 Updated by Mark Abraham over 6 years ago

  • Category set to build system
  • Status changed from New to Closed
  • Target version set to 4.6

Done.

Also available in: Atom PDF