Skip to content

Update site package behavior #232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
casparvl wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Update site package behavior #232

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0fcaacb
Update site package behavior to allow specifying a custom EESSI_SITE_…
May 19, 2026
a3229cf
Merge branch 'main' into update_site_package_behavior
May 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion init/modules/EESSI/2023.06.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,16 @@ local eessi_modules_subdir = pathJoin("modules", "all")
-- eessi_module_path is the location of the _CPU_ module files, e.g.,
-- /cvmfs/software.eessi.io/versions/<EESSI_VERSION>/software/linux/x86_64/amd/zen3/modules/all
local eessi_module_path = pathJoin(eessi_software_path, eessi_modules_subdir)
local eessi_site_software_path = string.gsub(eessi_software_path, "versions", "host_injections")
local eessi_site_software_path
-- If EESSI_SITE_SOFTWARE_PREFIX is defined, replace /cvmfs/software.eessi.io (or more generally EESSI_CVMFS_REPO)
-- by that prefix. This ensures that the directory still contains the os/vendor/arch/micro-arch/accelerator etc
-- If it is not defined, default to a site installation prefix under host_injections
site_prefix = os.getenv("EESSI_SITE_SOFTWARE_PREFIX")
Copy link
Copy Markdown
Member

@ocaisa ocaisa May 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love this approach, there is value in having the path in a fully defined location.

Copy link
Copy Markdown
Member

@ocaisa ocaisa May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about it more, and I am coming around. I was a bit stuck on having the fixed path because we require a fixed path to be able to do MPI injection. However, these are actually separate issues and this change doesn't affect that.

Copy link
Copy Markdown
Member

@ocaisa ocaisa May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will require some documentation updates, and some additional checks in EESSI-extend. We should also update the dev.eessi.io workflow to use this.

Issues for all these are enough for now, but they should be addressed.

Copy link
Copy Markdown
Contributor

@bedroge bedroge May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caspar and I discussed this quite a bit yesterday. In some way I also still prefer to have everything in a single and fixed location, but I also see Caspar's point, and being able to do the software installations elsewhere does make sense to me. I still have some concerns, e.g. about where to put the Lmod site hooks: Caspar wanted to keep them under host_injections, as they (may) also affect EESSI modules, and though that makes sense, it does mean that you may have two locations with this 2023.06/software/x86_64/amd/zen3 trees (one for locally built software, one for Lmod hooks). Same question for the site's Lmod cache: for that one it does definitely make sense to store them near the software, meaning you will end up with two of those .lmod directories in different places. Personally, I still don't really like that, it can easily confuse users/admins.

The current situation, with just a single host_injection directory, also has some drawbacks, e.g. in case you want to do site installations on a local CVMFS repo, while having MPI/GPU libraries on a local disk. You can do it by doing some symlink trickery, but that's not ideal either.

Anyway, since the default behavior won't change, I was okay with adding the possibility for overriding the software installation prefix.

if site_prefix then
eessi_site_software_path = string.gsub(eessi_software_path, os.getenv("EESSI_CVMFS_REPO"), site_prefix)
else
eessi_site_software_path = string.gsub(eessi_software_path, "versions", "host_injections")
end
-- Site module path is the same as the EESSI one, but with `versions` changed to `host_injections`, e.g.,
-- /cvmfs/software.eessi.io/host_injections/<EESSI_VERSION>/software/linux/x86_64/amd/zen3/modules/all
local eessi_site_module_path = pathJoin(eessi_site_software_path, eessi_modules_subdir)
Expand Down Expand Up @@ -169,8 +178,14 @@ if ( mode() ~= "spider" ) then
prepend_path("MODULEPATH", eessi_module_path)
eessiDebug("Adding " .. eessi_module_path .. " to MODULEPATH")
end

-- Make sure the EESSI cache is found, this is specified in the lmodrc.lua in the eessi_software_path
prepend_path("LMOD_RC", pathJoin(eessi_software_path, ".lmod", "lmodrc.lua"))
eessiDebug("Adding " .. pathJoin(eessi_software_path, ".lmod", "lmodrc.lua") .. " to LMOD_RC")
-- Make sure that a cache for site installations can also be found
Copy link
Copy Markdown
Member

@ocaisa ocaisa May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to generate the cache also needs an update, but I need to check that works first. We need to let Lmod know that the module path requires a gateway module, then the hierarchy can be represented in the cache.

It's a separate issue, this just reminds me that I need to look into it.

Copy link
Copy Markdown
Contributor

@bedroge bedroge May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we currently even support caches for local installation? Caspar and I looked into that a bit yesterday, but we couldn't find anything. We plan to cover this in the tutorial, but we should indeed also add documentation for this. With Caspar's proposed change, sites can easily call the create_lmodrc.py from our repo to generate the lmodrc.lua file for their local stack (the EESSI module will then add it to $LMOD_RC), and then it should just be a matter of calling for instance https://github.com/EESSI/filesystem-layer/blob/main/scripts/update_lmod_caches.sh to actually generate the cache files.

prepend_path("LMOD_RC", pathJoin(eessi_site_software_path , ".lmod", "lmodrc.lua"))
eessiDebug("Adding " .. pathJoin(eessi_site_software_path , ".lmod", "lmodrc.lua") .. " to LMOD_RC")

-- Use pushenv for LMOD_PACKAGE_PATH as this may be set locally by the site
pushenv("LMOD_PACKAGE_PATH", pathJoin(eessi_software_path, ".lmod"))
eessiDebug("Setting LMOD_PACKAGE_PATH to " .. pathJoin(eessi_software_path, ".lmod"))
Expand Down
Loading