The ndexr platform, open source
An open data-science platform: interactive R sessions on shared compute, governed packages, and published apps - built from Gentoo Prefix, EasyBuild, CVMFS, and Open OnDemand , run in production, and handed over here as source. This page is the way in: the address to keep, the archive that contains every layer, and the staged path for standing it up on your own infrastructure - independently of us.
~/ndexr
, and carries its own README, install
guide, runbooks, and license - nothing after the download requires this site, or us.
~/ndexr
: the portal and its apps, the package pipeline, the build recipes and both
RStudio fixes, the CVMFS and S3 scripts, the edge tier, the schema, the deploy
CLI, and the runbooks. Apache-2.0; the LICENSE is at the archive root.
The path in §5 is nine stages, but almost nobody needs all nine, and the first useful one costs nothing. Pick where you want to stop and the pathway below marks the stages that are yours and dims the rest. You can change it at any point, and the page remembers which checkpoints you have passed.
1 What ndexr is
ndexr is a data-science platform organized as a multi-domain model : one Shiny application serving many subdomains, where each capability - the session portal, the package repository, the build recipes, the edge tier, this page - is a directory that is at once its own git repository, its own live site, and the code that stands its subject up. Underneath the sites sits a software distribution stack: a Gentoo Prefix userland that borrows only the kernel from the host, EasyBuild recipes that make every build re-runnable, and a signed CVMFS tree with its origin in S3 that delivers the result read-only to any machine with a client.
The combination does the three jobs teams usually buy the commercial Posit suite for - launch IDE sessions on shared compute, serve curated packages, host what gets built - plus one layer nobody sells: an agent side panel that sits beside whichever site you are changing. Every layer is running in production, and every layer is in the archive above. Sections 2 and 3 give the short version of the two arguments; each ends with a pointer to the long-form dossier it condenses.
2 The model: how software reaches every machine
Installing scientific software is not hard; installing it once per machine, forever is. Enterprise Linux is old on purpose, so the software researchers need will not build against the host - and containers do not settle the question either, because a container answers “how do I ship one application”, not “how does a whole evolving stack reach every machine I will ever touch”.
The architecture that does settle it has two halves.
Delivery
:
CernVM-FS (CVMFS), built at CERN for the LHC computing grid, presents a read-only tree
under
/cvmfs
that is streamed on demand, cached locally, and covered
by one cryptographic signature - a machine mounts it and has the entire stack without
installing any of it.
Content
: a compatibility layer - a complete
userland, own glibc and loader, that is deliberately not the host's - with the
scientific stack built on top from EasyBuild recipes and made choosable by Lmod.
None of this is ours, and that is the point. Two national programmes run exactly this four-layer architecture in production: the Compute Canada / Digital Research Alliance stack, in service the longest, and EESSI, the same design rebuilt for the machines that did not exist in 2017. ndexr is a third, small, fully-published implementation of the same model - which means adopting it does not tie you to us any more than it ties you to CERN.
3 The platform: three jobs, no license required
Strip the branding and a commercial Posit deployment does three jobs: Workbench brokers IDE sessions on shared compute, Package Manager serves curated prebuilt packages, Connect hosts what gets built. Each job is real and each deserves a serious answer - and each is an infrastructure pattern, not a product , with an open, battle-tested implementation.
| The job | The commercial answer | The open answer, in production here |
|---|---|---|
| Launch IDE sessions on shared compute | Posit Workbench | Open OnDemand - the portal pattern academic supercomputing runs at scale |
| Serve curated, prebuilt packages | Posit Package Manager | repo.ndexr.io - validated binaries, served per architecture, baked into R as the default |
| The IDE itself | RStudio Pro | open-source RStudio Server - fixed by us to run on current R, ahead of the vendor's own release |
| Host published apps | Posit Connect | the multi-domain site you are reading right now - one app, many domains |
| Get all four running | a procurement cycle and a license | the archive on this page - unzip, follow INSTALL.md, deploy |
The open foundation is the stricter one, not the cheaper one: own userland instead of the vendor's binary, every build a reviewable recipe instead of an appliance, a signed and versioned software tree instead of a mutable install. The RStudio in the middle of the map is the piece that makes the argument honest - the vendor's build did not run on current R, and two published fixes (not patches: the software did not work) put it ahead of the vendor's own release. Those fixes are in the archive as EasyBuild recipes anyone can re-run.
4 What is in the archive
One archive, staged live from the running repository at download time: roughly 1,900
files across the platform's infrastructure domains, in the real
~/ndexr
hierarchy. It contains the portal and its apps, the package pipeline, the build
recipes and both RStudio fixes, the CVMFS and S3 scripts, the edge tier, the database
schema, the deploy CLI, and the runbooks - plus a generated
README.md
,
INSTALL.md
,
CLAUDE-DEPLOY.md
(addressed to an
agent rather than a person), and a
MANIFEST.txt
. Secrets are
excluded by path rules and a per-file content scan; anything filtered is listed in
EXCLUDED.txt
so the omissions are visible too.
LICENSE
is at the archive root, and the
build refuses to run without it.
5 The pathway: standing it up independently
This is the most open route we can offer into running ndexr on your own platform: everything below works from the archive alone, on infrastructure you control, with nothing gated on us. Stage 1 does not even need a server - a laptop with a CVMFS client mounts the validated software tree in minutes, having compiled nothing - and every later stage ends in one checkpoint command with an expected output, so you always know whether you have actually passed.
Section references inside the stages (§2, §8, §10.1) point into the dossier at ymnnpc.ndexr.io , which this path was written alongside.
There is no installer here, and that is not an omission.
An installer assumes a finished product with a fixed shape, which it
copies onto your machine. This is not that. What is being handed over is a
way of working
- so the honest form is a guided path rather
than a
curl | sh
. Nine stages, each one bringing a
repository from the archive to life inside your own environment, and each
ending in
one checkpoint command with an expected output
. A stage you have not passed is a stage you cannot skip, because the next
one depends on it and will fail later and more confusingly than it would
have failed here.
The assumption throughout is that
you and an agent do this
together
(§10.1).
CLAUDE-DEPLOY.md
in the
archive is addressed to an agent rather than to a person, which is why the
steps below are described rather than scripted: the runbook is the script,
the agent reads it, and you supply the judgement and the credentials.
The order is the dossier's own argument pointed at your machine. Stage 1 is mount, not install - a laptop with a CVMFS client gets a complete validated R toolchain in minutes, having compiled nothing and created no cloud resources. A reader who does only stage 1 has still watched the thesis work on their own hardware. Everything after it is about becoming a publisher instead of a consumer, and that is the point where you need a machine of your own.
Get the code
Unzip the archive into your home directory, or clone the repository. Either way it must land at ~/ndexr — every path in every runbook assumes that, and nothing later is worth debugging against a different root.
test -d ~/ndexr/src/domains && test -f ~/ndexr/LICENSE && echo readyYou have the source and a way to manage it. Nothing runs yet.
Mount, do not install
Install the CVMFS client from your distribution, point it at the software repository, and mount. This works on a laptop. You are not building anything and you are not creating any cloud resources — you are attaching to a signed, read-only software tree that already exists. Then load R from it.
ls /cvmfs/software.ndexr.io/ && module load R && R --versionThis is the whole thesis, proven on your own machine in minutes: a complete validated toolchain, compiled by nobody local, installed nowhere. If you stop here you have still gained something real.
Get credentials worth trusting
Read aws.ndexr.io — it is the cloud surface for the whole suite, and it is where instances, keys, security groups and DNS come from. Then create a role scoped to this job: the bucket prefix that will hold your tree, the hosted zone for the subdomain you are standing up, and nothing else. Configure the AWS CLI with it. Do not use console-grade keys, and do not hand console-grade keys to an agent.
aws sts get-caller-identityYou can now create things. The negative check matters as much as the positive one: the same credentials should fail to list buckets that are not yours to touch.
One server, turned on
Launch a single EC2 host and point a wildcard DNS record at it. Build the CLI, write your own .env, and deploy. INSTALL.md in the archive is the step-by-step; this stage is done when the platform answers on your own domain, serving your own first page.
curl -sI https://<your-first-domain> | head -1You are hosting. Everything above this line was consumption; everything below it requires a machine you control.
The build layer
Now, and not before, build something with EasyBuild. A build needs real cores and somewhere to put the result, which is why it comes after the server rather than before it. Start with one recipe from the easyconfigs in the archive.
readelf -l $(which R) | grep interpreterYou can produce the stack rather than only mount it. The checkpoint is the one from the dossier: if the interpreter is still the host's, the build did not do what you think it did.
Publish your own tree
Point CVMFS at your own S3 bucket as origin, and publish what you built as one transaction. Then mount it from a second machine — your laptop from stage 1 will do.
ls /cvmfs/<your-repo>/You have crossed from consumer to publisher. This is the point at which adding a machine stops being a software problem.
The package layer
Stand up the internal repository and bake its address into your R build, so every session resolves packages through one door. This needs the build layer, because the binaries it serves are built by it, for the exact toolchain asking.
Rscript -e 'install.packages("jsonlite"); cat(getOption("repos")[[1]])'Governed packages: qualify once, and everyone gets the qualified build.
Sessions in the browser
Deploy the portal and its launch forms. It needs a server to run on and a tree to serve the IDE from, which is why it is last of the capability stages rather than first — it is the visible part, but it is the part that depends on everything else.
open the portal, pick R and cores, click LaunchYour users have what the commercial suite is bought for.
Identity, and who may do what
Single sign-on, home directories, and the mapping from a person to a POSIX user on the compute side. Sketched here rather than specified: this is the stage that depends most on your existing organisation, and the least on anything in this archive.
sign in as a second, non-admin personThe platform belongs to an organisation rather than to you.
Not at “installed”. At the point where your environment has the same shape ours does: every capability one directory that is at once a repository, a live site, and the code that stands its subject up (§2), with an agent panel beside whichever one you are changing (§8). The stages above are how the repositories get there; the workflow is what they add up to.
Which is also why the checkpoints are commands rather than screenshots. At the end of this you are not running someone else's appliance - you are operating a platform whose every layer you can rebuild, and the checks are the same ones we run.
6 License and provenance
The platform source is licensed Apache-2.0 . The license text ships at the root of the archive, the generated README restates it, and the archive build hard-fails if the file is absent - it cannot ship unlicensed.
Provenance for every claim on this page lives in the two dossiers it condenses: what.ndexr.io for the distribution model, with numbered citations to primary sources, and ymnnpc.ndexr.io for the platform itself, whose archive is the same one served here. The pages are kept because an argument for open infrastructure should be checkable - and the archive is included because the argument is not checkable without it.