Installing and testing the MadAnalysis 5 particle physics platform on Win10 using the Windows Subsystem for Linux (WSL)

in #utopian-io6 years ago

Repository

https://github.com/BFuks/mad5-utopian-exercises

What Will I Learn?

MadAnalysis5 is a suite of software used to generate c/c++ code to analyze particle collider data. It was developed for Linux systems, but with some tweaks can be run using the Windows Subsystem for Linux (WSL, a.k.a. Bash on Ubuntu on Windows) available in Windows 10.

  • You will learn how to install MadAnalysis5 and its core dependencies under the WSL
  • You will learn how to generate and run the equivalent of 'hello world' to test the installation.

Requirements

  • An current installation (build 16215 or later) of Windows 10
  • Approximately 1 to 2 gigabytes of drive space, mainly for the WSL installation
  • An activated WSL with Ubuntu installed
    • Installing WSL itself is beyond the scope of this tutorial, as there are already detailed installation instructions from Microsoft and from third parties
    • This tutorial was specifically tested using Ubuntu 16.04.4 LTS, the user may check their version by typing lsb_release -a in the console.

Difficulty

  • Intermediate. The user does not need to be an expert using the command line, but must not be intimidated by it.

Tutorial Contents

There are three components to successfully run MadAnalysis5 using a fresh WSL:

  1. Installing relevant development tools
  2. Installing the software and its dependencies
  3. Testing the installation using the equivalent of 'Hello, world!' to analyze some Large Hadron Collider (LHC) data.

All of these steps will be performed within the Bash shell, which is available from the start menu or can be found by pressing the Windows key and searching for 'ubuntu'.

Preparing the WSL environment for development

A base install of Ubuntu under WSL will not include the necessary development tools for c/c++ (e.g. gcc and make). The simplest way to install these is to use the build-essential metapackage. The following three commands will update your package manager and install build-essential:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential

Installing MadAnalysis5 and its dependencies

Although MadAnalysis5 has the ability to install some of its dependencies, that route does not always work under WSL. It is advisable to follow the order of installation listed here.

Python 2.7

As installed under the WSL, Ubuntu does not have Python, which MadAnalysis5 depends on. You must first install Python itself and then set up the environment so that MadAnalysis5 can find it.

sudo apt install python2.7
sudo ln -s /usr/bin/python2.7 /usr/bin/python

Note: This method assumes that a custom WSL instance is being set up for MadAnalysis5. Although creating a symbolic link to python2.7 is the simplest solution, it can cause issues in environments where other projects requiring Python3 are being run. Setting up a Python virtualenv is probably a workable solution in that case, but is beyond the scope of this tutorial.

Installing the ROOT dependency

In this section, ROOT does not refer to the Unix term, but rather the CERN particle physics data analysis package.

Although other versions of ROOT are likely to work, this tutorial was developed using release 6.10.08. The binary release for the appropriate Ubuntu version (here, 16.xx) can be installed using the following commands:

cd ~
curl https://root.cern.ch/download/root_v6.10.08.Linux-ubuntu16-x86_64-gcc5.4.tar.gz | tar xvz
cd root/bin/
source thisroot.sh
cd ~

Note: To avoid having to source thisroot.sh for every new WSL instance, the user is advised to update their .bashrc to do so.

Installing madanalysis5

MadAnalysis5 can be installed similarly to ROOT:

wget https://launchpad.net/madanalysis5/trunk/v1.6/+download/ma5_v1.6.tgz
tar -xvzf ma5_v1.6.tgz

After unarchiving, run ./bin/ma5.

ma5_install.png

The screen displayed should be similar to above and list mostly disabled dependencies, but indicate that ROOT has been found. There should be a prompt asking the number of cores to be configured for. Choose the default number of cores and let the program run. It should list the component number and total number of components it is preparing, as below:

ma5_cores.png

Installing delphes

This is one of the major installation steps which differs from installing MadAnalysis5 under a dedicated Linux environment. Instead of using the MadAnalysis5 prompt to 'install delphes', the user should clone the delphes git repository into the appropraite location and build from source:

From the madanalysis5 directory

cd tools
git clone https://github.com/delphes/delphes.git
cd delphes
make

Compilation should take approximately 30 to 60 minutes on a contemporary computer (e.g. an i7-3xxx with 8 gb RAM was sufficient for this tutorial).

MadAnalysis5 should be run again (~/madanalysis/bin/ma5) to check that delphes was found and to reconfigure the components.

Installing PAD

PAD can be installed within the ma5 prompt as per the the original instructions :

install pad

Generating the equivalent of 'Hello, world!' to test the installation.

MadAnalysis5 serves as a code generator to create programs for specific analyses of data. The code resulting from this section of tutorial serves as both a test of the installation and as a completed Task 1a for the associated Utopian project. The user is advised to work through the example themselves, but a full working version is available on github as well as at the primary github repository for the associated task.

Code generation

As in the associated task, the first step is to tell MadAnalysis5 where to create the skeleton code:

./bin/ma5 -E test_folder test_analysis

For the generated code to compile under the WSL an additional step is needed. This is because the environment variables referenced in the generated Build/setup.sh script, when expanded, contain spaces. This is a result of Windows naming conventions and the conditionals used in the scrip, as generated, cannot handle them.

To fix the script, edit Build/setup.sh in in vim or your text editor of choice, and surround all environment variables in conditional statements with double quotes .

For example:

if [[ $MA5_BASE && $PATH && $LD_LIBRARY_PATH ]]; then

should become

if [[ "$MA5_BASE" && "$PATH" && "$LD_LIBRARY_PATH" ]]; then

(User wishing to use a GUI editor such as gedit will need to setup their system to use X ).

Downloading example data

It is generally best to treat the WSL file system and the host windows file system as two separate entities. The best way to download the example data file is then to use wget, such as we did with the ROOT binaries. This is also a good time to create the input file read by the generated program.

cd test_folder\test_analysis\Input
wget http://madanalysis.irmp.ucl.ac.be/raw-attachment/wiki/MA5PublicSandBox/tth_aa.root
realpath tth_aa.root > tth_aa.list

Running the software

The generated code, after the modifications above, will run, but will not produce the output required by Task 1a. The user is encouraged to figure out how to modify test_folder/Build/SampleAnalyzer/User/Analyzer/test_analysis.cpp to produce the desired output. However, one approach is published at the github repository associated with this tutorial, should they become stuck.

For completeness, the rest of the build process is the same as under native Ubuntu and should not take more than a few minutes.

cd test_folder/Build
source setup.sh
make
./MadAnalysis5job ../Input/tth_aa.list

The expected output will differ slightly, based on how the user chose to display the output, but should appear similar to below:

ma5_run.png

Proof of Work Done

The modified generated code used in this tutorial will be available after pulling at the primary github repository for the associated task https://github.com/BFuks/mad5-utopian-exercises and is currently available at the author's forked repository at https://github.com/effofex/mad5-utopian-exercises/blob/master/ex1a_effofex.cpp

Sort:  

Thanks a million for this! This will allow windows users to use MadAnalysis 5!

Of course, I can't test all of this. But I am sure that if there are issues, we will find out during the course of the project.

I am planning to add a link to your post here. Please provide me a way to credit you. I can use your real name, nickname, etc... Just let me know! :)

I appreciate it, but I'd prefer to keep @effofex and my real name separate for now, until I get a better feel for my field's response to steemit. Feel free to credit either @effofex or anonymous.

edit: I would also not turn down a beer if I'm ever in your neck of the woods.

Woo, look at me, contributing to science and stuff! :)

Congratulations ;)

Hey @effofex
Thanks for contributing on Utopian.
Congratulations! Your contribution was Staff Picked to receive a maximum vote for the tutorials category on Utopian for being of significant value to the project and the open source community.

We’re already looking forward to your next contribution!

Contributing on Utopian
Learn how to contribute on our website or by watching this tutorial on Youtube.

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!

Oh wow! I saw this earlier today and still am not quite sure what to say. I'm kind of blown away. It's wonderful to see y'all liked my tutorial, I was a bit hesitant to put it out there - thanks to everyone who read drafts and gave me feedback.

For anyone else reading this who has been interested in contributing to utopian, jump on in, they're nice.

Thank you for the beautiful work on this tutorial.

For future similar/related tutorials, just make sure to use the following github repository instead https://github.com/BFuks/madanalysis-utopian

Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]

Oh, very cool that y'all are transparent on the evaluation. That sort of feedback is really helpful.

Interesting stuff.. wud be nice to see this being done on highly efficient quantum computers one day..sooner than later.

I'm not an expert in the internals of the software (yet), but it's mainly data analysis of existing sets, rather than simulating the actual events. Computing time certainly wasn't an issue for the first task.

The problem is the simulation of the detector. Running delphes. This takes ages.

I want to also parallelize the entire code, which is high on the to-do list. But this development is not the priority at the moment, mainly because of a lack of manpower. We are two developers overwhelmed by many other things. You can guess the rest ;)

What sort of parallelization are you looking at?

We are two developers overwhelmed by many other things. You can guess the rest ;)

I've been there. Am there, actually.

What sort of parallelization are you looking at?

We have machines with lots of CPUs. This is what I am targeting. I have vague ideas but no time :)

@effofex could you equate the Windows Subsystem for Linux on Windows 10 as a build in virtual machine? I have recently installed windows 10 with a dual boot of Linux Mint xfce. My chipset doesnt allow for virtualization so i gave up on it a long time ago . Is this a new feature? Im assuming the ubuntu code is still relevant on this version. Awesome write up either way!!!

I'm glad you liked it.
Once you get WSL setup, it's great for a lightweight Unix environment - particularly if you don't need to deal with X or sharing files between the subsystem and host (though both are possible). In your situation, I'd find it much more bearable than dual-booting, which I came to loathe.

Running a VM will give you the full Linux experience with a minimum of hassles, but like you said, not all hardware is happy with that. In my own situation, I started using WSL on my professional box because our IT department is super restrictive on what OSs we can virtualize, and some software I needed required a specific distro.

For those wondering, I live in a windows host because a) 95% of my colleagues use the windows environment and, especially as a junior member, it's easier for me to live in their environment and b) roughly half the software I use professionally has only a windows version (of course, the other half really wants to live in *nix, hence the WSL).

Coin Marketplace

STEEM 0.21
TRX 0.20
JST 0.034
BTC 98129.50
ETH 3322.67
USDT 1.00
SBD 3.05